US20170109338A1 - System and method for interacting in layers in channels over the display of a resource by another application - Google Patents
System and method for interacting in layers in channels over the display of a resource by another application Download PDFInfo
- Publication number
- US20170109338A1 US20170109338A1 US15/389,896 US201615389896A US2017109338A1 US 20170109338 A1 US20170109338 A1 US 20170109338A1 US 201615389896 A US201615389896 A US 201615389896A US 2017109338 A1 US2017109338 A1 US 2017109338A1
- Authority
- US
- United States
- Prior art keywords
- user
- channel
- user layer
- display
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000000694 effects Effects 0.000 claims description 28
- 238000004590 computer program Methods 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 9
- 230000014509 gene expression Effects 0.000 claims description 2
- 238000012360 testing method Methods 0.000 claims description 2
- 108091006146 Channels Proteins 0.000 description 85
- 230000008569 process Effects 0.000 description 15
- 230000003993 interaction Effects 0.000 description 7
- 230000002093 peripheral effect Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 5
- 230000003213 activating effect Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000013500 data storage Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000003466 anti-cipated effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/169—Annotation, e.g. comment data or footnotes
-
- G06F17/241—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/9038—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
-
- G06F17/30876—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
- G09G5/37—Details of the operation on graphic patterns
- G09G5/377—Details of the operation on graphic patterns for mixing or overlaying two or more graphic patterns
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2320/00—Control of display operating conditions
- G09G2320/06—Adjustment of display parameters
- G09G2320/0693—Calibration of display systems
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2340/00—Aspects of display data processing
- G09G2340/12—Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2370/00—Aspects of data communication
- G09G2370/02—Networking aspects
- G09G2370/022—Centralised management of display operation, e.g. in a server instead of locally
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2370/00—Aspects of data communication
- G09G2370/02—Networking aspects
- G09G2370/027—Arrangements and methods specific for the display of internet documents
Definitions
- the invention relates generally to graphical user interfaces enabling users to automate processing of and action over electronic information resources generated by underlying software applications and graphically showing the results in a visual overlay.
- a single user's activities which may be one or more of:
- a user may switch from one channel to another and send one or more annotations to a different channel from a currently active channel.
- a plurality of users may participate in a channel to create one or more annotations that are shared between them within said channel.
- channels may be established that give some users restricted access, such as view only permissions.
- the system provides access to the annotations for the purposes of creating analyses of individual user activities, group user activities, activities within a channel, activities across a group of channels, real-time trending information by activity type and electronic information resource or combinations thereof.
- FIG. 1 shows user layer software installed on a user's computing device connected to a user display and a communications network according to an embodiment of the invention.
- FIG. 2 is a high level view process flow of user layer software's operation from a user's perspective showing that after selecting to use user layer software they may edit user layer configuration data to produce user layer displays and search said layer configuration data and the history produced by its operation, according to an embodiment of the invention.
- FIG. 3 shows the relationship between a user display and one or more display windows containing output from retrieval and display software and a user layer display produced by user layer software comprising one or more annotations of that output, according to an embodiment of the invention.
- FIG. 4 illustrates a process flow of user layer software for one or more sets of user layer configuration data, which sets are applied to one or more display windows, according to an embodiment of the invention.
- FIG. 5 shows the relationship between a user and a user computing device running user layer software and information resources retrieved from a resource server via a communications network, according to an embodiment of the invention.
- FIG. 6 is similar to FIG. 5 with the addition of a layer server attached to the communications network to store information related to user layers remotely from a user's computing device, according to an embodiment of the invention.
- FIG. 7 illustrates user layer software being used to search and edit user layer configuration data and user layer history data, according to an embodiment of the invention.
- FIG. 8 is a schematic of the data flows in user layer configuration data being processed by user layer software to produce a graphic overlay, according to an embodiment of the invention.
- FIG. 9 exemplifies a result of processing a set of user layer configuration data that screens out items not meeting a criterion, according to an embodiment of the invention.
- FIG. 10 illustrates examples of user layer display where the user layer software and user layer configuration data has resulted in notifications of objects also found on a set of reference sites defined in the user layer configuration data, according to an embodiment of the invention.
- FIG. 11 illustrates an effect of configuring and combining two or more sets of user layer configuration data, according to an embodiment of the invention.
- FIG. 12 is a schematic data flow diagram showing how multiple sets of user layer configuration data may interact with each other through the results from one layer being an input to one or more other layers.
- FIG. 13 illustrates the result of configuring the system to annotate resources that have previously been viewed in a user's display window.
- FIG. 14 illustrates the result of configuring the system to annotate text that appears in two sets of retrieved information.
- FIG. 1 illustrates a channel carousel and expanded channel thumbnail detail according to an embodiment of the invention.
- FIG. 15 illustrates a channel user interface components according to an embodiment of the invention.
- FIG. 16 illustrates an Internet browser display view with the channel header above the browser
- FIG. 17 illustrates a channel member detail according to an embodiment of the invention.
- FIG. 18 illustrates a channel annotation list according to an embodiment of the invention.
- FIG. 19 illustrates chat and sticker annotations in overlays according to an embodiment of the invention.
- FIG. 20 illustrates single and plural user drawing annotations overlaid on both web pages and a user-generated photo image according to an embodiment of the invention.
- FIG. 21 illustrates a process flow method for user interactions with the channel user interface illustrated at FIG. 15 .
- FIG. 22 illustrates a process flow method for user interactions associated with an Internet browser view and overlaying annotations.
- FIG. 23 illustrates a process flow method for user interactions associated with creating a new overlaid annotation and/or channel for overlaid annotations.
- FIG. 24 illustrates a process flow method for user interactions associated with managing interactions with other users.
- FIG. 25 illustrates a process flow method for user interactions associated with responding to annotations made by other users.
- a user using a computing device 101 comprises a processor, program memory and data storage memory together with input/output hardware that connects the computing device to peripheral devices and a communications network 106 .
- Peripheral devices to the computing device may comprise one or more of a user display 104 , keyboard, pointing mechanism and other peripherals may include voice input, gesture recognition and motion detection devices.
- the user operates retrieval and display software 102 loaded onto the computing device 101 that initiates information requests and receives responses over a communications network 106 to display retrieved information 103 on the a user display 104 .
- the user loads user layer software 105 to process retrieved information 108 according to the specifications contained in one or more sets of user layer configuration data 107 to generate one or more user layer displays 302 .
- a user may control the behavior of the user layer software 105 through a main task interface for which a control flow chart is illustrated at FIG. 2 .
- a user may decide whether or not to use user layer software 105 and if they do they may create and edit one or more sets of user layer configuration data 107 and control which of said sets of user layer configuration data are active for the generation of user layer displays 302 .
- each user layer's behavior resulting from the application of user layer configuration data 107 to retrieved information 108 by user layer software 105 , is configurable using parameters and instructions supplied by a user 501 that may comprise comparison tests against one or more of keywords, strings of words or phrases, logic sequences, language expressions, paradigms, object recognition and target outcomes as further illustrated at FIG. 7 .
- user layer software 105 processes one or more sets of user layer configuration data 107 to compute one or more graphical overlays 303 for display in context with associated retrieved information 108 as rendered by retrieval and display software 102 on a user display 104 within a user display window 301 and this is illustrated in FIG. 3 .
- a graphical overlay 303 may be separately displayable, even to the extent of being shown in a separate user display window 301 than retrieved information 108 .
- rendered retrieved information 108 plus one or more graphical overlays 303 form a user layer display 302 .
- FIG. 4 is a flow diagram of core functions performed by the user layer software 105 on an active set of user layer configuration data 107 .
- user layer configuration data 107 contains parameters and instructions that require data in addition to retrieved information 103 the user layer software 105 executes actions to access that additional data which may reside in a user layer history 110 or, as in FIG. 5 , on a storage device 503 attached to a resource server accessed via a communications network 106 .
- FIG. 5 shows a preferred embodiment where a user's 501 computing device 101 is connected to a communications network 106 via which a user may cause the retrieval and display software 102 running on the device to make requests to information resource servers 502 which in turn accesses information storage devices 503 to access information resources 504 which are sent as a response to the computing device 101 over the communications network where they become retrieved information 108 for rendering by the retrieval and display software 102 on the user display 104 and for any processing by the user layer software 105 .
- user layer configuration data 107 and user layer history 110 information may also be stored on a layer server 601 as illustrated in FIG. 6 .
- a user 501 may thus access their user layer configuration data 107 and said histories thereof using any computing device 101 from which they are able to sign in to said layer server.
- user layer software 105 provides a user 501 the ability to search and optionally edit one or more of user layer configuration data 107 and user layer history 110 data.
- user layer software 105 allows for the naming of each set of user layer configuration data 107 , illustrated in FIG. 7 , for easy reference and for sets of user layer configuration data 107 to be grouped together to form a layer group and for each group to be named and activated together
- a user 501 may combine two or more sets of user layer configuration data 107 through cross-reference information that defines one or more connections between such sets of user layer configuration data 107 including the precedence of their processing and the condition activating said cross-reference; for example, one said set of user layer configuration data 107 may highlight duplicate items within the retrieved information 108 , one said set may search for lowest price information for non-duplicate items by accessing additional retrieved information 108 and another said set may compare the results with prior results stored in the user layer history 110 , all resulting a graphical overlay 303 annotating unique items with the current lowest price for each and prior lowest prices.
- a set of user layer configuration data 801 to be processed is loaded 802 into data storage memory 108 for user layer configuration data 107 in the user's computing device 101 where it is processed by user layer software 105 against retrieved information 103 .
- a graphical overlay 303 if any results, is output 803 by user layer software 105 as part of a user layer display 302 in context with the rendered retrieved information 103 .
- FIG. 8 provides a schematic representation of the above steps.
- use layer software 105 stores a user layer history 110 that retains retrieved information 108 and results of its processing said retrieved information 108 according to user layer configuration data 107 .
- An illustration of including user layer history 110 to produce a graphical overlay 303 is contained in FIG. 9 , which is identical to FIG. 8 except that the user layer configuration data 107 has been edited to trigger a “Lower price” comparison rather than “Similar”.
- a user is conducting a search of the Internet using retrieval and display software 102 and wishes to know which elements or group of elements in resulting retrieved information 108 were already returned by their prior searches, such conditions would be specified by the user as part of user layer configuration data 108 which user layer software 105 would process to detect such conditions and generate a graphical overlay 303 to assist the user in distinguishing new results from prior duplicates.
- sets of user configuration layer data 107 are configurable by a user to automatically cause actions.
- an automated process for finding matching data elements across a range of information resources 504 may be organized as follows: user layer configuration data 107 causes the user layer software 105 to first search a web page stored as retrieved information 108 for specific tagged elements by invoking the standard javascript function Element.gelElementsByTagName( )to return a list of all elements in said retrieved information 108 matching a tag name specified in user layer configuration data 107 .
- said user layer software 105 initiates retrieval and a search of said additional retrieved information 108 from said other information resources 504 to identify elements or groups of elements that match said tag name and generate a graphical overlay 303 highlighting elements for which matches were found and any ancillary information specified in the user layer configuration data 107 such as price and color.
- FIG. 10 the screen shot shows a graphical overlay 303 annotating similar results from said other information resources 504 within a user layer display 302 .
- methods of annotation in graphical overlays 303 are specified in user layer configuration data 107 as illustrated at FIG. 7 .
- a user may define the color, style, positioning and other parameters of an annotation to distinguish between them and suit their preference either through public domain javascript methods such as window.alert (“annotation text”); to create a temporary pop-up window and rect(x, y, widthinpixels, heightinpixels); to draw a rectangle on top or proprietary or patented annotation methods.
- user configuration data 107 is used by user layer software 105 to display different distinguishing marks, style or modes of annotation in order to differentiate between results in a graphical overlay 303 .
- the preferred embodiment includes logic within the user layer software 105 to identify which subset of retrieved information 103 information has been visibly displayed within the display window 301 rendered by the retrieval and display software 102 .
- the preferred embodiment accomplishes this by capturing a user's viewing position by means of positional parameters using one or more methods anticipated in claims 2 through 8 of patent application Ser. No. 13/938,122 filed Jul. 9, 2013 and storing said positional parameters as part of user layer history 110 data.
- User layer configuration data 107 may, therefore, additionally determine whether the user layer software 105 acts or acted on all retrieved information 108 or only retrieved information 108 that was visible to the user using viewing position information collected within user layer history data 110 as illustrated in FIG. 7 .
- the preferred embodiment of user layer software 105 may further assist the user 501 by allowing them to define how multiple sets of user layer configuration data 107 are applied to retrieved information 108 .
- This embodiment allows a user 501 to specify, for example, sets of user layer configuration data 107 that should be applied on retrieved information 108 by user layer software 105 in series like successive filters, whether they are applied in parallel and whether iteration is applied.
- user layer software 105 will produce a user display 104 annotating items that meet the conditions in all sets of user layer configuration data 107 whereas, by applying sets of user layer configuration data 107 in parallel, user layer software 105 will produce a user display 104 annotating items that meet the conditions defined within said sets of user layer configuration data 107 and when applying sets of user layer configuration data 107 iteratively, user layer software 105 will produce a user display 104 that is updated when underlying conditions change.
- This preferred embodiment of combining application of multiple sets of user configuration data 107 also solves for the user 501 problem of identifying similar but unique elements in a single graphical overlay 303 within a user layer display 302 .
- FIG. 11 A example of combining user layers to produce a graphical overlay 303 showing how many comparable items were found in a reference database is illustrated at the block diagram at FIG. 11 and an example block diagram of multiple user layers that together seek comparable items, seek pricing for them and displaying changes against historical results in a graphical overlay 303 , is shown at FIG. 12 .
- user layer software has access to a user's viewing history as an information resource 504 and user layer history 101 from which is produced a graphical overlay 303 that annotates a portion or portions of previously retrieved information 108 already seen by the user in the manner exemplified in FIG. 13 .
- user layer configuration data 107 is configurable by the a user to access one or more reference data sources as an information resource 504 such that elements within retrieved information 108 that also occur in said reference data are highlighted within a graphical overlay 303 . Such a configuration is illustrated at FIG. 14 .
- user layer software 105 allows a user 501 to suppress a graphical overlay 303 , thereby permitting them to see underlying rendered retrieved information 108 uncluttered by graphical overlay(s) 303 .
- the preferred embodiment of user layer software 105 assists users 501 in organizing their work by adding category meta-data, see example at FIG. 7 , to user layer configuration data 107 .
- the preferred embodiment of user layer software 105 provides users 501 the ability to share at least one of user layer configuration data 107 , graphical overlay 303 , user layer display 302 and user layer history 110 with other users by stipulating so as illustrated in FIG. 7 .
- an investment by one user in creating a user layer may be transferred for reuse to another user with the potential for monetary exchange in return for such transfer or reuse.
- an apparatus for employing user layer software includes a display, an input/output hardware unit and a program memory coupled to a processor.
- Input/output hardware unit is capable of exchanging data with a network and data associated electronic resources accessible over a network.
- Program memory stores program instructions and a processor executes program instructions to (i) produce output from one or more software applications, (ii) execute user layer software for any user layer configuration data that has been enabled for those software applications, and (iii) display a graphical overlay derived from the output of user layer software placed in context with the output from said software applications in accordance with user layer configuration data.
- a system for employing user layer software includes a database, an input/output hardware unit, and a program memory coupled to a processor.
- Input/output hardware unit is capable of exchanging data with a network.
- Program memory stores program instructions and the a processor is capable of executing program instructions to: (i) store and retrieve user layer configuration data from the database (ii) store and retrieve from the database results from the user layer software having been executed using user layer configuration data iii) initiate exchange with other users of user layer configuration data, iv) initiate exchange with another user of results from user layer software having been executed using their user layer configuration data.
- a computer program product includes computer program logic stored in a tangible medium, wherein computer program logic executes program instructions to (i) execute user layer software for any user layer configuration data that has been enabled for a software application, and (iii) display a graphical overlay derived from the output of user layer software over the output from said software applications in accordance with user layer configuration data.
- a computer program product includes computer program logic stored in a tangible medium, wherein computer program logic includes logic for causing a computer program to: (i) store and retrieve user layer configuration data from the database (ii) store and retrieve from the database the results from user layer software having been executed using user layer configuration data iii) initiate exchange with other users of user layer configuration data, iv) initiate exchange with another user of the results from user layer software having been executed using their user layer configuration data.
- a user using a computing device comprises a processor, program memory and data storage memory together with input/output hardware that connects a computing device to peripheral devices and a communications network.
- Peripheral devices to the a computing device may comprise one or more of a user display, keyboard, pointing mechanism and other peripherals may include voice input, gesture recognition and motion detection devices.
- a user operates retrieval and display software loaded onto the a computing device that initiates information requests and receives responses over a communications network to display retrieved information on the a user display.
- a user 105 loads user layer software to process retrieved information in relation to the user's display location within the retrieved information to display overlay information.
- a user 501 may organize their electronic information resource 1604 viewing activities into channels 1501 or expeditions as illustrated in FIG. 15 , each of which is represented by a channel thumbnail 1502 comprising one or more of a channel image 1503 , a channel name 1504 , last channel activity information 1505 and channel user role information 1506 .
- channels thumbnails 1502 may be in channel categories 1507 and may be searched through a channel search box 1508 to allow for the case, for example, of finding all channels with a particular user as a member.
- a user 501 once a user 501 has entered a channel they will be presented a channel screen as illustrated at FIG. 16 comprising one or more of:
- a channel header 1601 containing one or more of a channel name 1504 , channel user role information 1506 , channel category 1507 , channel member information 1602 which allows access to details of members of the a channel in the case there is a plurality of users with access to the a channel, a channel activity indicator 1603 which allows access to the activity in the a channel which may be spread over more than one electronic information resource and in some embodiments additional information including the number of users in a channel, the number of resources being viewed within a channel or expedition or other information related to activity in the a channel;
- a resource display area 1604 showing an electronic information resource currently being viewed and optionally containing a sharing control element 1605 allowing a current user to select a specific display location for one or more of a new chat thread, sticker, drawing including a user generated drawing or insertion of other user generated content such as a photo image;
- a resource browsing control area 1606 that permits a user to search for and select an electronic information resource to be showing in the a resource display area 1604 ;
- an annotation notification area 1607 providing information about activities not always visible within a current electronic information resource display area 1604 for the a channel 1501 currently in focus.
- activating channel member information 1602 which allows access to details of members of the a channel in the form of a channel member list 1701 .
- activating a channel activity indicator 1603 allows access to activity and any resulting annotations in the a channel which may be spread over more than one electronic information resource in the form of a channel activity list 1801 .
- activating a sharing control element 1605 permits a user to create one or more of:
- chat thread attached to a display location which may be displayed in an overlay to a current electronic information resource 1604 as either a compact or closed chat thread indicator 1701 or an open chat thread 1702 ;
- a sticker attached to a display location 1703 in an overlay to a current electronic information resource 1604 .
- activating the a sharing control element 1605 permits a user 501 to create a drawing 2001 as an overlay to an electronic information resource 1604 .
- those users can, in some embodiments providing they have access to the channel, separately and contemporaneously create drawings in the same channel as an overlay to a same electronic information resource as additionally exemplified as drawing from user A at 2001 and drawing from user B at 2902 .
- a current electronic information resource 1604 may also include user generated content such as photo images exemplified here at 2003 and with simultaneously drawn lines from a plurality of users 501 exemplified as drawing from user A at 2004 and drawing from user B at 2005 .
- a user 501 interacts with a channel user interface displaying channels 1501 using a method illustrated in FIG. 21 which allows users 501 to select from the at least one channel using its thumbnail 1502 .
- a user may press a menu button in 2101 resulting in a user having access to a profile and settings menu or menus. There, the a user may edit their profile and settings by selecting from a submenu or entering information.
- a user may further select an annotation notification 2102 , which allows a user to view and act on annotations by other users.
- FIG. 21 also illustrates that by selecting the a channel name 2013 a user is able to view channel detail information including which other users are members of the a channel.
- selecting an unread bell 2014 provides a user with a list of all annotation notifications that they have not yet read and a channel search function 2105 enables a user to select a set of channels fitting a set of search characteristics they enter. Subsequent processes may include those depicted in FIGS. 22 through 25 .
- electronic information resources 1604 are displayed in an Internet browser view integrated with overlaid annotations e.g. 1901 , 1902 and 1903 and any annotation notifications using the method illustrated at FIG. 22 .
- the approach illustrated allows each user 501 to browse any electronic information resource 1604 independently of the activities of any other user 501 in the a current channel 1501 or any other channel 1501 .
- annotations e.g. 1901 , 1902 and 1903 are initiated by users 501 using a share button 1605 indicator which is dragged and dropped to a display location within a current electronic information resource 1604 viewing method as indicated in a drag and drop operation at 2201 .
- FIG. 22 are also shown methods that are added to standard browser operations 2204 which include accessing annotation notifications 2202 through annotation notification elements 1607 and the ability to modify how overlaid annotations e.g. 1901 , 1902 and 1903 are displayed using selector 2203 .
- FIG. 22 also illustrates a method of accessing information relevant to a channel 1501 through a channel header 1601 by selecting action elements 2205 .
- a user 501 which includes a joining user, can initiate new overlaid annotations using the method illustrated at FIG. 23 which also permits an overlaid annotation to be used to start new channels 1501 .
- FIG. 23 illustrates a method that can be used following selection of the a drag and drop operation 2201 such that a user may choose one or more options such as sending an overlaid annotation to a different or new channel 2301 , select an annotation type such as a doodle 2302 , sticker 2303 or text 2304 , repositioning a draft overlaid annotation 2305 to a different viewing position and communicating a finalized annotation 2306 through a chosen channel 1501 .
- An annotation e.g. 1901 , 1902 and 1903 information is communicated to other users through a communications network to which users are connected as illustrated in FIG. 5 and FIG. 6 .
- Annotations may be stored by all users 501 or stored for reference on one or more servers 501 .
- a user 501 can manage their interactions with other users including which other users 501 have access to channels 1501 they created or are leading through either a main menu, a channel thumbnail 1502 or a channel header 1601 through a member detail process as illustrated at FIG. 24 .
- FIG. 24 shows a method in which members may be added or modified 2401 whether in a user's contact list or not, in which the a current user may select a specific member 2402 to see more detail and optionally jump to that member's viewing position 2403 .
- FIG. 24 also illustrates the ability to block or suppress 2404 overlay annotations and associated annotation notifications.
- users 501 that are members of a channel 1501 may receive notifications of annotations in channels 1501 they are members of and can view and respond to those annotations using the method illustrated in FIG. 25 .
- a user may access annotation notifications in the form of a channel list 2501 or an unread notification list 2502 and select to jump 2503 to the a browser view of that annotation e.g. 1901 , 1902 and 1903 or to make a quick reply 2504 directly from an annotation notification area concerned.
- a computer of mobile device includes a display, a processor, memory and input/output devices, such as a touch screen, buttons, keys, microphone, and speakers and a network interface for coupling the computer electrically, wirelessly or optically to at least one network, such as Wi-Fi network, a LAN, WAN or the Internet.
- the processor is coupled to the display, memory, input/output devices and the Internet and executes program instructions stored in the memory to perform various functions described herein.
- the display includes a plurality of channels and other information as shown in FIGS. 15-20 .
- the memory may store an application or app which allows users to login or logout, which information may be stored on a remote server.
- Channels shown on the display may be created when one or more users use the application to access content, for example on the Internet or another network, and the number of users in the channel can grow as users invite other users of the app or other users join a channel, who are permitted to do so by the app or by friendship status among one or more users.
- Data associated with a user's browsing within the app and/or channel within the app is stored to facilitate the collection and sharing of user activities described herein.
- a user may sign in, view channels he or she is permitted to join, join channels, receive notifications of the activities of other users, and select notifications and be transported to electronic resources that are annotated by other users in the channel based on various user activities as described herein.
- Users may also access content while affiliated with one or more channels and perform activities that are pushed out to other users associated with or within at least one channel in the form of notifications or stored data on the user's browsing activity and annotations.
- the app itself may also provide statistics relating to the channels, update icons corresponding to channels based on user activity and provide other information to facilitate enabling users to engage and annotate content and perform activities around content in a channel.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Business, Economics & Management (AREA)
- Finance (AREA)
- Strategic Management (AREA)
- Accounting & Taxation (AREA)
- Development Economics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Computer Hardware Design (AREA)
- Entrepreneurship & Innovation (AREA)
- Game Theory and Decision Science (AREA)
- Economics (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- General Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- User Interface Of Digital Computer (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
- The invention relates generally to graphical user interfaces enabling users to automate processing of and action over electronic information resources generated by underlying software applications and graphically showing the results in a visual overlay.
- Users employ software applications to search for, view information resources. These individual software applications have varying capabilities for automating user actions to finding, filtering, analyzing and updating the information resources.
- Without automated support for processing results displayed by the underlying software application, achieving desired objectives can be very time consuming and mentally taxing for users, for example requiring them to copy reference identifiers, mentally memorize and compare displayed results to satisfy a set of conditions they hold in their mind. Furthermore, when undertaking numerous and/or complex tasks users experience cognitive stress which can be relieved by having a system that a) enables them to organize activities and tasks within distinct channels and b) remembers for them the display location context of marks that they place over that information. The benefits of these capabilities include that, rather than having to bookmark web pages, take screenshots, or maintain a separate document with links and comments, these activities are captured as part of the task process with no special user effort.
- Furthermore, users need automated support regardless of the type of computing device or several devices that they may be using, where each may have different screen sizes and orientations.
- According to an embodiment of the present invention, a single user's activities, which may be one or more of:
- examining a plurality of electronic information resources;
- attaching a chat thread to a viewing position within an electronic information resource;
- placing a sticker at a viewing position within an electronic information resource;
- attaching a user-generated drawing to a viewing position within an electronic information resource;
- injecting user-generated content such as photo images as new electronic information resources;
- or data resulting from such activities are automatically associated as annotations overlaid at their designated display location in a channel which may be determined by the user.
- According to an embodiment of the present invention, a user may switch from one channel to another and send one or more annotations to a different channel from a currently active channel.
- According to an embodiment of the present invention, a plurality of users may participate in a channel to create one or more annotations that are shared between them within said channel.
- According to an embodiment of the present invention, channels may be established that give some users restricted access, such as view only permissions.
- According to an embodiment of the present invention, the system provides access to the annotations for the purposes of creating analyses of individual user activities, group user activities, activities within a channel, activities across a group of channels, real-time trending information by activity type and electronic information resource or combinations thereof.
- The foregoing features and advantages are illustrative only and will be more fully appreciated in view of the appended figures described below.
-
FIG. 1 shows user layer software installed on a user's computing device connected to a user display and a communications network according to an embodiment of the invention. -
FIG. 2 is a high level view process flow of user layer software's operation from a user's perspective showing that after selecting to use user layer software they may edit user layer configuration data to produce user layer displays and search said layer configuration data and the history produced by its operation, according to an embodiment of the invention. -
FIG. 3 shows the relationship between a user display and one or more display windows containing output from retrieval and display software and a user layer display produced by user layer software comprising one or more annotations of that output, according to an embodiment of the invention. -
FIG. 4 illustrates a process flow of user layer software for one or more sets of user layer configuration data, which sets are applied to one or more display windows, according to an embodiment of the invention. -
FIG. 5 shows the relationship between a user and a user computing device running user layer software and information resources retrieved from a resource server via a communications network, according to an embodiment of the invention. -
FIG. 6 is similar toFIG. 5 with the addition of a layer server attached to the communications network to store information related to user layers remotely from a user's computing device, according to an embodiment of the invention. -
FIG. 7 illustrates user layer software being used to search and edit user layer configuration data and user layer history data, according to an embodiment of the invention. -
FIG. 8 is a schematic of the data flows in user layer configuration data being processed by user layer software to produce a graphic overlay, according to an embodiment of the invention. -
FIG. 9 exemplifies a result of processing a set of user layer configuration data that screens out items not meeting a criterion, according to an embodiment of the invention. -
FIG. 10 illustrates examples of user layer display where the user layer software and user layer configuration data has resulted in notifications of objects also found on a set of reference sites defined in the user layer configuration data, according to an embodiment of the invention. -
FIG. 11 illustrates an effect of configuring and combining two or more sets of user layer configuration data, according to an embodiment of the invention. -
FIG. 12 is a schematic data flow diagram showing how multiple sets of user layer configuration data may interact with each other through the results from one layer being an input to one or more other layers. -
FIG. 13 illustrates the result of configuring the system to annotate resources that have previously been viewed in a user's display window. -
FIG. 14 illustrates the result of configuring the system to annotate text that appears in two sets of retrieved information.FIG. 1 illustrates a channel carousel and expanded channel thumbnail detail according to an embodiment of the invention. -
FIG. 15 illustrates a channel user interface components according to an embodiment of the invention. -
FIG. 16 illustrates an Internet browser display view with the channel header above the browser -
FIG. 17 illustrates a channel member detail according to an embodiment of the invention. -
FIG. 18 illustrates a channel annotation list according to an embodiment of the invention. -
FIG. 19 illustrates chat and sticker annotations in overlays according to an embodiment of the invention. -
FIG. 20 illustrates single and plural user drawing annotations overlaid on both web pages and a user-generated photo image according to an embodiment of the invention. -
FIG. 21 illustrates a process flow method for user interactions with the channel user interface illustrated atFIG. 15 . -
FIG. 22 illustrates a process flow method for user interactions associated with an Internet browser view and overlaying annotations. -
FIG. 23 illustrates a process flow method for user interactions associated with creating a new overlaid annotation and/or channel for overlaid annotations. -
FIG. 24 illustrates a process flow method for user interactions associated with managing interactions with other users. -
FIG. 25 illustrates a process flow method for user interactions associated with responding to annotations made by other users. - This is a continuation and priority is claimed back to U.S. patent application Ser. No. 13/938,122 SYSTEM AND METHOD FOR EXCHANGING AND DISPLAYING RESOURCE VIEWING POSITION AND RELATED INFORMATION filed Jul. 9, 2013, U.S. provisional patent application no. 62/196856 SYSTEM AND METHOD FOR GENERATING INTERACTIVE LAYERS OVER THE DISPLAY OF A RESOURCE BY ANOTHER APPLICATION filed Jul. 24, 2015 and U.S. patent application Ser. No. 15/215140 SYSTEM AND METHOD FOR GENERATING INTERACTIVE LAYERS OVER THE DISPLAY OF A RESOURCE BY ANOTHER APPLICATION filed Jul. 20, 2016 which are each hereby incorporated by reference in their entirety herein.
- According to an embodiment of the present invention and as illustrated at
FIG. 1 , a user using acomputing device 101 comprises a processor, program memory and data storage memory together with input/output hardware that connects the computing device to peripheral devices and acommunications network 106. Peripheral devices to the computing device may comprise one or more of auser display 104, keyboard, pointing mechanism and other peripherals may include voice input, gesture recognition and motion detection devices. The user operates retrieval and displaysoftware 102 loaded onto thecomputing device 101 that initiates information requests and receives responses over acommunications network 106 to display retrievedinformation 103 on the auser display 104. Under this embodiment, the user loadsuser layer software 105 to process retrievedinformation 108 according to the specifications contained in one or more sets of userlayer configuration data 107 to generate one or more user layer displays 302. - In a preferred embodiment, a user may control the behavior of the
user layer software 105 through a main task interface for which a control flow chart is illustrated atFIG. 2 . A user may decide whether or not to useuser layer software 105 and if they do they may create and edit one or more sets of userlayer configuration data 107 and control which of said sets of user layer configuration data are active for the generation of user layer displays 302. - In a preferred embodiment, each user layer's behavior, resulting from the application of user
layer configuration data 107 to retrievedinformation 108 byuser layer software 105, is configurable using parameters and instructions supplied by a user 501 that may comprise comparison tests against one or more of keywords, strings of words or phrases, logic sequences, language expressions, paradigms, object recognition and target outcomes as further illustrated atFIG. 7 . - In the preferred embodiment
user layer software 105 processes one or more sets of userlayer configuration data 107 to compute one or moregraphical overlays 303 for display in context with associated retrievedinformation 108 as rendered by retrieval and displaysoftware 102 on auser display 104 within auser display window 301 and this is illustrated inFIG. 3 . - In another embodiment a
graphical overlay 303 may be separately displayable, even to the extent of being shown in a separateuser display window 301 than retrievedinformation 108. In combination, whether shown in the same window or not, rendered retrievedinformation 108 plus one or moregraphical overlays 303 form auser layer display 302. - A preferred embodiment is illustrated in
FIG. 4 which is a flow diagram of core functions performed by theuser layer software 105 on an active set of userlayer configuration data 107. Where userlayer configuration data 107 contains parameters and instructions that require data in addition to retrievedinformation 103 theuser layer software 105 executes actions to access that additional data which may reside in auser layer history 110 or, as inFIG. 5 , on astorage device 503 attached to a resource server accessed via acommunications network 106. -
FIG. 5 shows a preferred embodiment where a user's 501computing device 101 is connected to acommunications network 106 via which a user may cause the retrieval anddisplay software 102 running on the device to make requests toinformation resource servers 502 which in turn accessesinformation storage devices 503 to accessinformation resources 504 which are sent as a response to thecomputing device 101 over the communications network where they become retrievedinformation 108 for rendering by the retrieval anddisplay software 102 on theuser display 104 and for any processing by theuser layer software 105. - In the preferred embodiment, user
layer configuration data 107 anduser layer history 110 information may also be stored on alayer server 601 as illustrated inFIG. 6 . A user 501 may thus access their userlayer configuration data 107 and said histories thereof using anycomputing device 101 from which they are able to sign in to said layer server. - In a preferred embodiment as illustrated at
FIG. 7 ,user layer software 105 provides a user 501 the ability to search and optionally edit one or more of userlayer configuration data 107 anduser layer history 110 data. - In a preferred embodiment,
user layer software 105 allows for the naming of each set of userlayer configuration data 107, illustrated inFIG. 7 , for easy reference and for sets of userlayer configuration data 107 to be grouped together to form a layer group and for each group to be named and activated together - In the preferred embodiment, through layer precedence information stored within user
layer configuration data 107 illustrated inFIG. 7 , a user 501 may combine two or more sets of userlayer configuration data 107 through cross-reference information that defines one or more connections between such sets of userlayer configuration data 107 including the precedence of their processing and the condition activating said cross-reference; for example, one said set of userlayer configuration data 107 may highlight duplicate items within the retrievedinformation 108, one said set may search for lowest price information for non-duplicate items by accessing additional retrievedinformation 108 and another said set may compare the results with prior results stored in theuser layer history 110, all resulting agraphical overlay 303 annotating unique items with the current lowest price for each and prior lowest prices. - In the preferred embodiment, a set of user
layer configuration data 801 to be processed is loaded 802 intodata storage memory 108 for userlayer configuration data 107 in the user'scomputing device 101 where it is processed byuser layer software 105 against retrievedinformation 103. Agraphical overlay 303, if any results, isoutput 803 byuser layer software 105 as part of auser layer display 302 in context with the rendered retrievedinformation 103.FIG. 8 provides a schematic representation of the above steps. - In the preferred embodiment,
use layer software 105 stores auser layer history 110 that retains retrievedinformation 108 and results of its processing said retrievedinformation 108 according to userlayer configuration data 107. An illustration of includinguser layer history 110 to produce agraphical overlay 303 is contained inFIG. 9 , which is identical toFIG. 8 except that the userlayer configuration data 107 has been edited to trigger a “Lower price” comparison rather than “Similar”. For example, if a user is conducting a search of the Internet using retrieval anddisplay software 102 and wishes to know which elements or group of elements in resulting retrievedinformation 108 were already returned by their prior searches, such conditions would be specified by the user as part of userlayer configuration data 108 whichuser layer software 105 would process to detect such conditions and generate agraphical overlay 303 to assist the user in distinguishing new results from prior duplicates. - In a preferred embodiment, sets of user
configuration layer data 107 are configurable by a user to automatically cause actions. For example, an automated process for finding matching data elements across a range ofinformation resources 504 may be organized as follows: userlayer configuration data 107 causes theuser layer software 105 to first search a web page stored as retrievedinformation 108 for specific tagged elements by invoking the standard javascript function Element.gelElementsByTagName( )to return a list of all elements in said retrievedinformation 108 matching a tag name specified in userlayer configuration data 107. Second, from the locations ofother information resources 504 supplied as part if userlayer configuration data 107, saiduser layer software 105 initiates retrieval and a search of said additional retrievedinformation 108 from saidother information resources 504 to identify elements or groups of elements that match said tag name and generate agraphical overlay 303 highlighting elements for which matches were found and any ancillary information specified in the userlayer configuration data 107 such as price and color. The result of such an operation is illustrated atFIG. 10 , in which the screen shot shows agraphical overlay 303 annotating similar results from saidother information resources 504 within auser layer display 302. - In the preferred embodiment, methods of annotation in
graphical overlays 303 are specified in userlayer configuration data 107 as illustrated atFIG. 7 . By employing a variety of such methods, a user may define the color, style, positioning and other parameters of an annotation to distinguish between them and suit their preference either through public domain javascript methods such as window.alert (“annotation text”); to create a temporary pop-up window and rect(x, y, widthinpixels, heightinpixels); to draw a rectangle on top or proprietary or patented annotation methods. In this way,user configuration data 107 is used byuser layer software 105 to display different distinguishing marks, style or modes of annotation in order to differentiate between results in agraphical overlay 303. - The preferred embodiment includes logic within the
user layer software 105 to identify which subset of retrievedinformation 103 information has been visibly displayed within thedisplay window 301 rendered by the retrieval anddisplay software 102. The preferred embodiment accomplishes this by capturing a user's viewing position by means of positional parameters using one or more methods anticipated inclaims 2 through 8 of patent application Ser. No. 13/938,122 filed Jul. 9, 2013 and storing said positional parameters as part ofuser layer history 110 data. Userlayer configuration data 107 may, therefore, additionally determine whether theuser layer software 105 acts or acted on all retrievedinformation 108 or only retrievedinformation 108 that was visible to the user using viewing position information collected within userlayer history data 110 as illustrated inFIG. 7 . - The preferred embodiment of
user layer software 105 may further assist the user 501 by allowing them to define how multiple sets of userlayer configuration data 107 are applied to retrievedinformation 108. This embodiment allows a user 501 to specify, for example, sets of userlayer configuration data 107 that should be applied on retrievedinformation 108 byuser layer software 105 in series like successive filters, whether they are applied in parallel and whether iteration is applied. For example, by applying sets of userlayer configuration data 107 in seriesuser layer software 105 will produce auser display 104 annotating items that meet the conditions in all sets of userlayer configuration data 107 whereas, by applying sets of userlayer configuration data 107 in parallel,user layer software 105 will produce auser display 104 annotating items that meet the conditions defined within said sets of userlayer configuration data 107 and when applying sets of userlayer configuration data 107 iteratively,user layer software 105 will produce auser display 104 that is updated when underlying conditions change. This preferred embodiment of combining application of multiple sets ofuser configuration data 107 also solves for the user 501 problem of identifying similar but unique elements in a singlegraphical overlay 303 within auser layer display 302. A example of combining user layers to produce agraphical overlay 303 showing how many comparable items were found in a reference database is illustrated at the block diagram atFIG. 11 and an example block diagram of multiple user layers that together seek comparable items, seek pricing for them and displaying changes against historical results in agraphical overlay 303, is shown atFIG. 12 . - In a preferred embodiment, user layer software has access to a user's viewing history as an
information resource 504 anduser layer history 101 from which is produced agraphical overlay 303 that annotates a portion or portions of previously retrievedinformation 108 already seen by the user in the manner exemplified inFIG. 13 . - In a preferred embodiment, user
layer configuration data 107 is configurable by the a user to access one or more reference data sources as aninformation resource 504 such that elements within retrievedinformation 108 that also occur in said reference data are highlighted within agraphical overlay 303. Such a configuration is illustrated atFIG. 14 . - In a preferred embodiment,
user layer software 105 allows a user 501 to suppress agraphical overlay 303, thereby permitting them to see underlying rendered retrievedinformation 108 uncluttered by graphical overlay(s) 303. - The preferred embodiment of
user layer software 105 assists users 501 in organizing their work by adding category meta-data, see example atFIG. 7 , to userlayer configuration data 107. - The preferred embodiment of
user layer software 105 provides users 501 the ability to share at least one of userlayer configuration data 107,graphical overlay 303,user layer display 302 anduser layer history 110 with other users by stipulating so as illustrated inFIG. 7 . Thus, an investment by one user in creating a user layer may be transferred for reuse to another user with the potential for monetary exchange in return for such transfer or reuse. - According to another embodiment of the invention, an apparatus for employing user layer software includes a display, an input/output hardware unit and a program memory coupled to a processor. Input/output hardware unit is capable of exchanging data with a network and data associated electronic resources accessible over a network. Program memory stores program instructions and a processor executes program instructions to (i) produce output from one or more software applications, (ii) execute user layer software for any user layer configuration data that has been enabled for those software applications, and (iii) display a graphical overlay derived from the output of user layer software placed in context with the output from said software applications in accordance with user layer configuration data.
- According to another embodiment of the invention, a system for employing user layer software includes a database, an input/output hardware unit, and a program memory coupled to a processor. Input/output hardware unit is capable of exchanging data with a network. Program memory stores program instructions and the a processor is capable of executing program instructions to: (i) store and retrieve user layer configuration data from the database (ii) store and retrieve from the database results from the user layer software having been executed using user layer configuration data iii) initiate exchange with other users of user layer configuration data, iv) initiate exchange with another user of results from user layer software having been executed using their user layer configuration data.
- According to another embodiment of the invention, a computer program product includes computer program logic stored in a tangible medium, wherein computer program logic executes program instructions to (i) execute user layer software for any user layer configuration data that has been enabled for a software application, and (iii) display a graphical overlay derived from the output of user layer software over the output from said software applications in accordance with user layer configuration data.
- According to another embodiment of the invention, a computer program product includes computer program logic stored in a tangible medium, wherein computer program logic includes logic for causing a computer program to: (i) store and retrieve user layer configuration data from the database (ii) store and retrieve from the database the results from user layer software having been executed using user layer configuration data iii) initiate exchange with other users of user layer configuration data, iv) initiate exchange with another user of the results from user layer software having been executed using their user layer configuration data.
- According to an embodiment of the present invention and as illustrated in the figures from which the present invention is a continuation, a user using a computing device comprises a processor, program memory and data storage memory together with input/output hardware that connects a computing device to peripheral devices and a communications network. Peripheral devices to the a computing device may comprise one or more of a user display, keyboard, pointing mechanism and other peripherals may include voice input, gesture recognition and motion detection devices. A user operates retrieval and display software loaded onto the a computing device that initiates information requests and receives responses over a communications network to display retrieved information on the a user display. Under this embodiment, a
user 105 loads user layer software to process retrieved information in relation to the user's display location within the retrieved information to display overlay information. - In a preferred embodiment, a user 501 may organize their
electronic information resource 1604 viewing activities intochannels 1501 or expeditions as illustrated inFIG. 15 , each of which is represented by achannel thumbnail 1502 comprising one or more of achannel image 1503, achannel name 1504, lastchannel activity information 1505 and channeluser role information 1506. In a preferred embodiment, channels thumbnails 1502 may be inchannel categories 1507 and may be searched through achannel search box 1508 to allow for the case, for example, of finding all channels with a particular user as a member. - In a preferred embodiment, once a user 501 has entered a channel they will be presented a channel screen as illustrated at
FIG. 16 comprising one or more of: - a
channel header 1601 containing one or more of achannel name 1504, channeluser role information 1506,channel category 1507,channel member information 1602 which allows access to details of members of the a channel in the case there is a plurality of users with access to the a channel, achannel activity indicator 1603 which allows access to the activity in the a channel which may be spread over more than one electronic information resource and in some embodiments additional information including the number of users in a channel, the number of resources being viewed within a channel or expedition or other information related to activity in the a channel; - a
resource display area 1604 showing an electronic information resource currently being viewed and optionally containing asharing control element 1605 allowing a current user to select a specific display location for one or more of a new chat thread, sticker, drawing including a user generated drawing or insertion of other user generated content such as a photo image; - a resource
browsing control area 1606 that permits a user to search for and select an electronic information resource to be showing in the aresource display area 1604; - an
annotation notification area 1607 providing information about activities not always visible within a current electronic informationresource display area 1604 for the achannel 1501 currently in focus. - In a preferred embodiment illustrated at
FIG. 17 , activatingchannel member information 1602 which allows access to details of members of the a channel in the form of achannel member list 1701. - In a preferred embodiment illustrated at
FIG. 18 , activating achannel activity indicator 1603 allows access to activity and any resulting annotations in the a channel which may be spread over more than one electronic information resource in the form of achannel activity list 1801. - In a preferred embodiment illustrated at
FIG. 19 , activating asharing control element 1605 permits a user to create one or more of: - a chat thread attached to a display location which may be displayed in an overlay to a current
electronic information resource 1604 as either a compact or closedchat thread indicator 1701 or an open chat thread 1702; - a sticker attached to a display location 1703 in an overlay to a current
electronic information resource 1604. - In a preferred embodiment illustrated at
FIG. 20 , activating the asharing control element 1605 permits a user 501 to create adrawing 2001 as an overlay to anelectronic information resource 1604. In the case where a plurality of users is interacting using the present invention those users can, in some embodiments providing they have access to the channel, separately and contemporaneously create drawings in the same channel as an overlay to a same electronic information resource as additionally exemplified as drawing from user A at 2001 and drawing from user B at 2902. - In a preferred embodiment also illustrated at
FIG. 20 . a currentelectronic information resource 1604 may also include user generated content such as photo images exemplified here at 2003 and with simultaneously drawn lines from a plurality of users 501 exemplified as drawing from user A at 2004 and drawing from user B at 2005. - In a preferred embodiment a user 501 interacts with a channel user
interface displaying channels 1501 using a method illustrated inFIG. 21 which allows users 501 to select from the at least one channel using itsthumbnail 1502. Referring toFIG. 21 , a user may press a menu button in 2101 resulting in a user having access to a profile and settings menu or menus. There, the a user may edit their profile and settings by selecting from a submenu or entering information. Also referring toFIG. 21 , a user may further select anannotation notification 2102, which allows a user to view and act on annotations by other users.FIG. 21 also illustrates that by selecting the a channel name 2013 a user is able to view channel detail information including which other users are members of the a channel. InFIG. 21 , selecting an unread bell 2014 provides a user with a list of all annotation notifications that they have not yet read and achannel search function 2105 enables a user to select a set of channels fitting a set of search characteristics they enter. Subsequent processes may include those depicted inFIGS. 22 through 25 . - In a preferred embodiment
electronic information resources 1604 are displayed in an Internet browser view integrated with overlaid annotations e.g. 1901, 1902 and 1903 and any annotation notifications using the method illustrated atFIG. 22 . The approach illustrated allows each user 501 to browse anyelectronic information resource 1604 independently of the activities of any other user 501 in the acurrent channel 1501 or anyother channel 1501. - In a preferred embodiment also as illustrated in
FIG. 22 , annotations e.g. 1901, 1902 and 1903 are initiated by users 501 using ashare button 1605 indicator which is dragged and dropped to a display location within a currentelectronic information resource 1604 viewing method as indicated in a drag and drop operation at 2201. InFIG. 22 are also shown methods that are added tostandard browser operations 2204 which include accessingannotation notifications 2202 throughannotation notification elements 1607 and the ability to modify how overlaid annotations e.g. 1901, 1902 and 1903 are displayed usingselector 2203.FIG. 22 also illustrates a method of accessing information relevant to achannel 1501 through achannel header 1601 by selectingaction elements 2205. - In a preferred embodiment a user 501, which includes a joining user, can initiate new overlaid annotations using the method illustrated at
FIG. 23 which also permits an overlaid annotation to be used to startnew channels 1501.FIG. 23 illustrates a method that can be used following selection of the a drag anddrop operation 2201 such that a user may choose one or more options such as sending an overlaid annotation to a different ornew channel 2301, select an annotation type such as adoodle 2302,sticker 2303 ortext 2304, repositioning a draft overlaidannotation 2305 to a different viewing position and communicating a finalizedannotation 2306 through a chosenchannel 1501. An annotation e.g. 1901, 1902 and 1903 information is communicated to other users through a communications network to which users are connected as illustrated inFIG. 5 andFIG. 6 . Annotations may be stored by all users 501 or stored for reference on one or more servers 501. - In a preferred embodiment, a user 501 can manage their interactions with other users including which other users 501 have access to
channels 1501 they created or are leading through either a main menu, achannel thumbnail 1502 or achannel header 1601 through a member detail process as illustrated atFIG. 24 .FIG. 24 shows a method in which members may be added or modified 2401 whether in a user's contact list or not, in which the a current user may select aspecific member 2402 to see more detail and optionally jump to that member'sviewing position 2403.FIG. 24 also illustrates the ability to block or suppress 2404 overlay annotations and associated annotation notifications. - In a preferred embodiment users 501 that are members of a
channel 1501 may receive notifications of annotations inchannels 1501 they are members of and can view and respond to those annotations using the method illustrated inFIG. 25 . As illustrated inFIG. 25 , a user may access annotation notifications in the form of achannel list 2501 or anunread notification list 2502 and select to jump 2503 to the a browser view of that annotation e.g. 1901, 1902 and 1903 or to make aquick reply 2504 directly from an annotation notification area concerned. - According to some embodiments of the invention, a computer of mobile device includes a display, a processor, memory and input/output devices, such as a touch screen, buttons, keys, microphone, and speakers and a network interface for coupling the computer electrically, wirelessly or optically to at least one network, such as Wi-Fi network, a LAN, WAN or the Internet. The processor is coupled to the display, memory, input/output devices and the Internet and executes program instructions stored in the memory to perform various functions described herein.
- According to some embodiments, the display includes a plurality of channels and other information as shown in
FIGS. 15-20 . The memory may store an application or app which allows users to login or logout, which information may be stored on a remote server. Channels shown on the display may be created when one or more users use the application to access content, for example on the Internet or another network, and the number of users in the channel can grow as users invite other users of the app or other users join a channel, who are permitted to do so by the app or by friendship status among one or more users. Data associated with a user's browsing within the app and/or channel within the app is stored to facilitate the collection and sharing of user activities described herein. - According to one embodiment, a user may sign in, view channels he or she is permitted to join, join channels, receive notifications of the activities of other users, and select notifications and be transported to electronic resources that are annotated by other users in the channel based on various user activities as described herein. Users may also access content while affiliated with one or more channels and perform activities that are pushed out to other users associated with or within at least one channel in the form of notifications or stored data on the user's browsing activity and annotations. The app itself may also provide statistics relating to the channels, update icons corresponding to channels based on user activity and provide other information to facilitate enabling users to engage and annotate content and perform activities around content in a channel.
- While particular embodiments are disclosed herein, it will be understood by one having ordinary skill in the art that changes may be made to those embodiments without departing from the spirit and scope of the invention.
Claims (11)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/389,896 US10579718B2 (en) | 2013-07-09 | 2016-12-23 | System and method for interacting in layers in channels over the display of a resource by another application |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/938,122 US9674260B2 (en) | 2013-07-09 | 2013-07-09 | System and method for exchanging and displaying resource viewing position and related information |
US15/215,140 US10699678B2 (en) | 2015-07-24 | 2016-07-20 | System and method for generating interactive layers over the display of a resource by another application |
US15/389,896 US10579718B2 (en) | 2013-07-09 | 2016-12-23 | System and method for interacting in layers in channels over the display of a resource by another application |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/215,140 Continuation-In-Part US10699678B2 (en) | 2013-07-09 | 2016-07-20 | System and method for generating interactive layers over the display of a resource by another application |
Publications (2)
Publication Number | Publication Date |
---|---|
US20170109338A1 true US20170109338A1 (en) | 2017-04-20 |
US10579718B2 US10579718B2 (en) | 2020-03-03 |
Family
ID=58523971
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/389,896 Active 2034-03-06 US10579718B2 (en) | 2013-07-09 | 2016-12-23 | System and method for interacting in layers in channels over the display of a resource by another application |
Country Status (1)
Country | Link |
---|---|
US (1) | US10579718B2 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190018581A1 (en) * | 2017-07-11 | 2019-01-17 | Facebook, Inc. | Methods and systems for defining and transmitting a drawing stroke |
US20190089658A1 (en) * | 2013-10-01 | 2019-03-21 | Lg Electronics Inc. | Mobile terminal and method of controlling therefor |
US11425060B2 (en) * | 2016-09-20 | 2022-08-23 | Google Llc | System and method for transmitting a response in a messaging application |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110258526A1 (en) * | 2010-04-20 | 2011-10-20 | International Business Machines Corporation | Web content annotation management web browser plug-in |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5271097A (en) | 1988-06-30 | 1993-12-14 | International Business Machines Corporation | Method and system for controlling the presentation of nested overlays utilizing image area mixing attributes |
US5874959A (en) | 1997-06-23 | 1999-02-23 | Rowe; A. Allen | Transparent overlay viewer interface |
US6584479B2 (en) | 1998-06-17 | 2003-06-24 | Xerox Corporation | Overlay presentation of textual and graphical annotations |
US7209928B2 (en) | 1998-07-29 | 2007-04-24 | Qwest Communications International Inc. | Information organization and navigation by user-generated associative overlays |
US20030187968A1 (en) | 2002-03-28 | 2003-10-02 | Gateway, Inc. | Layer menus and multiple page displays for web GUI |
US8122014B2 (en) | 2003-07-02 | 2012-02-21 | Vibrant Media, Inc. | Layered augmentation for web content |
-
2016
- 2016-12-23 US US15/389,896 patent/US10579718B2/en active Active
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110258526A1 (en) * | 2010-04-20 | 2011-10-20 | International Business Machines Corporation | Web content annotation management web browser plug-in |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190089658A1 (en) * | 2013-10-01 | 2019-03-21 | Lg Electronics Inc. | Mobile terminal and method of controlling therefor |
US10931606B2 (en) * | 2013-10-01 | 2021-02-23 | Lg Electronics Inc. | Mobile terminal and method of controlling therefor |
US11711325B2 (en) | 2013-10-01 | 2023-07-25 | Lg Electronics Inc. | Mobile terminal and method of controlling therefor for selectively sending messages using multiple message input windows |
US11425060B2 (en) * | 2016-09-20 | 2022-08-23 | Google Llc | System and method for transmitting a response in a messaging application |
US20190018581A1 (en) * | 2017-07-11 | 2019-01-17 | Facebook, Inc. | Methods and systems for defining and transmitting a drawing stroke |
US11048406B2 (en) * | 2017-07-11 | 2021-06-29 | Facebook, Inc. | Methods and systems for defining and transmitting a drawing stroke |
Also Published As
Publication number | Publication date |
---|---|
US10579718B2 (en) | 2020-03-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11861150B2 (en) | Methods and apparatus for managing and exchanging information using information objects | |
US9690831B2 (en) | Computer-implemented system and method for visual search construction, document triage, and coverage tracking | |
EP3221803B1 (en) | Relevant file identification using automated queries to disparate data storage locations | |
US8375035B2 (en) | Visual aid to assist making purchase by tracking key product characteristics | |
US20200371647A1 (en) | Systems and methods for semi-automated data transformation and presentation of content through adapted user interface | |
US8407576B1 (en) | Situational web-based dashboard | |
US20120221553A1 (en) | Methods for electronic document searching and graphically representing electronic document searches | |
US20100241947A1 (en) | Advanced features, service and displays of legal and regulatory information | |
US20060069690A1 (en) | Electronic file system graphical user interface | |
US9245045B2 (en) | Aggregating missing bibliographic information in a collaborative environment | |
US10762292B2 (en) | Systems and methods for collaborative editing of interactive walkthroughs of content | |
US20100205238A1 (en) | Methods and apparatus for intelligent exploratory visualization and analysis | |
US20130086079A1 (en) | Graphical user interface for interacting with automatically generated user profiles | |
US20160210021A1 (en) | Computer-Implemented Tools for Exploring Event Sequences | |
US11910082B1 (en) | Mobile interface for marking and organizing images | |
US20160125527A1 (en) | Financial Information Management System and User Interface | |
US10579718B2 (en) | System and method for interacting in layers in channels over the display of a resource by another application | |
US20160042080A1 (en) | Methods, Systems, and Apparatuses for Searching and Sharing User Accessed Content | |
JP7528301B2 (en) | SYSTEM AND METHOD FOR STORING AND SURFACING CONTENT - Patent application | |
US10699678B2 (en) | System and method for generating interactive layers over the display of a resource by another application | |
US10713270B2 (en) | Emerging issue detection and analysis | |
CA2764319A1 (en) | Advanced features, service and displays of legal and regulatory information | |
US8645845B2 (en) | Collaboration plane | |
Chatterjee | Design research: Building human-centered system | |
Lu et al. | HARVEST: Situational Visual Analytics for the Masses |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: MICROENTITY |
|
FEPP | Fee payment procedure |
Free format text: SURCHARGE FOR LATE PAYMENT, MICRO ENTITY (ORIGINAL EVENT CODE: M3554); ENTITY STATUS OF PATENT OWNER: MICROENTITY |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, MICRO ENTITY (ORIGINAL EVENT CODE: M3551); ENTITY STATUS OF PATENT OWNER: MICROENTITY Year of fee payment: 4 |