WO2023069068A1 - Gui based methods and systems for working with large numbers of interactive items - Google Patents

Gui based methods and systems for working with large numbers of interactive items Download PDF

Info

Publication number
WO2023069068A1
WO2023069068A1 PCT/US2021/055422 US2021055422W WO2023069068A1 WO 2023069068 A1 WO2023069068 A1 WO 2023069068A1 US 2021055422 W US2021055422 W US 2021055422W WO 2023069068 A1 WO2023069068 A1 WO 2023069068A1
Authority
WO
WIPO (PCT)
Prior art keywords
selection set
entities
gui
selection
selectable
Prior art date
Application number
PCT/US2021/055422
Other languages
French (fr)
Inventor
Jason Kim
Original Assignee
Jason Kim
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jason Kim filed Critical Jason Kim
Priority to PCT/US2021/055422 priority Critical patent/WO2023069068A1/en
Publication of WO2023069068A1 publication Critical patent/WO2023069068A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482Interaction with lists of selectable items, e.g. menus
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842Selection of displayed objects or displayed text elements

Definitions

  • GUI interfaces for computers have been in common use since the mid-1980s.
  • One of the most fundamental (and earliest) of these GUIs shows the layout and location of files in the computer’s local storage in graphical form, using the adopted nomenclature “window” and “icons” where each icon can represent a single file or directory.
  • the user selects the files to operate on (e.g., creates a selection set) by “clicking” on the icon.
  • the user then initiates the desired operation on the selected entity (in this case, a file or directory) such as copying or moving the selected files, or deleting them, or other operations.
  • the user manually selects multiple items (or entities) by clicking on them in sequence.
  • Other common selection methods include “shift-click” methods, where a contiguous sequence of entities can be selected (or unselected) by clicking on the first entity, then clicking on the last item in the sequence, typically with another modifier (typically the "Shift” key).
  • Individual entities can also be added or removed from the selections by “control-click”, where another modifier is held active while individual entities are clicked, thus adding and removing them from the selection set.
  • Boolean selection methods are also known in prior art. For example, Borovoy, US patent 7,769,741 teaches a portable browsing interface for information retrieval. Neale, US patent 7,552,395 teaches a graphical user interface for building Boolean queries and viewing search results. Such methods are commonly available as an “advanced” option on many databases, including the LISPTO patent search database.
  • the invention is based, in part, on the insight that although prior art GUI selection methods work well when operating with at most a few dozen entities at a time. However, these prior art methods can become increasingly inadequate when the GUI is dealing with a large number of selectable entities (e.g. hundreds, thousands or more selectable entities). These prior art methods also become inadequate if similar tasks must be repeated across multiple entities. Similarly, although prior art Boolean search methods can operate with larger numbers of selectable entities, such methods tend to be difficult to use. The invention is based, in part, on the insight that it is not realistic, in an easy to use system, to expect the user to be able to manually add and remove, in an error-free manner, a few elements from a set comprising hundreds or thousands of different elements.
  • prior art GUI tend to operate in a way where a single errant click can reset the entire selection set.
  • Such prior art GUI methods can be both frustrating to users, and can also be inefficient in terms of time and effort, particularly if the user must then repeat similar type selection-and-operate cycles, with slightly varying parameters, when the content of a folder directory changes, or when the user is operating in different directories.
  • the invention is also based, in part, on the insight that in order to better deal with the above issues (e.g., using prior art GUI methods to operate on a large number of items), improved GUI methods and systems are needed that add additional functions to the GUI.
  • the invention disclosed herein can be any of a method, system, or a non- transitory computer storage medium for computer code for a processor to implement the method and system versions of the invention.
  • a processor to implement the method and system versions of the invention.
  • the method embodiments of the invention will be provided as specific examples.
  • the invention may be a graphical user interface (GUI) based method for managing selectable entities.
  • GUI graphical user interface
  • This GUI method will typically comprise designating, using a GUI and a computer processor, from a plurality of selectable entities, a selection set comprising entities from plurality of selectable entities, and saving this selection set in computer memory as a first selection set.
  • the method will also comprise designating, using this GUI and a computer processor, from this plurality of selectable entities, at least one different selection set comprising different entities from the plurality of selectable entities, and saving each at least one different selection set in computer memory as at least one different selection set.
  • the method will also comprise selecting, using the GUI and a computer processor, at least one set operation to be performed on the first selection set and on the at least one different selection set.
  • the method will then use the computer processor, this at least one set operation, the first selection set, and the at least one different selection set, and produce at least one new selection set.
  • the method can also use this least one new selection set to manage at least some of the various selectable entities.
  • Fig. 1 shows a GUI screenshot where the user has manually selected a single file.
  • Fig. 2 shows a GUI screenshot where the user is using a “Search” operation to find JPEG files.
  • Fig. 3 shows a GUI screenshot showing the result of the Fig. 2 Search operation. Note that Files other than JPEGs are hidden from view.
  • Fig. 4 shows using a GUI to save a Selection Set.
  • the user has selected a subset of the search and is saving the subset as “Mockup JPEGs Only”.
  • Fig. 5 shows a detail of the selected “JPEGs Only” selection set previously selected in Fig. 4.
  • Fig. 6 shows a GUI screenshot of the user changing the Active Selection (To “Mockup JPEGs Only”).
  • Fig. 7 shows a GUI screenshot of the result of the user Changing the Active Selection Set to “Mockup JPEGs Only”.
  • Fig. 8 shows a GUI screenshot where the user has set the "JPEGs Only" Selection set as Active.
  • Fig. 9 shows a GUI screenshot where the user is Subtracting "Mockup JPEGs Only” from the “JPEGs Only” selection.
  • Fig. 10 shows a GUI screenshot showing the result of the Subtract operation from Fig. 9. Note that the JPEGs in the “Mockup” Directory are no longer selected.
  • Fig. 11 shows a GUI screenshot showing the Screen for adding user-supplied functions/scripts to the GUI. These scripts can be used during the Filter operation to arbitrarily query the Objects and/or modify selection sets.
  • Fig. 12 shows the website youtube.com, as rendered by the present invention browser.
  • Fig. 13 shows Creating a new filter for Video Streams less than 1 GB in size.
  • Fig. 14 shows the result of the search in Fig. 13.
  • Fig. 15 shows creating a “filter chain”, according to function 8 (described below in Detailed Description Section).
  • Fig. 16 shows the result of applying the filter chain (shown on Fig 15) (an exemplar of function 8), thus creating a new selection set.
  • Fig. 17 shows a video game example, here a video game inventory screen, “Searching for Ores” is shown
  • Fig. 18 shows creating a new item set “Cheap Ores” for the video game of fig. 17.
  • Fig. 19 shows the result of creation of the “Cheap Ores” item set from fig. 18.
  • Fig. 20 shows the result of visiting a shop, and selling all of the “Cheap Ores” from Fig. 19
  • Fig. 21 shows the result of visiting a different shop, and performing the operation of refining all “Expensive Ores”.
  • Fig. 22 shows the result of revisiting the first shop, and selling “Cheap Ores” and “Junk”.
  • Fig. 23 shows a first flow chart of an embodiment of the invention.
  • Fig. 24 shows a second flow chart of an embodiment of the invention. It focuses in on Box (E) first shown in Figure 23. Note that the large four-pointed arrow in the center of the figure is used to simply mean that steps (F) through (K) as well as (B1 ) and (B2) can be done in any order.
  • the entity that the GUI manipulates can also be “selected” by the user, either one entity at a time or multiple entities at a time, for additional manipulation and/or operation.
  • the underlying files and/or directories can be selected, by way of their linked entities, so that the user can copy, rename, move, or delete, one or more files/directories at the same time.
  • the GUI may be configured to receive the user input by any of a touch interface, pointing device, gesture recognition device, or voice recognition device.
  • a touch interface pointing device
  • gesture recognition device or voice recognition device.
  • voice recognition device any of a touch interface, pointing device, gesture recognition device, or voice recognition device.
  • these entities are displayed by the computer processor on the GUI, and are selected (e.g. by using any of a touch interface and pointing device, gesture recognition device, voice recognition device, or other method) to designate at least some of the entities displayed by the computer processor on the GUI.
  • activating - this disclosure also occasionally uses the term “activate” to mean that something or someone has caused an entity (or a set of entities) to become selected (or “clicked”, as per above in Definition 2). Note, however, that since “activation” is implied any time a user directs a computer to perform an operation, the term “activation” is generally not intended as a limitation unless otherwise so specified.
  • This invention is primarily concerned with selectable entities, and generally ignores or is silent with respect to hidden or invisible objects.
  • the actual object that the entity refers to becomes important when the user carries out an operation on the selected entities (see definition 4 below).
  • this disclosure only distinguishes between the entity and its associated object when working with selections or selection sets.
  • a selectable entity refers to those entity linked object or objects that are managed by the GUI.
  • the entities provide the mapping or linkage between the GUI representation of the objects to the actual objects that the GUI operates on.
  • the GUI when one or more entities are somehow selected (e.g. via a “click”) or activated through some other means, the GUI then has the links to the set of actual objects that the GUI can operate on.
  • the entities describe or link to objects.
  • objects are the embodiment of entities.
  • GUI typically provides various features and capabilities to operate on objects.
  • Each GUI has different capabilities.
  • a file browser might offer the capability to copy, rename, move, delete files and directories.
  • this disclosure so describes it as “GUI operates on the selected entities.”
  • this disclosure often uses the term entities to refer to their associated objects.
  • Selection Set simply, a set of selectable entities that have been “clicked on” or otherwise activated somehow. Most GUIs visually distinguish between unselected entities and the selected ones. The actual graphical or other type selection cues are not important for this invention, nor is the method, systems or devices used for the selection process.
  • GUI User Interface
  • Computer processor Modern computers and computerized devices typically comprise at least one processor, often an x86, MIPS, Arm, or another type processor, that often may contain more than one processor core. Often such computerized devices may contain specialized co-processors, such as graphics co-processors, and other processor devices.
  • processor will be used to denote any and all of these combinations.
  • computer processor can be understood as being equivalent to “at least one computer processor.”
  • the invention may be a graphical user interface (GUI) based method for managing selectable entities.
  • GUI graphical user interface
  • This method can comprise using a GUI (often by human user input on the GUI) and a computer processor to designate, from a plurality of selectable entities, a first selection set comprising various entities from a plurality of selectable entities, and save this selection set in computer memory as a first selection set.
  • the method will often also use this GUI (and often the same human’s input into this GUI) and usually the same computer processor, and this plurality of selectable entities to designate at least one different selection set comprising different entities from this plurality of selectable entities.
  • This at least one different selection set will typically be saved in computer memory as at least one different selection set.
  • the method will also use this GUI (again often by the same human user providing input to this GUI), and usually the same computer processor, to select at least one set operation to be performed on the first selection set and the at least one different selection set.
  • the invention will then typically automatically create, using this computer processor, the at least one set operation, the first selection set, and the at least one different selection set, at least one new selection set.
  • the invention can then either automatically, or as designated by the user using the GUI, use this at least one new selection set to manage at least some of the selectable entities.
  • the invention may be a computerized method, system or even a non-transitory computer-readable medium for improving GUI usability.
  • the invention operates by performing one or more of the following eight operations: These eight operations or functions, which may be implemented according to various combinations and permutations, may thus be viewed as comprising important components of the present invention. These functions can include:
  • the invention may further comprise adding or removing additional entities (often by input from a human user on the GUI) from any of the first selection set, the at least one different selection set, or the at least one new selection set, thereby creating any of a modified first selection set, a modified different selection set, and a modified new selection set.
  • the invention may further comprise saving any of the modified first selection set, the modified different selection set, and the modified new selection set under any of their original names and their modified names.
  • (5) Optionally removing any unneeded saved selection sets.
  • (6) Performing simple set operations on one or more selection sets or saved selection sets (e.g. combine sets, subtract one set from an another, find common Entities in multiple Selection sets) to create new selection sets which can then be saved per function 1 .
  • These set operations are Boolean set operations, which are carried out on one or more named set of selections to compose more named selection sets. The most common ones are INTERSECTION (i.e. “AND”), UNION (i.e. “OR”) SUBTRACT (i.e. “MINUS”), and INVERT (i.e. “NEGATE” or “NOT”).
  • the user “Bob” might create a selection set of files modified during the last week, and subtract from this first set the set of files that were modified by another user “Mary”.
  • the one or more set operations can comprise any of set union, set intersection, and set inversion (i.e. set negation).
  • a “complete group of set operations” means that you can express any set operation as a sequence of these more basic “complete” operations.
  • a exclusive-or B is ((A UNION B) INTERSECT (INVERT (A INTERSECT B))).
  • UNION, INTERSECTION, INVERT means also reads on an “exclusive-or” operation.
  • a setsubtract operation can be phrased in terms of the UNION, INTERSECT, INVERT set operations.
  • the exclusive-OR and set subtraction operations are also claimed.
  • Selection set and a saved selection set so that additional changes to the selection set and saved selection set can be made later, at the convenience of the user.
  • the GUI and the computer processor can be further configured to search the plurality of selectable entities for at least one GUI selectable search parameter.
  • the invention can further enable the user to use the GUI to define at least one GUI selectable parameter.
  • the invention can then use the computer processor, this at least one GUI selectable parameter, and the plurality of selectable entities to determine any of the first selection set and the at least one different selection set.
  • the invention may carry out arbitrary computationally driven decision making about the entities present in a selection set or saved selection set, either through a search feature/operation or a more in-depth method, such as a script or program, to create new selection sets which can then be saved per function 1 .
  • the invention allows additional reasoning about the individual entities in a selection, such that intelligent choices can be made by the processor about which entities to add or remove from a selection set. For example, the user might wish to add to (or remove from) a selection set the list of files that contain a specific marker or markers, allowing for more complex machine reasoning (such as if-then-else) about the pending entities.
  • a possible list of traits, that may become part of the processor’s decision process for selection inclusion or exclusion, includes the size of the file, date of last modification, date of creation, the file format, custom labels, user comments about the file etc. The intention here is that this last item fully allows arbitrary computation about the entities and objects under consideration.
  • the invention further can use the GUI and the computer processor, to create or select at least one script or program, thereby determining at least one selected script or program.
  • this at least one script or program can be configured to accept this at least one new selection set as any of input or output parameters for this at least one script or program.
  • the invention can then use the computer processor to execute this script according to the at least one new selection set.
  • the user can continue adding/removing entities from any of the named selection sets created earlier.
  • saving these modified selection sets, activating these selection sets, and operating on these selection sets i.e. carry out a GUI operation on the selected entities, as per definition 4
  • creating new selection sets by various means described in the functions are all possible.
  • the actual objects that the entity refers to (e.g. the linked files/directories objects) need not themselves be saved or loaded. Often, what is being saved is just enough information that signifies within the GUI that an entity is selected. That is, the selection sets are made so that the user can choose to do an operation on the selected entities (and the underlying objects) at a later point in time, as well as add/subtract new entities to the selection set to fine-tune which entities are selected for an operation.
  • Figures 1 through 10 show an implemented browser GUI with the functions (i.e. various aspects of the present invention) built-in.
  • the selectable entities are files and directories.
  • the entities can comprise any of files and directories.
  • the at least one new selection set can be used to perform any of memory storage operations and network transmission operations upon any of the underlying files and directories objects in this at least one new selection set.
  • the entities may comprise of any object, manipulatable by the GUI, that can be selected by any methodology available to the GUI.
  • Figures 1 through 10 demonstrate the invention in a file browser GUI, and thus the "selectable entities”, as well as the underlying objects, are files and directories. However, there are many kinds of GUIs and many kinds of selectable entities.
  • Figure 8-10 detail set operation on selection sets.
  • Fig. 11 shows how a scripting window (e.g. for function 8) may be used.
  • Figures 12 through 16 show the application of the Invention where the selectable entities are uniform resource locators (URLs) for video streams within a web browser, i.e. they are not files in a local directory.
  • Figures 17 through 22 show another example of the Claims in action where the Selectable Entities are Inventory items in a Video Game. In the figures, areas of specific interest, such as selected entries or discussed functions, are often indicated with an arrow.
  • the entities can comprise items in a video game, where this video game can be configured to be run by any of one or more computer processors and/or at least one remote server.
  • the entities can comprise any of remotely hosted audio and video streams.
  • the GUI and the computer processor can be further configured with web browser functionality, and the first selection set, the at least one different selection set, and the at least one new selection set can comprise sets comprising any of these remotely hosted audio and video streams.
  • Fig. 1 shows a File Browser GUI with a single file manually selected (i.e. “clicked” by the user. This single selected file is now an unnamed selection set containing one entity. The user can then choose an operation on this selected entity at any time.
  • Fig. 2 shows using a search capability of the GUI where the user is searching for JPEG files. Clarification: In the nomenclature used by the GUI, a “Filter” is used to “filter” the Entities for the desired items, i.e. Filter is-a Search.
  • Fig. 3 shows the result of the search. Notice that files other than JPEGs are hidden from view. This tying of search results to selection is a crucial function of this invention. Typically, searches are for finding entities with a specific trait. This function allows for the result of multiple searches to be saved as selection sets, allowing for the user to more efficiently generate the desired selection sets by incorporating multiple searches, manual modification as well as arbitrary computation. Examples of functions 5, 7 and 8 are below.
  • Fig. 4 the user is creating (and saving) a newly named selection set from JPEG files in the Mockup directory.
  • the user has manually selected (or clicked on) the rows that signify “Mockup/” and is creating (and saving) the selected rows.
  • the files themselves are not being saved here (they are already there!).
  • What is being saved is the information that signifies that JPEG files in the Mockup directory have been selected (e.g. “clicked on”) by the user, and that the selection set is given a name for future use and reference by the user.
  • the circled button near the top was pressed to activate the shown dialog.
  • Fig. 6 the user is changing the active selection set to “Mockup JPEGs Only”.
  • the button indicated by the circle and arrow near the top was pressed by the user to activate the shown dialog.
  • Fig. 7 shows the result of the operation in Fig. 6. Notice that JPEGs in Mockup directory are still selected, but other JPEG files are no longer selected, i.e. the selections have been reset to be identical to that of Fig. 4. (For example, note that "Tree Browser.jpg" is no longer selected)
  • the selection set is not the fi les/d irectories, but rather is a type of short-lived ephemeral metadata pertaining to the “clicked” state of the individual Entity (which is, in this case, a row element that contains some detailed information about the file).
  • the process of creating, saving, naming, modifying, or deleting the selection sets does not actually impact the selected entities (in this case files and directories) at all.
  • What is being saved is metadata pertaining to whether or not a particular entity was selected, as well as the given name for this set.
  • the underlying files/directories objects are only modified when the user chooses to carry out an operation on the selected entities that link to these underlying objects.
  • Figures 8-10 demonstrate the power of set operations in modifying Selections
  • Fig. 8 shows the “JPEGs Only” selection set is activated. Notice that the JPEG files (including the ones in the Mockup directory) are selected.
  • Fig. 9 shows the initiation of a set subtract operation, where the “Mockups JPEGs Only” selection set is being chosen to subtract from “JPEGs Only”. The button indicated by the circle and arrow (resembling a "minus sign") was pressed by the user to activate this function.
  • Fig. 10 shows the resulting selections, where the JPEGs NOT in the Mockup directory are still selected. Note that the resulting selection has been saved to a new selection set named “Subtract”. Other operations that are featured are the expected common set operations like “intersect” (find common entities), “union” (add entities from one set to another), “select all” (select all entities that are in the current view), “exclusive or” (all entities in one set but not in another), and “invert” (flip the selection status of all entities in the current view). Note that the current view, as demonstrated prior as well in the following figures, can be the default view, or the result of a filter operation.
  • One advantage of the invention’s improved functionality here is that with set operations, and the ability to have multiple named selection sets, the invention provides greater ease-of-use and improved file management capability.
  • Fig. 11 shows the interface in the GUI where the user can supply a program or script.
  • the script can query the objects (i.e. what the selectable entities link or refer to).
  • the invention can use this information to make choices about which entities are found as part of the search/filter process, as well as directly modify selection sets.
  • the scripts can also be used to program (implement) a wider range of actions that might otherwise be difficult to compose directly with the GUI.
  • Figures 12 - 16 show an Embodiment where the selectable entities are URLs (uniform resource locators) for video streams within a web browser (e.g., the underlying objects are not merely files in a local directory).
  • the operations involve video URL operations, such as play videos in sequence, etc.
  • Fig. 12 shows the front page of a popular website youtube.com as rendered by the invention’s browser in detail view. The actual URLs and file sizes have been intentionally mangled for diagram purposes.
  • Fig. 13 shows a filter where the user is selecting the video streams that are less than 1 GB in size.
  • Fig. 14 shows the results of the search where a bunch of unneeded URLs (these mostly are for category pages) are showing up.
  • Fig. 15 shows an exemplar of the invention’s function 8, which is the catch-all capability where additional computation can be done to drill down on the specific desired entities.
  • the user has modified the search to be a chain of two filters, where the first filter selects all URLs that point to video streams less than 1 gigabyte (GB) and the second filter selects those with sizes greater than 10,000,000 bytes.
  • GB gigabyte
  • chaining filters is not the only way to implement function 8.
  • this aspect of the invention can also be used to support arbitrary code (i.e. a script or program). This code can be inserted into the filter-select loop, which can then direct the computer processor to automatically query the entities (and objects) in arbitrary ways to reach a decision on selection.
  • function 8 can be “computationally complete”. In other embodiments, however, it may be useful to limit function 8 to a “sandbox” subset of functions or otherwise limit the functionality of function 8 to protect the user from inadvertent mistakes, or for higher security.
  • Fig. 16 shows the results of a filter chain, where meaningful video URLs of the specified sizes are shown. Several URLs are then manually selected and is saved as a selection set named “To look at later”.
  • Figures 17-22 shows a hypothetical GUI for a video game.
  • the selectable entities are inventory items of the game character.
  • a “game character”, in this case is the fictional person, in-game, that is under the control of the user (i.e. player) of the video game.
  • Inventory items are the ephemeral objects within the video game that the game character has somehow acquired through play.
  • Fig. 17 shows an inventory screen where the user (or player, in this case) is searching for metallic ores. Here the user has highlighted 3 ores. Copper ores and tin ores are shown as selected. One of the three selected ores is not shown in the Inventory listing.
  • the “create new item set” button is Active.
  • the player has elected to create a new “Item Set”, i.e. “selection set” in this milieu.
  • This item set is called “cheap ores”.
  • Fig. 19 shows the results of the creation of this item set.
  • the three selected entities i.e. inventory items, in this milieu
  • They are copper, tin, and lead ores.
  • Figures 16-18 are exemplars of function 1 ,2,3 and 7.
  • Fig. 21 the game character is visiting a different shop, where the action being contemplated is to refine the ores in the “expensive ores” to produce ingots, presumably for some future crafting use.
  • Figures 23 and 24 describe the operational flow diagram of the GUI, as experienced by the user, in one embodiment of the invention.
  • the user starts the GUI (Box A) and starts work by choosing some entities to operate on (Boxes B1 and B2).
  • the user initiates the work by invoking the desired operation on the selected entities (Boxes (C) and (D)).
  • Box (B1 ) and Box (B2) are where users typically would spend the majority of time interacting with the GUI. As the number of selectable entities increase, the task of selecting exactly the desired entities for operation becomes a much more burdensome task.
  • GUI that implements a preferred embodiment of this disclosure (examples of such an implementation were presented in the previous figures), the user has the option of employing the various functions described herein to enhance productivity, especially if the number of selectable entities is unwieldy.
  • Figure 23 shows the top-level view of the operational flow diagram of the GUI.
  • Boxes (A) through (D) show a typical workflow when the number of selectable entities are small and manageable. However, if the number of selectable entities is large or the exact desired entities must be discovered, then the enhancements as diagramed in box (E) can greatly enhance productivity, as well as reduce user frustration.
  • Fig. 24 expands on the box (E) shown in Fig 23. This figure shows how the invention can then operate on one or more selection sets. If the selection set is empty, the invention then can switch to a saved selection set (Box G). If the selection set is not empty, the invention can then save the selection set to a given name supplied by a user. As per B1 and B2, the user has the option to manually add or delete entities in the current selection set, and/or use set operations on one or more named selection sets that were previously described (Box H).
  • the user can also direct the invention to run a search or filter operation (Box F), and/or refine a selection set by composing one or more filter/search results and/or selection sets, or by Boolean operations or other in-depth method, such as calling user defined programs (Box I).
  • the user can also direct the invention to remove unwanted selection sets (Box J).
  • the large 4-point arrow in the figure signifies that these steps can be done in any order to produce the exact set of entities for the desired operation. In other words, steps (F) through (K) and (B1/B2) can be done in any order for these purposes.
  • the invention will have a set of selected entities (Box C), from which the user can invoke the desired operation on said selected entities.
  • the present invention may thus comprise an improved GUI equipped to implement at least some of the various combinations of the previously described functions 1-8, here described again in the list below.
  • GUI automatically as a selection set.
  • the actual presentation, the visual cues, the iconography, the display devices and their related methods and systems, the interactive devices and their methods and systems being used with (or embody) the GUI need not impact the applicability of the invention.
  • at least some embodiments of this invention may apply to all types of GUIs, no matter the specific GUI form or function.
  • GUI executes on a Linux server, but the objects are stored at a different Windows server).
  • the objects in figures 12-16 that the URLs refer to are actually hosted at a remote website youtube.com but it could just as easily have been hosted somewhere else.
  • the GUI could initiate network transmission of the objects in question, as well as modify the objects through memory storage operations, executed either locally (if the objects are local), or through some method of communication between one or more servers (if the objects are not locally accessible).
  • the operations featured by the GUI for managing/manipulating the objects may not always be relevant to the functions (described above) on the selectable entities that a user may elect to invoke.
  • GUI graphical user interface
  • a file browser e.g., whether or not the GUI is a file browser, a video game, a web browser, or a custom tool for managing robots, cars, or whether it’s a tool for executing just-in-time inventory management for factories and warehouses.
  • Some embodiments of the invention may also apply regardless of the extra feature sets (e.g. functions) employed by the GUI.
  • the GUI used to generate Figures 1 -7 includes some ease-of-use shortcuts (e.g. results of a search is automatically saved as a selection set of the same name).
  • Some embodiments of the invention may also apply regardless of whether such ease-of-use shortcuts are featured.
  • the key issue is if the GUI offers, to the user, the capability to create selection sets (through various means), save the selection sets (with names), load the selection sets sometime later, manipulate the selection set as needed through various means, and finally operate on the selection sets as desired.
  • GUI tool pictured in the Figures may be implemented using computation time-efficient, as well as storage-space efficient methods and techniques for representation of the selection set as well as other components of the GUI.
  • some embodiments of the invention apply regardless of the efficiency (or the lack there-of) of the underlying systems and methods that implement the named selection set mechanism and other components of the GUI.
  • the invention may be configured so that selection sets themselves may also be selectable entities.
  • the GUI features discussed herein for selectable entities can also be used to manipulate selection sets in such embodiments. For example, after Mary goes on vacation, Bob may want to find the answer to "what were the selection sets that Mary created yesterday?" and then continue to operate the GUI, continuing where Mary left off.
  • the capabilities built-in to the GUI to find e.g. via search or filter
  • group i.e. create named selection sets for later use
  • manipulate can also be applied to the selection sets as well. This can be especially useful if the number of selection sets created by the users are also large.
  • the invention can implement these sets-of-selection sets in a variety of ways.
  • these sets of selection-sets were implemented in the same way as selection sets - i.e. it has a name (chosen by the user) and a list of entities.
  • these entities can refer to either selection sets or other selectable entities.
  • these set-of-sets can themselves be selection set for all intents and purposes.
  • these sets-of-selection-sets can also carry around (e.g. be linked with) additional metadata similar to the metadata associated with the underlying objects (such as time of creation, and the user-ID of the creator). Additional types of user provided metadata may also be linked with the sets-of-selection-sets.
  • additional metadata can also be searchable by the same search/filter interface in the GUI (i.e. Function 7). Boolean operations (i.e. functions 6), as well as filter composition (i.e. exemplar of function 8) can also be implemented so as to work on both kinds of selection sets.
  • selection-set abstraction methods can be implemented to also ontologically include sets-of-selection-sets.
  • the GUI operations that create, search, manipulate e.g. Boolean operations and compositions as well as arbitrary manipulation via user supplied programs), save (i.e. for later use), load (from a prior session) can be configured to all operate in the same manner for both types.
  • function number 8 i.e. the ability to invoke an arbitrary user supplied program
  • function number 8 can also be configured to include the ability to reason about sets of selection sets.
  • the system is configured so that the concept of sets of selection sets is ontologically and epistemologically included within Function 8.
  • the invention may fully apply regardless of the kinds of selectable entities, the objects that the entities refer to, the actual GUI technology being used, the "look-and-feel” and the actual devices or methods or systems being used to select an entity within the GUI.
  • GUI and computer processor a variety of different devices and methods may be used to convey user intentions to the system’s GUI and computer processor.
  • GUI display may also comprise head mounted augmented reality or virtual reality displays.
  • holographic displays where three- dimensional images, text and other visual iconographies are rendered, either in stereoscopic format (e.g. where slightly varying images are rendered to each visual sensor (e.g. the human eye) to simulate the presence of a three-dimensional object are also contemplated.
  • Such 3D displays may operate directly in open air (such as via interference of laser beams).
  • Such 3D displays may also be implemented by other means such as a multitude of thin light emitting rods being raised/lowered/moved about in a darkened space quickly).
  • this invention and claims herein are intended for scenarios where a user interacts with a GUI.
  • the GUI is configured to accept at least designating the selectable entities, the different entities, and selecting at least one set operation as input from a human user, and at least designating the selectable entities, the different entities, and selecting the at least one set operation is done by the human user using the GUI.
  • the invention can be a system for generating a graphical user interface (GUI), the system comprising: a processing device; and a memory device including instructions that are executable by the processing device for causing the processing device to create a set of selected entities as a selection set, and to save the selection set under a name to form a saved selection set, and then choosing a saved selection set to activate the entities previously saved in the selection set so chosen, for operation with the GUI.
  • GUI graphical user interface
  • the memory device can further include instructions that are executable by the processing device for causing the processing device to add or remove additional entities from the saved selection set.
  • the memory device can further include instructions that are executable by the processing device for causing the processing device to save the so modified selection set, either with the name of the original saved selection set or a new name.
  • the memory device can further include instructions that are executable by the processing device for causing the processing device to perform set operations on one or more selection sets or saved selection sets (e.g. combine sets, subtract one set from an another, find common entities in multiple selection sets, etc. as per function 6) to create new selection sets which can then be saved, activated and modified (per previously discussed functions 1 ,2,3,6), for operation with the GUI.
  • one or more selection sets or saved selection sets e.g. combine sets, subtract one set from an another, find common entities in multiple selection sets, etc. as per function 6
  • the memory device can further include instructions that are executable by the processing device for causing the processing device to perform a search operation with the GUI, and the results of the search (or some subset of the results of the search as per function 7) then can be saved as a selection set, and modified at will, and then can be activated later (as per previously discussed functions 1 ,2,3,7), for operation with the GUI.
  • the memory device can further include instructions that are executable by the processing device for causing the processing device to perform arbitrary computation for the purposes of creating or modifying selection sets (as per function 8), which then can be saved as a selection set, and modified at will, then to activate the selection set later (as per previously discussed functions 1 ,2,3,8), for operation with the GUI.
  • the invention may be a non-transitory computer- readable medium comprising program code that is executable by a processing device for causing the processing device to create a set of selected entities as a selection set, and to save the selection set under a name to form a saved selection set, and then choosing a saved selection set to activate the entities previously saved in the selection set so chosen, for operation with the GUI.
  • this non-transitory computer-readable medium can further comprise program code that is executable by the processing device for causing the processing device to add or remove additional entities from the saved selection set.
  • the non-transitory computer-readable medium can further comprise program code that is executable by the processing device for causing the processing device to save the so modified selection set, either with the name of the original saved selection set or a new name.
  • the non-transitory computer-readable medium can further comprise program code that is executable by the processing device for causing the processing device to perform set operations on one or more selection sets or saved selection sets (e.g. combine sets, subtract one set from an another, find common Entities in multiple Selection sets, etc. as per function 6) to create new selection sets which can then be saved, activated and modified (per previously discussed functions 1 ,2, 3, 6), for operation with the GUI.
  • program code that is executable by the processing device for causing the processing device to perform set operations on one or more selection sets or saved selection sets (e.g. combine sets, subtract one set from an another, find common Entities in multiple Selection sets, etc. as per function 6) to create new selection sets which can then be saved, activated and modified (per previously discussed functions 1 ,2, 3, 6), for operation with the GUI.
  • the non-transitory computer-readable medium can further comprise program code that is executable by the processing device for causing the processing device to perform a search operation with the GUI, and the results of the search (or some subset of the results of the search as per function 7) then can be saved as a selection set, and modified at will, and then can be activated later (as per previously discussed functions 1 ,2,3,7), for operation with the GUI.
  • the non-transitory computer-readable medium of can further comprise program code that is executable by the processing device for causing the processing device to perform arbitrary computation for the purposes of creating or modifying selection sets (as per function 8), which then can be saved as a selection set, and modified at will, then to activate the selection set later (as per previously discussed functions 1 ,2,3,8), for operation with the GUI.
  • the invention may be a method for improving GUI usability. This method may comprise the steps of creating a set of selected entities as a selection set, and to save the selection set under a name to form a saved selection set, and then choosing a saved selection set to activate the entities previously saved in the selection set so chosen, for operation with the GUI.
  • the method for improving GUI usability may further comprise adding or removing additional entities from the saved selection set.
  • the method for improving GUI usability may further comprise saving the so modified selection set, either with the name of the original saved selection set or a new name.
  • the method for improving GUI usability may further comprise performing set operations on one or more selection sets or saved selection sets (e.g. combine sets, subtract one set from an another, find common Entities in multiple Selection sets, etc. as per function 6) to create new selection sets which can then be saved, activated and modified (per previously discussed functions 1 ,2, 3, 6)
  • the method for improving GUI usability can further comprise performing a search operation with the GUI, and the results of the search (or some subset of the results of the search as per function 7) then can be saved as a selection set, and modified at will, and then can be activated later (as per previously discussed function 1 ,2,3,7), for operation with the GUI.
  • the method for improving GUI usability can further comprise performing arbitrary computation for the purposes of creating or modifying selection sets (as per function 8), which then can be saved as a selection set, and modified at will, then to activate the selection set later (as per previously discussed functions 1 ,2,3,8), for operation with the GUI.
  • the invention may be a system for generating a graphical user interface (GUI).
  • GUI graphical user interface
  • This system may comprise: a processing device; and a memory device including instructions that are executable by the processing device for causing the processing device to perform the following steps and/or any combination or sequence of the steps - create a set of selected entities as a selection set; saving the selection set under a name to form a saved selection set; choosing a saved selection set to activate the entities previously saved in the selection set so chosen, for operation with the GUI; modify a selection set (named or not); perform set operations on one or more selection sets or saved selection sets to create new selection sets; tie search capability such that the search results (or a subset of the results) can be saved as a named selection set; perform arbitrary computation on selection sets (named or not) to create new selection sets; and removing unneeded selection sets.
  • GUI graphical user interface
  • the invention may comprise a non-transitory computer- readable medium comprising program code that is executable by a processing device for causing the processing device to perform the following steps and/or any combination or sequence of the steps - create a set of selected entities as a selection set; saving the selection set under a name to form a saved selection set; choosing a saved selection set to activate the entities previously saved in the selection set so chosen, for operation with the GUI; modify a selection set (named or not); perform set operations on one or more selection sets or saved selection sets to create new selection sets; tie search capability such that the search results (or a subset of the results) can be saved as a named selection set; perform arbitrary computation on selection sets (named or not) to create new selection sets; and removing unneeded selection sets.
  • the invention may comprise a method for improving a GUI usability comprising the following steps and/or any combination or sequence of the steps - creating a set of selected entities as a selection set; saving the selection set under a name to form a saved selection set; choosing a saved selection set to activate the entities previously saved in the selection set so chosen; modify a selection set (named or not); perform set operations on one or more selection sets or saved selection sets to create new selection sets; tie search capability such that the search results (or a subset of the results) can be saved as a named selection set; perform arbitrary computation on selection sets (named or not) to create new selection sets; and removing unneeded selection sets.
  • the invention may be a graphical user interface (GUI) based method for managing selectable entities that comprises various combinations of the above functions, such as
  • the entities can comprise any of files and directories;
  • the GUI can be configured to receive user input by any of a touch interface, pointing device, and gesture recognition device, and voice recognition device.
  • the GUI can be configured to receive user input by any of a touch interface, pointing device, and gesture recognition device, and voice recognition device.
  • at least some of the entities are displayed by the computer processor on the GUI, and are selected by the human user using any of the touch interface, pointing device and gesture recognition device, and voice recognition device to designate at least some of the entities that are displayed by the computer processor on the GUI.
  • the at least one set operation may comprise any of set union, set intersection, and set inversion (i.e. set negation) operations.
  • d) Creating, using the computer processor, and the at least one set operation, and the first selection set, and the at least one different selection set, at least one new selection set.
  • an example of such management is to use the at least one new selection set is used to perform (often automatically perform) any of memory storage operations and network transmission operations upon any of files and directories in this at least one new selection set.

Abstract

A GUI and processor-based method for managing selectable entities such as files and directories. When the number of selectable entities in the GUI is large (e.g. over 50 entities) or if similar operations must be done repeatedly, the user experience can be drastically improved by generating and operating on sets of selectable entities. Users (such as a human user) start by picking a plurality of entities displayed on the GUI and save them in memory as a named selection set. Users operate the GUI to manipulate any such selection sets via various means (such as intersection, union, inversion/negation, or via user defined programs) until a selection set contains only the entities that the user wishes to operate on. The user then directs the GUI to automatically perform any of memory/storage operations and/or network transmission operations upon any files and directories in the selection set as desired.

Description

GUI BASED METHODS AND SYSTEMS FOR WORKING WITH LARGE
NUMBERS OF INTERACTIVE ITEMS
CROSS-REFERENCE TO RELATED APPLICATION(S)
[0001] This application is related to U.S. Application No. 16/940,197, filed July 27, 2020, which claims the benefit of and priority to U.S. provisional patent application 62/879,761 , filed July 29, 2019, both of which are incorporated herein by reference in their entireties.
BACKGROUND
[0002] GUI interfaces for computers have been in common use since the mid-1980s. One of the most fundamental (and earliest) of these GUIs shows the layout and location of files in the computer’s local storage in graphical form, using the adopted nomenclature “window” and “icons” where each icon can represent a single file or directory. Using a pointing device, the user selects the files to operate on (e.g., creates a selection set) by “clicking” on the icon. Once selected, the user then initiates the desired operation on the selected entity (in this case, a file or directory) such as copying or moving the selected files, or deleting them, or other operations.
[0003] Typically, the user manually selects multiple items (or entities) by clicking on them in sequence. Other common selection methods include “shift-click" methods, where a contiguous sequence of entities can be selected (or unselected) by clicking on the first entity, then clicking on the last item in the sequence, typically with another modifier (typically the "Shift" key). Individual entities can also be added or removed from the selections by “control-click", where another modifier is held active while individual entities are clicked, thus adding and removing them from the selection set.
[0004] Boolean selection methods: Boolean selection methods are also known in prior art. For example, Borovoy, US patent 7,769,741 teaches a portable browsing interface for information retrieval. Neale, US patent 7,552,395 teaches a graphical user interface for building Boolean queries and viewing search results. Such methods are commonly available as an “advanced” option on many databases, including the LISPTO patent search database.
SUMMARY
[0005] The invention is based, in part, on the insight that although prior art GUI selection methods work well when operating with at most a few dozen entities at a time. However, these prior art methods can become increasingly inadequate when the GUI is dealing with a large number of selectable entities (e.g. hundreds, thousands or more selectable entities). These prior art methods also become inadequate if similar tasks must be repeated across multiple entities. Similarly, although prior art Boolean search methods can operate with larger numbers of selectable entities, such methods tend to be difficult to use. The invention is based, in part, on the insight that it is not realistic, in an easy to use system, to expect the user to be able to manually add and remove, in an error-free manner, a few elements from a set comprising hundreds or thousands of different elements. This is because prior art GUI tend to operate in a way where a single errant click can reset the entire selection set. Such prior art GUI methods can be both frustrating to users, and can also be inefficient in terms of time and effort, particularly if the user must then repeat similar type selection-and-operate cycles, with slightly varying parameters, when the content of a folder directory changes, or when the user is operating in different directories.
[0006] The invention is also based, in part, on the insight that in order to better deal with the above issues (e.g., using prior art GUI methods to operate on a large number of items), improved GUI methods and systems are needed that add additional functions to the GUI.
[0007] The invention disclosed herein can be any of a method, system, or a non- transitory computer storage medium for computer code for a processor to implement the method and system versions of the invention. However, for simplicity, often the method embodiments of the invention will be provided as specific examples.
[0008] In some embodiments, the invention may be a graphical user interface (GUI) based method for managing selectable entities. This GUI method will typically comprise designating, using a GUI and a computer processor, from a plurality of selectable entities, a selection set comprising entities from plurality of selectable entities, and saving this selection set in computer memory as a first selection set. The method will also comprise designating, using this GUI and a computer processor, from this plurality of selectable entities, at least one different selection set comprising different entities from the plurality of selectable entities, and saving each at least one different selection set in computer memory as at least one different selection set. The method will also comprise selecting, using the GUI and a computer processor, at least one set operation to be performed on the first selection set and on the at least one different selection set.
[0009] The method will then use the computer processor, this at least one set operation, the first selection set, and the at least one different selection set, and produce at least one new selection set. The method can also use this least one new selection set to manage at least some of the various selectable entities.
[0010] Note that although the advantages of the invention become most apparent when working with a large number (hundreds or thousands or more) of selectable entities, it is not a requirement that a large number of entities exist for users to benefit from this invention. As will be shown below, the benefits of this invention can be demonstrated even when dealing with a much smaller (e.g. less than 100) number of entities as well.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Fig. 1 shows a GUI screenshot where the user has manually selected a single file.
[0012] Fig. 2 shows a GUI screenshot where the user is using a “Search” operation to find JPEG files.
[0013] Fig. 3 shows a GUI screenshot showing the result of the Fig. 2 Search operation. Note that Files other than JPEGs are hidden from view.
[0014] Fig. 4 shows using a GUI to save a Selection Set. Here the user has selected a subset of the search and is saving the subset as “Mockup JPEGs Only”.
[0015] Fig. 5 shows a detail of the selected “JPEGs Only” selection set previously selected in Fig. 4. [0016] Fig. 6 shows a GUI screenshot of the user changing the Active Selection (To “Mockup JPEGs Only”).
[0017] Fig. 7 shows a GUI screenshot of the result of the user Changing the Active Selection Set to “Mockup JPEGs Only”.
[0018] Fig. 8 shows a GUI screenshot where the user has set the "JPEGs Only" Selection set as Active.
[0019] Fig. 9 shows a GUI screenshot where the user is Subtracting "Mockup JPEGs Only" from the "JPEGs Only" selection.
[0020] Fig. 10 shows a GUI screenshot showing the result of the Subtract operation from Fig. 9. Note that the JPEGs in the “Mockup” Directory are no longer selected.
[0021] Fig. 11 shows a GUI screenshot showing the Screen for adding user-supplied functions/scripts to the GUI. These scripts can be used during the Filter operation to arbitrarily query the Objects and/or modify selection sets.
[0022] Fig. 12 shows the website youtube.com, as rendered by the present invention browser.
[0023] Fig. 13 shows Creating a new filter for Video Streams less than 1 GB in size.
[0024] Fig. 14 shows the result of the search in Fig. 13.
[0025] Fig. 15 shows creating a “filter chain”, according to function 8 (described below in Detailed Description Section).
[0026] Fig. 16 shows the result of applying the filter chain (shown on Fig 15) (an exemplar of function 8), thus creating a new selection set.
[0027] Fig. 17 shows a video game example, here a video game inventory screen, “Searching for Ores” is shown
[0028] Fig. 18 shows creating a new item set “Cheap Ores” for the video game of fig. 17.
[0029] Fig. 19 shows the result of creation of the “Cheap Ores” item set from fig. 18. [0030] Fig. 20 shows the result of visiting a shop, and selling all of the “Cheap Ores” from Fig. 19
[0031] Fig. 21 shows the result of visiting a different shop, and performing the operation of refining all “Expensive Ores”.
[0032] Fig. 22 shows the result of revisiting the first shop, and selling “Cheap Ores” and “Junk”.
[0033] Fig. 23 shows a first flow chart of an embodiment of the invention.
[0034] Fig. 24 shows a second flow chart of an embodiment of the invention. It focuses in on Box (E) first shown in Figure 23. Note that the large four-pointed arrow in the center of the figure is used to simply mean that steps (F) through (K) as well as (B1 ) and (B2) can be done in any order.
DETAILED DESCRIPTION
[0035] In addition to the discussion, additional description will also be made through referencing the various figures.
[0036] Definitions:
[0037] 1 . GUI - Graphical User Interface.
[0038] 2. Selectable Entity - The invention’s improved GUI operates on “entities”. Here the term “entity” is a description of, or a link to, or can denote, essentially any type of defined object. The object in turn can be a “lump” or “set” of digital data. The invention can be implemented using many kinds of GUIs, and the invention can operate on almost any kind of entities and associated objects. For example, a word processor type GUI operates on text entities, a spreadsheet type GUI operates on cells, rows, and column type entities, and a file browser type GUI operates on files and directory entities.
[0039] Typically, the entity that the GUI manipulates can also be “selected” by the user, either one entity at a time or multiple entities at a time, for additional manipulation and/or operation. For example, in a file browser, the underlying files and/or directories can be selected, by way of their linked entities, so that the user can copy, rename, move, or delete, one or more files/directories at the same time. For the purposes of this invention, we concern ourselves with the entities that are selectable by the user, irrespective of the GUI.
[0040] The exact method, system, and means for user entity selection is generally irrelevant to this invention. In this disclosure, the act of selecting an entity is occasionally designated as a “click” because a button click via a mouse or similar pointing device is one of the most common methodologies in use with computers for the user to select the entity, and its associated object, for one or more subsequent operations. However, this invention does not require that an actual clicking event occur. The main point is that something or someone has somehow caused the GUI to select one or more entities, and their associated objects, for subsequent operations.
[0041] Thus, for example, in some embodiments, the GUI may be configured to receive the user input by any of a touch interface, pointing device, gesture recognition device, or voice recognition device. These examples of specific devices, methods and systems are not meant to be exhaustive. For example, enhanced accessibility user interfaces used by individuals with disabilities or functional limitations, such as eye motion, tongue motion, breath activation, ECG sensing, nerve signal measurements, or even magnetic resonance imaging of the brain, can also be used to accept input by the user.
[0042] Here at least some of these entities are displayed by the computer processor on the GUI, and are selected (e.g. by using any of a touch interface and pointing device, gesture recognition device, voice recognition device, or other method) to designate at least some of the entities displayed by the computer processor on the GUI.
[0043] 2a. Activating - this disclosure also occasionally uses the term “activate” to mean that something or someone has caused an entity (or a set of entities) to become selected (or “clicked”, as per above in Definition 2). Note, however, that since “activation” is implied any time a user directs a computer to perform an operation, the term “activation” is generally not intended as a limitation unless otherwise so specified.
[0044] 3. Objects - the underlying data structure referred to by an entity. All entities are generally linked to an associated object, but not all objects may have a discoverable entity. For example, within a file browser GUI, entities typically refer to underlying objects such as files and directories. Note, however, that for a given GUI, some objects may be “hidden”, and thus may not have a visible or observable or discoverable linked entity.
[0045] This invention is primarily concerned with selectable entities, and generally ignores or is silent with respect to hidden or invisible objects. The actual object that the entity refers to becomes important when the user carries out an operation on the selected entities (see definition 4 below). Generally, this disclosure only distinguishes between the entity and its associated object when working with selections or selection sets. For our purposes, a selectable entity refers to those entity linked object or objects that are managed by the GUI.
[0046] Put alternatively, the entities provide the mapping or linkage between the GUI representation of the objects to the actual objects that the GUI operates on. Thus, when one or more entities are somehow selected (e.g. via a “click”) or activated through some other means, the GUI then has the links to the set of actual objects that the GUI can operate on. In other words, the entities describe or link to objects. In essence, objects are the embodiment of entities.
[0047] 4. Operation - A GUI typically provides various features and capabilities to operate on objects. Each GUI has different capabilities. For example, a file browser might offer the capability to copy, rename, move, delete files and directories. When such a capability of the GUI is used, this disclosure so describes it as “GUI operates on the selected entities.” Thus, this disclosure often uses the term entities to refer to their associated objects.
[0048] 5. Selection Set - simply, a set of selectable entities that have been “clicked on” or otherwise activated somehow. Most GUIs visually distinguish between unselected entities and the selected ones. The actual graphical or other type selection cues are not important for this invention, nor is the method, systems or devices used for the selection process.
[0049] 6. User - generally understood to be a human being operating a Graphical User
Interface (GUI). However, for the purpose of this Invention and the claims herein, all that is required is that a user is operating a GUI.
[0050] Although in some embodiments, the personhood or humanness of the user is not a limitation, in other embodiments, including some preferred embodiments, a human user is required. [0051] 7. Computer processor: Modern computers and computerized devices typically comprise at least one processor, often an x86, MIPS, Arm, or another type processor, that often may contain more than one processor core. Often such computerized devices may contain specialized co-processors, such as graphics co-processors, and other processor devices. Here, for simplicity, the term “computer processor” will be used to denote any and all of these combinations. Thus, in the alternative, the term “computer processor” can be understood as being equivalent to “at least one computer processor.”
[0052] As previously discussed, in some embodiments, the invention may be a graphical user interface (GUI) based method for managing selectable entities. This method can comprise using a GUI (often by human user input on the GUI) and a computer processor to designate, from a plurality of selectable entities, a first selection set comprising various entities from a plurality of selectable entities, and save this selection set in computer memory as a first selection set.
[0053] Further, according to the invention, the method will often also use this GUI (and often the same human’s input into this GUI) and usually the same computer processor, and this plurality of selectable entities to designate at least one different selection set comprising different entities from this plurality of selectable entities. This at least one different selection set will typically be saved in computer memory as at least one different selection set.
[0054] Further, according to the invention, the method will also use this GUI (again often by the same human user providing input to this GUI), and usually the same computer processor, to select at least one set operation to be performed on the first selection set and the at least one different selection set.
[0055] Based on this input, the invention will then typically automatically create, using this computer processor, the at least one set operation, the first selection set, and the at least one different selection set, at least one new selection set. The invention can then either automatically, or as designated by the user using the GUI, use this at least one new selection set to manage at least some of the selectable entities.
[0056] Put alternatively, the invention may be a computerized method, system or even a non-transitory computer-readable medium for improving GUI usability. The invention operates by performing one or more of the following eight operations: These eight operations or functions, which may be implemented according to various combinations and permutations, may thus be viewed as comprising important components of the present invention. These functions can include:
[0057] (1 ) Creating a set of elected entities, a selection set, and saving the selection set (e.g. “clicked state”) under a potentially user-chosen name (regardless of how that “click” or selection is achieved) to form a saved selection set. In other words, the selected entities are saved as a named set (name may be chosen by the user) which is persistent such that the GUI then can be shut down and when the GUI is restarted, the saved selection sets are still available. See function 2 and function 4 below.
[0058] (2) Choosing a saved selection set to activate (e.g. select) a previously saved selection set (i.e. it is as if the user has unselected all selected entities, and then freshly, manually, one-at-a-time, selected (or “clicked”) on each of the entities that were in the selection set).
[0059] (3) Manually modifying a current selection set (i.e. add or remove additional entities) and optionally saving the modified current selection set (as per function 1 ), either with the name of the original saved selection set or a new name.
[0060] Put alternatively, in some embodiments, the invention may further comprise adding or removing additional entities (often by input from a human user on the GUI) from any of the first selection set, the at least one different selection set, or the at least one new selection set, thereby creating any of a modified first selection set, a modified different selection set, and a modified new selection set.
[0061] Note also that in some embodiments, the invention may further comprise saving any of the modified first selection set, the modified different selection set, and the modified new selection set under any of their original names and their modified names.
[0062] (4) Loading saved selection sets from a previous session of the GUI.
[0063] (5) Optionally removing any unneeded saved selection sets. [0064] (6) Performing simple set operations on one or more selection sets or saved selection sets (e.g. combine sets, subtract one set from an another, find common Entities in multiple Selection sets) to create new selection sets which can then be saved per function 1 . These set operations are Boolean set operations, which are carried out on one or more named set of selections to compose more named selection sets. The most common ones are INTERSECTION (i.e. “AND"), UNION (i.e. “OR") SUBTRACT (i.e. “MINUS"), and INVERT (i.e. “NEGATE" or “NOT"). For example, the user “Bob” might create a selection set of files modified during the last week, and subtract from this first set the set of files that were modified by another user “Mary”.
[0065] Thus, according to the invention, the one or more set operations can comprise any of set union, set intersection, and set inversion (i.e. set negation).
[0066] Note that in set-theory terminology, “union”, “intersection”, and set inversion (also called “set negation”) form one possible “complete” group of set operations. A “complete group of set operations” means that you can express any set operation as a sequence of these more basic “complete” operations. For example, A exclusive-or B is ((A UNION B) INTERSECT (INVERT (A INTERSECT B))). Thus, for example, UNION, INTERSECTION, INVERT means also reads on an “exclusive-or” operation. Similarly, a setsubtract operation can be phrased in terms of the UNION, INTERSECT, INVERT set operations. Thus, for example, the exclusive-OR and set subtraction operations are also claimed.
[0067] (7) Tying the results of the search capability of the GUI automatically into a
Selection set and a saved selection set, so that additional changes to the selection set and saved selection set can be made later, at the convenience of the user.
[0068] Put alternatively, in some embodiments, the GUI and the computer processor can be further configured to search the plurality of selectable entities for at least one GUI selectable search parameter. Here, the invention can further enable the user to use the GUI to define at least one GUI selectable parameter. The invention can then use the computer processor, this at least one GUI selectable parameter, and the plurality of selectable entities to determine any of the first selection set and the at least one different selection set. [0069] (8) Further, in some embodiments, the invention may carry out arbitrary computationally driven decision making about the entities present in a selection set or saved selection set, either through a search feature/operation or a more in-depth method, such as a script or program, to create new selection sets which can then be saved per function 1 . In the cases where set operations, paired with search operation are insufficient or imprecise, the invention allows additional reasoning about the individual entities in a selection, such that intelligent choices can be made by the processor about which entities to add or remove from a selection set. For example, the user might wish to add to (or remove from) a selection set the list of files that contain a specific marker or markers, allowing for more complex machine reasoning (such as if-then-else) about the pending entities. A possible list of traits, that may become part of the processor’s decision process for selection inclusion or exclusion, includes the size of the file, date of last modification, date of creation, the file format, custom labels, user comments about the file etc. The intention here is that this last item fully allows arbitrary computation about the entities and objects under consideration.
[0070] See Fig. 23 and Fig. 24 for flow charts showing an embodiment of the invention.
[0071] Note that in some embodiments, the invention further can use the GUI and the computer processor, to create or select at least one script or program, thereby determining at least one selected script or program. Here, this at least one script or program can be configured to accept this at least one new selection set as any of input or output parameters for this at least one script or program. The invention can then use the computer processor to execute this script according to the at least one new selection set.
[0072] The user can continue adding/removing entities from any of the named selection sets created earlier. Thus, saving these modified selection sets, activating these selection sets, and operating on these selection sets (i.e. carry out a GUI operation on the selected entities, as per definition 4), as well as creating new selection sets by various means described in the functions are all possible.
[0073] When a selection set is being saved or loaded, the actual objects that the entity refers to (e.g. the linked files/directories objects) need not themselves be saved or loaded. Often, what is being saved is just enough information that signifies within the GUI that an entity is selected. That is, the selection sets are made so that the user can choose to do an operation on the selected entities (and the underlying objects) at a later point in time, as well as add/subtract new entities to the selection set to fine-tune which entities are selected for an operation.
[0074] With these capabilities in place, it becomes apparent that the task of choosing which entities to select for a GUI operation becomes significantly more powerful and less amenable to human-driven errors that can frustrate efforts to work with a large number of entities. In other words, these functions, demonstrated in the figures below, are also easy to use.
[0075] Exemplary Embodiments
[0076] As discussed above, Figures 1 through 10 show an implemented browser GUI with the functions (i.e. various aspects of the present invention) built-in. In this case, the selectable entities (and their underlying objects) are files and directories.
[0077] Note that in some embodiments, as will be discussed shortly in more detail, the entities can comprise any of files and directories. Here, for example, the at least one new selection set can be used to perform any of memory storage operations and network transmission operations upon any of the underlying files and directories objects in this at least one new selection set.
[0078] However, in some embodiments, the entities may comprise of any object, manipulatable by the GUI, that can be selected by any methodology available to the GUI.
[0079] Note that Figures 1 through 10 demonstrate the invention in a file browser GUI, and thus the "selectable entities”, as well as the underlying objects, are files and directories. However, there are many kinds of GUIs and many kinds of selectable entities.
[0080] Figure 8-10 detail set operation on selection sets. Fig. 11 shows how a scripting window (e.g. for function 8) may be used. Figures 12 through 16 show the application of the Invention where the selectable entities are uniform resource locators (URLs) for video streams within a web browser, i.e. they are not files in a local directory. Figures 17 through 22 show another example of the Claims in action where the Selectable Entities are Inventory items in a Video Game. In the figures, areas of specific interest, such as selected entries or discussed functions, are often indicated with an arrow. [0081] In some embodiments, the entities can comprise items in a video game, where this video game can be configured to be run by any of one or more computer processors and/or at least one remote server.
[0082] Further, in some embodiments, the entities can comprise any of remotely hosted audio and video streams. Here the GUI and the computer processor can be further configured with web browser functionality, and the first selection set, the at least one different selection set, and the at least one new selection set can comprise sets comprising any of these remotely hosted audio and video streams.
[0083] See Fig. 23 and Fig. 24 for flow charts showing an embodiment of the invention.
[0084] Examples of these operations with a File Browser GUI.
[0085] Fig. 1 shows a File Browser GUI with a single file manually selected (i.e. “clicked” by the user. This single selected file is now an unnamed selection set containing one entity. The user can then choose an operation on this selected entity at any time.
[0086] Fig. 2 shows using a search capability of the GUI where the user is searching for JPEG files. Clarification: In the nomenclature used by the GUI, a “Filter” is used to “filter” the Entities for the desired items, i.e. Filter is-a Search.
[0087] Fig. 3 shows the result of the search. Notice that files other than JPEGs are hidden from view. This tying of search results to selection is a crucial function of this invention. Typically, searches are for finding entities with a specific trait. This function allows for the result of multiple searches to be saved as selection sets, allowing for the user to more efficiently generate the desired selection sets by incorporating multiple searches, manual modification as well as arbitrary computation. Examples of functions 5, 7 and 8 are below.
[0088] In Fig. 4, the user is creating (and saving) a newly named selection set from JPEG files in the Mockup directory. In this case, the user has manually selected (or clicked on) the rows that signify “Mockup/” and is creating (and saving) the selected rows. Note that the files themselves are not being saved here (they are already there!). What is being saved is the information that signifies that JPEG files in the Mockup directory have been selected (e.g. “clicked on”) by the user, and that the selection set is given a name for future use and reference by the user. The circled button near the top (resembling a floppy disk) was pressed to activate the shown dialog.
[0089] In Fig. 5, the user changed the active selection set to “JPEGs Only”. Notice that JPEGs in the Mockup directory, as well as JPEGs outside the Mockup directory, are selected.
[0090] In Fig. 6, the user is changing the active selection set to “Mockup JPEGs Only”. The button indicated by the circle and arrow near the top was pressed by the user to activate the shown dialog.
[0091] Fig. 7 shows the result of the operation in Fig. 6. Notice that JPEGs in Mockup directory are still selected, but other JPEG files are no longer selected, i.e. the selections have been reset to be identical to that of Fig. 4. (For example, note that "Tree Browser.jpg" is no longer selected)
[0092] As stated earlier, the selection set is not the fi les/d irectories, but rather is a type of short-lived ephemeral metadata pertaining to the “clicked” state of the individual Entity (which is, in this case, a row element that contains some detailed information about the file). The process of creating, saving, naming, modifying, or deleting the selection sets does not actually impact the selected entities (in this case files and directories) at all. What is being saved is metadata pertaining to whether or not a particular entity was selected, as well as the given name for this set. The underlying files/directories objects are only modified when the user chooses to carry out an operation on the selected entities that link to these underlying objects.
[0093] In other words, once the user has clicked/selected at least one file or directory (which is automatically a selection set (of size 1 ), the user can choose to operate on the selected entities. In the early Figures 1 -7, such operations are what you might expect from a file browser - i.e. copy/move/rename/delete/make new directory etc.
[0094] Figures 8-10 demonstrate the power of set operations in modifying Selections
[0095] Fig. 8 shows the “JPEGs Only” selection set is activated. Notice that the JPEG files (including the ones in the Mockup directory) are selected. [0096] Fig. 9 shows the initiation of a set subtract operation, where the “Mockups JPEGs Only” selection set is being chosen to subtract from “JPEGs Only”. The button indicated by the circle and arrow (resembling a "minus sign") was pressed by the user to activate this function.
[0097] Fig. 10 shows the resulting selections, where the JPEGs NOT in the Mockup directory are still selected. Note that the resulting selection has been saved to a new selection set named “Subtract”. Other operations that are featured are the expected common set operations like “intersect” (find common entities), “union” (add entities from one set to another), “select all” (select all entities that are in the current view), “exclusive or” (all entities in one set but not in another), and “invert” (flip the selection status of all entities in the current view). Note that the current view, as demonstrated prior as well in the following figures, can be the default view, or the result of a filter operation. One advantage of the invention’s improved functionality here is that with set operations, and the ability to have multiple named selection sets, the invention provides greater ease-of-use and improved file management capability.
[0098] Fig. 11 shows the interface in the GUI where the user can supply a program or script. One of the uses of these scripts is during the filter process (i.e. function 8). Here, the script can query the objects (i.e. what the selectable entities link or refer to). The invention can use this information to make choices about which entities are found as part of the search/filter process, as well as directly modify selection sets. The scripts can also be used to program (implement) a wider range of actions that might otherwise be difficult to compose directly with the GUI.
[0099] Figures 12 - 16 show an Embodiment where the selectable entities are URLs (uniform resource locators) for video streams within a web browser (e.g., the underlying objects are not merely files in a local directory). In this case, the operations involve video URL operations, such as play videos in sequence, etc.) These figures are also screenshots of the same software used to generate the prior figures.
[0100] Fig. 12 shows the front page of a popular website youtube.com as rendered by the invention’s browser in detail view. The actual URLs and file sizes have been intentionally mangled for diagram purposes. [0101] Fig. 13 shows a filter where the user is selecting the video streams that are less than 1 GB in size.
[0102] Fig. 14 shows the results of the search where a bunch of unneeded URLs (these mostly are for category pages) are showing up.
[0103] Fig. 15 shows an exemplar of the invention’s function 8, which is the catch-all capability where additional computation can be done to drill down on the specific desired entities. In this example, the user has modified the search to be a chain of two filters, where the first filter selects all URLs that point to video streams less than 1 gigabyte (GB) and the second filter selects those with sizes greater than 10,000,000 bytes.
[0104] Note that chaining filters (or searches) is not the only way to implement function 8. In some embodiments, this aspect of the invention can also be used to support arbitrary code (i.e. a script or program). This code can be inserted into the filter-select loop, which can then direct the computer processor to automatically query the entities (and objects) in arbitrary ways to reach a decision on selection.
[0105] In some embodiments, essentially any arbitrary user-supplied function can be invoked during the filter-selection loop. Thus, in some embodiments, function 8 can be “computationally complete”. In other embodiments, however, it may be useful to limit function 8 to a “sandbox” subset of functions or otherwise limit the functionality of function 8 to protect the user from inadvertent mistakes, or for higher security.
[0106] Fig. 16 shows the results of a filter chain, where meaningful video URLs of the specified sizes are shown. Several URLs are then manually selected and is saved as a selection set named “To look at later”.
[0107] Figures 17-22 shows a hypothetical GUI for a video game. In these figures, the selectable entities are inventory items of the game character. A “game character”, in this case is the fictional person, in-game, that is under the control of the user (i.e. player) of the video game. Inventory items are the ephemeral objects within the video game that the game character has somehow acquired through play. [0108] Fig. 17 shows an inventory screen where the user (or player, in this case) is searching for metallic ores. Here the user has highlighted 3 ores. Copper ores and tin ores are shown as selected. One of the three selected ores is not shown in the Inventory listing.
[0109] In Fig. 18, the “create new item set” button is Active. Thus, the player has elected to create a new “Item Set”, i.e. “selection set” in this milieu. This item set is called “cheap ores”.
[0110] Fig. 19 shows the results of the creation of this item set. On the bottom left, the three selected entities (i.e. inventory items, in this milieu) are shown. They are copper, tin, and lead ores. Figures 16-18 are exemplars of function 1 ,2,3 and 7.
[0111] In Fig. 20, some time has passed, presumably, and the game character under the player’s control is visiting some shop, where some of the items for sale in this shop (weapons, in this case) is listed in the main browser on the upper right. Also, the previously generated “cheap ores” item set has been selected, and the user is executing a sale of these items in “cheap ores.” Notice that there is a second item set, called “expensive ores” which the user has created (sometime between Fig. 19 and Fig. 20).
[0112] In Fig. 21 , the game character is visiting a different shop, where the action being contemplated is to refine the ores in the “expensive ores” to produce ingots, presumably for some future crafting use.
[0113] It is reasonable to assume that like in Fig. 20 and 21 , if the game character receives more items that match the item sets as the game progresses further, the shops can then be revisited, and the same operations on the objects (e.g. sell “cheap ores” in Fig. 20 and “refine ores” in Fig. 21 ) can be re-executed by the player without needing to recreate the selection sets. This is exactly what happens in Fig. 22, where the player is revisiting the first shop and selling off both “cheap ores” and another item set called “junk”. This set has things like “broken broom” (i.e. is “junk”, as collected for cheap salvage in many different video games).
[0114] Note that even though this hypothetical III in Figures 17-22 is inspired by video games, this kind of interface can easily be adapted to the needs of many businesses that buy and sell commodities of all kinds. [0115] Flow charts:
[0116] Figures 23 and 24 describe the operational flow diagram of the GUI, as experienced by the user, in one embodiment of the invention. As shown in Fig. 23, in most GUIs, the user starts the GUI (Box A) and starts work by choosing some entities to operate on (Boxes B1 and B2). Once the user deems that the selected entities are exactly the ones desired (as well as having no undesired entities), the user initiates the work by invoking the desired operation on the selected entities (Boxes (C) and (D)). Box (B1 ) and Box (B2) are where users typically would spend the majority of time interacting with the GUI. As the number of selectable entities increase, the task of selecting exactly the desired entities for operation becomes a much more burdensome task.
[0117] In a GUI that implements a preferred embodiment of this disclosure (examples of such an implementation were presented in the previous figures), the user has the option of employing the various functions described herein to enhance productivity, especially if the number of selectable entities is unwieldy.
[0118] Figure 23 shows the top-level view of the operational flow diagram of the GUI.
[0119] Once some number of entities are selected, the user must decide whether all of the desired entities were chosen and that no undesired entities remain in the selection set. If the answer is YES, then the user initiates the desired operation via box (D). If the answer is no, the user would jump to box (E). Box (E) is described in detail in the next figure.
[0120] Boxes (A) through (D) show a typical workflow when the number of selectable entities are small and manageable. However, if the number of selectable entities is large or the exact desired entities must be discovered, then the enhancements as diagramed in box (E) can greatly enhance productivity, as well as reduce user frustration.
[0121] Fig. 24 expands on the box (E) shown in Fig 23. This figure shows how the invention can then operate on one or more selection sets. If the selection set is empty, the invention then can switch to a saved selection set (Box G). If the selection set is not empty, the invention can then save the selection set to a given name supplied by a user. As per B1 and B2, the user has the option to manually add or delete entities in the current selection set, and/or use set operations on one or more named selection sets that were previously described (Box H). The user can also direct the invention to run a search or filter operation (Box F), and/or refine a selection set by composing one or more filter/search results and/or selection sets, or by Boolean operations or other in-depth method, such as calling user defined programs (Box I). The user can also direct the invention to remove unwanted selection sets (Box J). As stated earlier, the large 4-point arrow in the figure signifies that these steps can be done in any order to produce the exact set of entities for the desired operation. In other words, steps (F) through (K) and (B1/B2) can be done in any order for these purposes.
[0122] When the user is done, the invention will have a set of selected entities (Box C), from which the user can invoke the desired operation on said selected entities.
[0123] Additional discussion:
[0124] The present invention may thus comprise an improved GUI equipped to implement at least some of the various combinations of the previously described functions 1-8, here described again in the list below.
[0125] (1 ) Creating a set of selected entities, (a selection set), and to save the selection set as a named selection set.
[0126] (2) Activating (if needed) a selection set previously saved
[0127] (3) Optionally manually modifying the current selection set
[0128] (4) Loading any saved selection sets from a previous session of the GUI,
[0129] (5) Removing any unneeded saved selection sets,
[0130] (6) Performing set operations on one or more selection sets or saved selection sets,
[0131] (7) Tying the results (or a subset of the results) of any search capability of the
GUI automatically as a selection set.
[0132] (8) Carrying out arbitrary computationally driven decision making about the entities present in a selection set or saved selection set, either through the search feature or a more in-depth method, such as a script or program, to create new selection sets, or modify existing selection sets, within the context of a Graphical User Interface. [0133] As stated earlier, there are many kinds of GUIs and many kinds of selectable entities. In some embodiments, the invention described herein may be fully independent of the GUI technology, such as the GUI’s "look-and-feel”, and the specific selection technique used by the GUI. Some embodiments of the invention may also be fully independent of the minutiae of GUI operations - (e.g. whether a "button” is pressed by the user to change the selection set, or whether there is a “drop down menu”, or whether there is a context-sensitive "pop-up window” that the user engages with). Thus, in some embodiments of the invention, the actual presentation, the visual cues, the iconography, the display devices and their related methods and systems, the interactive devices and their methods and systems being used with (or embody) the GUI need not impact the applicability of the invention. In other words, at least some embodiments of this invention may apply to all types of GUIs, no matter the specific GUI form or function.
[0134] It also follows that the location, nature, devices, systems, principles, and methods that pertain to the embodiments of the Entities (i.e. Objects in Definition 3) that are represented by a GUI as selectable entities may not always be relevant to the application of this invention. In other words, some embodiments do not distinguish between the machine that the GUI is executing on and the location of the objects that the entities refer to. In both groups of figures 1 -11 and 12-16, the GUI was executing on a common Windows server but the objects (files and directories) in Figures 1 -11 were hosted on a different (Linux) server, but it could just have easily been hosted on the same Windows PC that the GUI was executing on. It could also just as easily be the reverse (i.e. GUI executes on a Linux server, but the objects are stored at a different Windows server). The objects in figures 12-16 that the URLs refer to are actually hosted at a remote website youtube.com but it could just as easily have been hosted somewhere else. For example, in some embodiments, the GUI could initiate network transmission of the objects in question, as well as modify the objects through memory storage operations, executed either locally (if the objects are local), or through some method of communication between one or more servers (if the objects are not locally accessible). In some preferred embodiments, the operations featured by the GUI for managing/manipulating the objects may not always be relevant to the functions (described above) on the selectable entities that a user may elect to invoke. [0135] Some embodiments of the invention may also apply regardless of the actual purpose, or role of the GUI (e.g., whether or not the GUI is a file browser, a video game, a web browser, or a custom tool for managing robots, cars, or whether it’s a tool for executing just-in-time inventory management for factories and warehouses.
[0136] Some embodiments of the invention may also apply regardless of the extra feature sets (e.g. functions) employed by the GUI. For example, the GUI used to generate Figures 1 -7 includes some ease-of-use shortcuts (e.g. results of a search is automatically saved as a selection set of the same name). Some embodiments of the invention may also apply regardless of whether such ease-of-use shortcuts are featured. Here, the key issue is if the GUI offers, to the user, the capability to create selection sets (through various means), save the selection sets (with names), load the selection sets sometime later, manipulate the selection set as needed through various means, and finally operate on the selection sets as desired.
[0137] The GUI tool pictured in the Figures may be implemented using computation time-efficient, as well as storage-space efficient methods and techniques for representation of the selection set as well as other components of the GUI. Thus, some embodiments of the invention apply regardless of the efficiency (or the lack there-of) of the underlying systems and methods that implement the named selection set mechanism and other components of the GUI.
[0138] It should thus be evident that the present invention is generally applicable across a variety of situations, and also improves upon the usability of prior art GUI when the user must deal with a significant number of selectable entities.
[0139] Sets of selection sets:
[0140] In some embodiments, including some preferred embodiments, the invention may be configured so that selection sets themselves may also be selectable entities. Thus, the GUI features discussed herein for selectable entities can also be used to manipulate selection sets in such embodiments. For example, after Mary goes on vacation, Bob may want to find the answer to "what were the selection sets that Mary created yesterday?" and then continue to operate the GUI, continuing where Mary left off. In essence, the capabilities built-in to the GUI to find (e.g. via search or filter), group (i.e. create named selection sets for later use), and manipulate can also be applied to the selection sets as well. This can be especially useful if the number of selection sets created by the users are also large.
[0141] The invention can implement these sets-of-selection sets in a variety of ways. For example, In the example GUI implementation that generated Figures 1 -16, these sets of selection-sets were implemented in the same way as selection sets - i.e. it has a name (chosen by the user) and a list of entities. In this case, these entities can refer to either selection sets or other selectable entities. In other words, these set-of-sets can themselves be selection set for all intents and purposes. We can temporarily refer to these specifically as sets-of-selection-sets for clarification purposes, but in our GUI implementation of an embodiment, the software need not distinguish between the two.
[0142] In some embodiments, these sets-of-selection-sets can also carry around (e.g. be linked with) additional metadata similar to the metadata associated with the underlying objects (such as time of creation, and the user-ID of the creator). Additional types of user provided metadata may also be linked with the sets-of-selection-sets. Such additional metadata can also be searchable by the same search/filter interface in the GUI (i.e. Function 7). Boolean operations (i.e. functions 6), as well as filter composition (i.e. exemplar of function 8) can also be implemented so as to work on both kinds of selection sets.
[0143] There need not be a further hierarchy, such as sets of sets of selection sets, however. This is because in the present implementation, the selection-set abstraction methods can be implemented to also ontologically include sets-of-selection-sets. The GUI operations that create, search, manipulate (e.g. Boolean operations and compositions as well as arbitrary manipulation via user supplied programs), save (i.e. for later use), load (from a prior session) can be configured to all operate in the same manner for both types.
[0144] There are certain ease-of-use features that can distinguish between selection sets that only contain top level selectable entities versus ones that contain references to other selections sets in this exemplar GUI implementation, but they do not add additional capability other than saving the user some steps in certain situations. [0145] It is also important to note that in some embodiments, function number 8 (i.e. the ability to invoke an arbitrary user supplied program) can also be configured to include the ability to reason about sets of selection sets. Thus, in a preferred embodiment of the invention, the system is configured so that the concept of sets of selection sets is ontologically and epistemologically included within Function 8.
[0146] Further GUI discussion:
[0147] The invention may fully apply regardless of the kinds of selectable entities, the objects that the entities refer to, the actual GUI technology being used, the "look-and-feel” and the actual devices or methods or systems being used to select an entity within the GUI.
[0148] As previously discussed, a variety of different devices and methods may be used to convey user intentions to the system’s GUI and computer processor. This includes touch devices, gesture recognition devices (either by visual or by some other means such as ultrasound or radar, voice recognition devices, and pointing devices (including joysticks, mouse, trackball), brain or neural scanning devices (where measured electrical impulses from a nerve or the measured activity of the user’s brain are used to control the GUI) can also be potentially used as input devices for accepting commands from a user.
[0149] In some embodiments, GUI display may also comprise head mounted augmented reality or virtual reality displays. Additionally, holographic displays where three- dimensional images, text and other visual iconographies are rendered, either in stereoscopic format (e.g. where slightly varying images are rendered to each visual sensor (e.g. the human eye) to simulate the presence of a three-dimensional object are also contemplated. Such 3D displays may operate directly in open air (such as via interference of laser beams). Such 3D displays may also be implemented by other means such as a multitude of thin light emitting rods being raised/lowered/moved about in a darkened space quickly).
[0150] In a preferred embodiment, this invention and claims herein are intended for scenarios where a user interacts with a GUI.
[0151] Thus in a preferred embodiment, the GUI is configured to accept at least designating the selectable entities, the different entities, and selecting at least one set operation as input from a human user, and at least designating the selectable entities, the different entities, and selecting the at least one set operation is done by the human user using the GUI.
[0152] Further general discussion.
[0153] In some embodiments, the invention can be a system for generating a graphical user interface (GUI), the system comprising: a processing device; and a memory device including instructions that are executable by the processing device for causing the processing device to create a set of selected entities as a selection set, and to save the selection set under a name to form a saved selection set, and then choosing a saved selection set to activate the entities previously saved in the selection set so chosen, for operation with the GUI.
[0154] In some embodiments, the memory device can further include instructions that are executable by the processing device for causing the processing device to add or remove additional entities from the saved selection set.
[0155] In some embodiments, the memory device can further include instructions that are executable by the processing device for causing the processing device to save the so modified selection set, either with the name of the original saved selection set or a new name.
[0156] In some embodiments, the memory device can further include instructions that are executable by the processing device for causing the processing device to perform set operations on one or more selection sets or saved selection sets (e.g. combine sets, subtract one set from an another, find common entities in multiple selection sets, etc. as per function 6) to create new selection sets which can then be saved, activated and modified (per previously discussed functions 1 ,2,3,6), for operation with the GUI.
[0157] In some embodiments, the memory device can further include instructions that are executable by the processing device for causing the processing device to perform a search operation with the GUI, and the results of the search (or some subset of the results of the search as per function 7) then can be saved as a selection set, and modified at will, and then can be activated later (as per previously discussed functions 1 ,2,3,7), for operation with the GUI. [0158] In some embodiments, the memory device can further include instructions that are executable by the processing device for causing the processing device to perform arbitrary computation for the purposes of creating or modifying selection sets (as per function 8), which then can be saved as a selection set, and modified at will, then to activate the selection set later (as per previously discussed functions 1 ,2,3,8), for operation with the GUI.
[0159] In some embodiments, the invention may be a non-transitory computer- readable medium comprising program code that is executable by a processing device for causing the processing device to create a set of selected entities as a selection set, and to save the selection set under a name to form a saved selection set, and then choosing a saved selection set to activate the entities previously saved in the selection set so chosen, for operation with the GUI.
[0160] In some embodiments, this non-transitory computer-readable medium can further comprise program code that is executable by the processing device for causing the processing device to add or remove additional entities from the saved selection set.
[0161] In some embodiments, the non-transitory computer-readable medium can further comprise program code that is executable by the processing device for causing the processing device to save the so modified selection set, either with the name of the original saved selection set or a new name.
[0162] In some embodiments, the non-transitory computer-readable medium can further comprise program code that is executable by the processing device for causing the processing device to perform set operations on one or more selection sets or saved selection sets (e.g. combine sets, subtract one set from an another, find common Entities in multiple Selection sets, etc. as per function 6) to create new selection sets which can then be saved, activated and modified (per previously discussed functions 1 ,2, 3, 6), for operation with the GUI.
[0163] In some embodiments, the non-transitory computer-readable medium can further comprise program code that is executable by the processing device for causing the processing device to perform a search operation with the GUI, and the results of the search (or some subset of the results of the search as per function 7) then can be saved as a selection set, and modified at will, and then can be activated later (as per previously discussed functions 1 ,2,3,7), for operation with the GUI.
[0164] In some embodiments, the non-transitory computer-readable medium of can further comprise program code that is executable by the processing device for causing the processing device to perform arbitrary computation for the purposes of creating or modifying selection sets (as per function 8), which then can be saved as a selection set, and modified at will, then to activate the selection set later (as per previously discussed functions 1 ,2,3,8), for operation with the GUI.
[0165] In some embodiments, the invention may be a method for improving GUI usability. This method may comprise the steps of creating a set of selected entities as a selection set, and to save the selection set under a name to form a saved selection set, and then choosing a saved selection set to activate the entities previously saved in the selection set so chosen, for operation with the GUI.
[0166] In some embodiments, the method for improving GUI usability may further comprise adding or removing additional entities from the saved selection set.
[0167] In some embodiments, the method for improving GUI usability may further comprise saving the so modified selection set, either with the name of the original saved selection set or a new name.
[0168] In some embodiments, the method for improving GUI usability may further comprise performing set operations on one or more selection sets or saved selection sets (e.g. combine sets, subtract one set from an another, find common Entities in multiple Selection sets, etc. as per function 6) to create new selection sets which can then be saved, activated and modified (per previously discussed functions 1 ,2, 3, 6)
[0169] In some embodiments, the method for improving GUI usability can further comprise performing a search operation with the GUI, and the results of the search (or some subset of the results of the search as per function 7) then can be saved as a selection set, and modified at will, and then can be activated later (as per previously discussed function 1 ,2,3,7), for operation with the GUI. [0170] In some embodiments, the method for improving GUI usability can further comprise performing arbitrary computation for the purposes of creating or modifying selection sets (as per function 8), which then can be saved as a selection set, and modified at will, then to activate the selection set later (as per previously discussed functions 1 ,2,3,8), for operation with the GUI.
[0171] In some embodiments, the invention may be a system for generating a graphical user interface (GUI). This system may comprise: a processing device; and a memory device including instructions that are executable by the processing device for causing the processing device to perform the following steps and/or any combination or sequence of the steps - create a set of selected entities as a selection set; saving the selection set under a name to form a saved selection set; choosing a saved selection set to activate the entities previously saved in the selection set so chosen, for operation with the GUI; modify a selection set (named or not); perform set operations on one or more selection sets or saved selection sets to create new selection sets; tie search capability such that the search results (or a subset of the results) can be saved as a named selection set; perform arbitrary computation on selection sets (named or not) to create new selection sets; and removing unneeded selection sets.
[0172] In some embodiments, the invention may comprise a non-transitory computer- readable medium comprising program code that is executable by a processing device for causing the processing device to perform the following steps and/or any combination or sequence of the steps - create a set of selected entities as a selection set; saving the selection set under a name to form a saved selection set; choosing a saved selection set to activate the entities previously saved in the selection set so chosen, for operation with the GUI; modify a selection set (named or not); perform set operations on one or more selection sets or saved selection sets to create new selection sets; tie search capability such that the search results (or a subset of the results) can be saved as a named selection set; perform arbitrary computation on selection sets (named or not) to create new selection sets; and removing unneeded selection sets.
[0173] In some embodiments, the invention may comprise a method for improving a GUI usability comprising the following steps and/or any combination or sequence of the steps - creating a set of selected entities as a selection set; saving the selection set under a name to form a saved selection set; choosing a saved selection set to activate the entities previously saved in the selection set so chosen; modify a selection set (named or not); perform set operations on one or more selection sets or saved selection sets to create new selection sets; tie search capability such that the search results (or a subset of the results) can be saved as a named selection set; perform arbitrary computation on selection sets (named or not) to create new selection sets; and removing unneeded selection sets.
[0174] In another embodiment, the invention may be a graphical user interface (GUI) based method for managing selectable entities that comprises various combinations of the above functions, such as
[0175] a) Designating, using a GUI and a computer processor, from a plurality of selectable entities, a selection set comprising entities from the plurality of selectable entities, and saving this selection set in computer memory as a first selection set. Here the entities can comprise any of files and directories;
[0176] b) Designating, using this GUI and a computer processor, from the plurality of selectable entities, at least one different selection set comprising different entities from this plurality of selectable entities, and saving each at least one different selection set in computer memory as at least one different selection set.
[0177] c) Selecting, using the GUI and the computer processor, at least one set operation to be performed on this first selection set and the at least one different selection set. Here the GUI can be configured to receive user input by any of a touch interface, pointing device, and gesture recognition device, and voice recognition device. Here at least some of the entities are displayed by the computer processor on the GUI, and are selected by the human user using any of the touch interface, pointing device and gesture recognition device, and voice recognition device to designate at least some of the entities that are displayed by the computer processor on the GUI. Note further that the at least one set operation may comprise any of set union, set intersection, and set inversion (i.e. set negation) operations. [0178] d) Creating, using the computer processor, and the at least one set operation, and the first selection set, and the at least one different selection set, at least one new selection set.
[0179] e) Using (often automatically according to an operation designated by the user on the GUI) at least one new selection set to manage at least some of the selectable entities. Here, an example of such management is to use the at least one new selection set is used to perform (often automatically perform) any of memory storage operations and network transmission operations upon any of files and directories in this at least one new selection set.

Claims

1. A graphical user interface (GUI) based method for managing selectable entities, said method comprising: designating, using a GUI and a computer processor, from a plurality of selectable entities, irrespective of entity location or display status, a selection set comprising entities from said plurality of selectable entities, and optionally saving, in a recallable manner, said selection set in computer memory as a first selection set; designating, using said GUI and a computer processor, from said plurality of selectable entities, at least one different selection set comprising different entities, irrespective of entity location or display status, from said plurality of selectable entities, and optionally saving, in a recallable manner, each said at least one different selection set in computer memory as at least one different selection set; selecting, using said GUI and a computer processor, at least one set operation to be performed on said first selection set and said at least one different selection set; creating, using said computer processor, said at least one set operation, said first selection set, and said at least one different selection set, at least one new selection set; and using said at least one new selection set to manage at least some of said selectable entities, irrespective of entity location or display status.
2. The method of claim 1 , further comprising adding or removing additional entities from any of said first selection set, said at least one different selection set, or said at least one new selection set, thereby creating any of a modified first selection set, a modified different selection set, and a modified new selection set, irrespective of location or display status of the additional entities, said first selection set, said at least one different selection set, or said at least one new selection set.
3. The method of claim 2, further comprising loading or saving or removing any of said modified first selection set, said modified different selection set, and said modified new selection set under any of their original names and their modified names, irrespective of location or display status of said modified first selection set, said modified different selection set, or said modified new selection set.
4. The method of claim 1 , wherein said at least one set operation comprises any of set union, set intersection, and set inversion (i.e. set negation).
5. The method of claim 1 , wherein said GUI and said computer processor are further configured to search said plurality of selectable entities, irrespective of location or display status, for at least one GUI selectable search parameter; further using said GUI to define at least one GUI selectable parameter, and using said computer processor, said at least one GUI selectable parameter, and said plurality of selectable entities to determine any of said first selection set and said at least one different selection set, irrespective of location or display status.
6. The method of claim 1 , further using said GUI and said computer processor to create or select at least one script or program irrespective of location or display status of script or program, thereby determining at least one selected script or program; said at least one script or program configured to accept said at least one new selection set as any of input or output parameters for said at least one script or program; and using said computer processor to execute said script according to said at least one new selection set.
7. The method of claim 1 , wherein said entities comprise any of files and directories; and wherein said at least one new selection set is used to perform any of memory storage operations and network transmission operations upon any of files and directories in said at least one new selection set.
8. The method of claim 1 , wherein said entities comprise any of remotely hosted audio and video streams, said GUI and said computer processor are further configured with web browser functionality, and said first selection set, said at least one different selection set, and said at least one new selection set comprise sets comprising any of said remotely hosted audio and video streams.
9. The method of claim 1 , wherein said entities comprise items in a video game, said video game configured to be run by one selected from the group including a computer processor, a computer processor and at least one remote server, and at least one remote server, and wherein at least one of said first selection set, said at least one different selection set, and said at least one new selection set includes at least one of said items.
10. The method of claim 1 , wherein said GUI is configured to receive user input by any of a touch interface, pointing device, and gesture recognition device, and voice recognition device, and wherein at least some of said entities are displayed by said computer processor on said GUI, and are selected using any of said touch interface and pointing device and gesture recognition device, and voice recognition device to designate at least some of said entities displayed by said computer processor on said GUI.
11. The method of claim 1 , wherein said entities comprise of any object, manipulatable by the GUI, that can be selected by any methodology available in said GUI; or wherein any selection set further comprises a set of selection sets.
12. The method of claim 1 , wherein said GUI is configured to accept at least designating said selectable entities, said different entities, and selecting said at least one set operation as input from a user, and at least designating said selectable entities, said different entities, and selecting said at least one set operation is done by said user using said GUI.
13. A graphical user interface (GUI) based method for managing selectable entities, said method comprising: designating, using a GUI and a computer processor, from a plurality of selectable entities, irrespective of entities location or display status, a selection set comprising entities from said plurality of selectable entities, and optionally saving, in a recallable manner, said selection set in computer memory as a first selection set; wherein said entities comprise any of files and directories; designating, using said GUI and a computer processor, from said plurality of selectable entities, at least one different selection set comprising different entities from said plurality of selectable entities, and saving each said at least one different selection set in computer memory as at least one different selection set; selecting, using said GUI and a computer processor, at least one set operation to be performed on said first selection set and said at least one different selection set; wherein said GUI is configured to receive user input by any of a touch interface, pointing device, and gesture recognition device, and voice recognition device, and wherein at least some of said entities are displayed by said computer processor on said GUI, and are selected using any of said touch interface, pointing device and gesture recognition device, and voice recognition device to designate at least some of said entities displayed by said computer processor on said GUI; wherein said at least one set operation comprises any of set union, set intersection, and set inversion (i.e. set negation); creating, using said computer processor, said at least one set operation, said first selection set, and said at least one different selection set, at least one new selection set; and using said at least one new selection set to manage at least some of said selectable entities; wherein said at least one new selection set is used to perform any of memory storage operations and network transmission operations upon any of files and directories in said at least one new selection set; wherein location or display status of said first selection set, said one different selection set, and said one new selection is irrelevant.
14. The method of claim 13, further comprising adding or removing additional entities from any of said first selection set, said at least one different selection set, or said at least one new selection set, thereby creating any of a modified first selection set, a modified different selection set, and a modified new selection set; or wherein any selection set further comprises a set of selection sets.
15. The method of claim 14, further comprising loading, saving, or removing any of said modified first selection set, said modified different selection set, and said modified new selection set under any of their original names and their modified names.
16. The method of claim 13, wherein said GUI and said computer processor are further configured to search said plurality of selectable entities for at least one GUI selectable search parameter; further using said GUI to define at least one GUI selectable parameter, and using said computer processor, said at least one GUI selectable parameter, and said plurality of selectable entities to determine any of said first selection set and said at least one different selection set.
17. The method of claim 13, further using said GUI and said computer processor to create or select at least one script or program, irrespective of the location or display status of the script or program, thereby determining at least one selected script or program; said at least one selected script or program configured to accept said at least one new selection set as any of input or output parameters for said at least one script or program; and using said computer processor to execute said script according to said at least one new selection set.
18. The method of claim 13, wherein said entities comprise any of remotely hosted audio and video streams, said GUI and said computer processor are further configured with web browser functionality, and said first selection set, said at least one different selection set, and said at least one new selection set comprise sets comprising any of said remotely hosted audio and video streams.
19. The method of claim 13, wherein said entities comprise items in a video game, said video game configured to be run by one selected from the group including a computer processor, a computer processor and at least one remote server, and at least one remote server, and wherein at least one of said first selection set, said at least one different selection set, and said at least one new selection set includes at least one of said items.
20. The method of claim 13, wherein said GUI is configured to accept at least designating said selectable entities, said different entities, and selecting said at least one set operation as input from a user, and at least designating said selectable entities, said different entities, and selecting said at least one set operation is done by said user using said GUI.
PCT/US2021/055422 2021-10-18 2021-10-18 Gui based methods and systems for working with large numbers of interactive items WO2023069068A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2021/055422 WO2023069068A1 (en) 2021-10-18 2021-10-18 Gui based methods and systems for working with large numbers of interactive items

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2021/055422 WO2023069068A1 (en) 2021-10-18 2021-10-18 Gui based methods and systems for working with large numbers of interactive items

Publications (1)

Publication Number Publication Date
WO2023069068A1 true WO2023069068A1 (en) 2023-04-27

Family

ID=86059494

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2021/055422 WO2023069068A1 (en) 2021-10-18 2021-10-18 Gui based methods and systems for working with large numbers of interactive items

Country Status (1)

Country Link
WO (1) WO2023069068A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020133491A1 (en) * 2000-10-26 2002-09-19 Prismedia Networks, Inc. Method and system for managing distributed content and related metadata
US20070129148A1 (en) * 2005-10-14 2007-06-07 Leviathan Entertainment, Llc Method and System for Allocating Resources in a Video Game
US20120005605A1 (en) * 2008-01-10 2012-01-05 At&T Intellectual Property I, Lp System for managing media content
US20150065241A1 (en) * 2013-08-27 2015-03-05 Zynga Inc. Gaming systems and methods for facilitating item grouping and group actions
US20160048286A1 (en) * 2014-08-13 2016-02-18 King.Com Limited Selecting objects on a user interface

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020133491A1 (en) * 2000-10-26 2002-09-19 Prismedia Networks, Inc. Method and system for managing distributed content and related metadata
US20070129148A1 (en) * 2005-10-14 2007-06-07 Leviathan Entertainment, Llc Method and System for Allocating Resources in a Video Game
US20120005605A1 (en) * 2008-01-10 2012-01-05 At&T Intellectual Property I, Lp System for managing media content
US20150065241A1 (en) * 2013-08-27 2015-03-05 Zynga Inc. Gaming systems and methods for facilitating item grouping and group actions
US20160048286A1 (en) * 2014-08-13 2016-02-18 King.Com Limited Selecting objects on a user interface

Similar Documents

Publication Publication Date Title
US7958171B1 (en) System and method for organizing search criteria match results
US8775955B2 (en) Attraction-based data visualization
CN101300621B (en) System and method for providing three-dimensional graphical user interface
Terry et al. Variation in element and action: supporting simultaneous development of alternative solutions
Chilana et al. LemonAid: selection-based crowdsourced contextual help for web applications
US8607149B2 (en) Highlighting related user interface controls
US7844036B2 (en) Visual test automation tool for message-based applications, web applications and SOA systems
US5953731A (en) HTML integration utility for a program development environment
JP2001514418A (en) Browser for hierarchical structure
US8091043B2 (en) Modeling environment graphical user interface
US11150783B1 (en) GUI based methods and systems for working with large numbers of interactive items
WO2006000310A1 (en) Object based navigation
US20080244444A1 (en) Contextual computer workspace
KR20150058237A (en) Method and system for simplified knowledge engineering
JP2012527043A (en) Method and system for interacting with and manipulating information
US8769439B2 (en) Method for creation, management, and presentation of user-scoped navigation topologies for web applications
CN102193960B (en) Content sharing system
CN105745644A (en) Establishment of state representation of a web page represented in a web browser
US8230319B2 (en) Web-based visualization, refresh, and consumption of data-linked diagrams
WO2020231576A1 (en) Scoped view of file tree
WO2023069068A1 (en) Gui based methods and systems for working with large numbers of interactive items
JP2786194B2 (en) Information retrieval apparatus and information retrieval method
EP4086755B1 (en) Robotic process automation (rpa) comprising automatic document scrolling
CN109814857B (en) Method and device for customizing primitive linkage
US8631338B2 (en) Generating universal graphical user interface element and relation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21961568

Country of ref document: EP

Kind code of ref document: A1