KR20200132999A - Single device multiple authentication system - Google Patents

Abstract

Multiple authentication systems support a variety of password entry mechanisms (eg, alphanumeric, visual, voice, etc.) that can be used to authenticate users to access multiple application/website targets. Exemplary methods and systems include several different service providers (e.g., third-party applications, cloud services, websites, etc. that include user authentication) without storing passwords in local or network memory (e.g., password vault). ), it includes a real-time password generator that generates unique and complex passwords regardless of your internet connection. In response to receiving the login request, the user device prompts the user to provide an access code and generates a target key based on the securely stored identifier code. The target key can be regenerated using the stored identifier code and access code. The same stored identifier code and received access code can be used to regenerate different target keys for different applications or services.

Description

Single device multiple authentication system

Various embodiments described herein relate to user authentication, and more particularly, to systems and methods for visual access codes.

The conventional media object indexing technology is limited. For example, the library of media objects may be configured according to a tree hierarchy (eg, folders and subfolders or albums and subalbums) in which each node corresponds to a separate label. Membership of media objects in nodes of the tree (eg, folders or albums) is maintained manually. Thus, to reclassify a media object from one type of classification (eg, location) to another type (eg, event), all media objects in the library must be reclassified.

Keyword-based classification (e.g., text tagging) may be more suitable for creating and maintaining dynamic albums or folders. However, keyword-based classification techniques involve manual entry to add all possible tag permutations. Thus, you may just need to add multiple tags for a single location (eg nature, parks, trees, grass). Also, some tags tend to be ambiguous (eg "Paris" could be a city name or a person's name).

Classification based on direct attributes (eg Apple® iPhoto®) allows users to create dynamic albums based on direct attributes associated with the constituent media objects. For example, an album may contain all photos of a specific period (eg, from April 11, 2013 to May 5, 2014). However, the direct attribute system provides a poor user interface (UI) and imposes significant restrictions on searches performed based on direct attributes.

Limited semantic tagging (eg Facebook® tagging) provides dynamic classification of media objects based on a limited set of indirect attributes. In particular, tags do not distinguish between the different types of relationships that may exist with media objects. For example, the person tagged in the photo may appear in the photo, may be interested in the content of the photo, or may be the person who created the photo.

Fully automated media object indexing techniques are generally inaccurate. For example, image recognition systems only have a 70% success rate in identifying generic objects (eg, shoes) depicted in media objects. Such systems are also unable to determine relationships with respect to media objects (eg, owners, designers and/or retailers of shoes). In contrast, the manual method of indexing media objects is tedious and error prone with little user incentive.

There is a need for a system and method for indexing media objects that can support advanced search and browsing functions.

An authentication system and method using a visual access code is provided. According to various embodiments, the user registration and authentication system is based on a visual access code. The method includes presenting an image to a user, receiving a selection of a first image, receiving a selection of at least a first set of hotspots from a plurality of hotspots included in the first image; And generating a visual access code based at least in part on the selection of the first image and the first hotspot set.

Other features and advantages of the inventive concept will become apparent from the following description describing exemplary aspects of the inventive concept.

Included in the context of the present invention.

The above and other aspects and features of the present invention will become more apparent by describing exemplary embodiments with reference to the accompanying drawings.
1 is a network diagram illustrating various network environments.
2 illustrates semantic indexing according to various embodiments.
3 illustrates a stamping user interface according to various embodiments.
4 is a flow chart showing a stamping process according to various processes.
5 illustrates a process of adding an association to a stamp according to various embodiments.
6 illustrates an object selector according to various embodiments.
7A illustrates an association selector according to various embodiments.
7B illustrates single and multiple association peaking according to various embodiments.
8 illustrates a visual interface according to various embodiments.
9 is a diagram illustrating a visual browsing menu according to various embodiments.
10 illustrates selection criteria according to various embodiments.
11 illustrates a process of adding a new search selector according to various embodiments.
12 illustrates a facet navigation interface according to various embodiments.
13 illustrates a faceted display section according to various embodiments.
14 illustrates facet representation according to various embodiments.
15 illustrates a limited discrete index according to various embodiments.
16 illustrates a simple derivation index according to various embodiments.
17 illustrates a fuzzy derivation index according to various embodiments.
18 illustrates a multi-mode control switch according to various embodiments.
19A-C illustrate a multi-mode UI according to various embodiments.
20 illustrates data access calculations according to various embodiments.
21 illustrates a process for enforcing access control according to various embodiments.
22 illustrates an automatic change of an access control rule state according to various embodiments.
23 illustrates a manual change to an access control rule state according to various embodiments.
24 illustrates synchronization based on a central server according to various embodiments.
25 illustrates peer-to-peer synchronization according to various embodiments.
26 illustrates hierarchical synchronization according to various embodiments.
27 illustrates an access control rule cascade according to various embodiments.
28 illustrates a peer-to-peer browsing session according to various embodiments.
29 illustrates a process for initiating a peer-to-peer browsing session according to various embodiments.
30 illustrates a process of constructing a visual access code according to various embodiments.
31 shows an exemplary lock code management interface for use with a visual access code.
Fig. 32 shows an exemplary flow for configuring a visual access code according to an exemplary embodiment.
33 illustrates an exemplary process of mapping a user identifier to photo selection mapping according to various embodiments.
34 illustrates an exemplary unique user hex digest according to various embodiments.
35 shows an exemplary registration process for assigning a visual access code according to an exemplary embodiment.
Fig. 36 shows an exemplary process for encoding a visual access code using a password according to an exemplary embodiment.
37 illustrates an exemplary mobile interface according to various embodiments.
Fig. 38 shows an exemplary implementation of entering a visual access code according to an exemplary embodiment.
39A-C illustrate exemplary implementations of a virtual input method in a web site according to various embodiments.
40 illustrates an image blending process according to various embodiments.
41 illustrates an example implementation of hotspot location shifting according to various embodiments.
42 is a block diagram illustrating a wired or wireless system according to various embodiments.
43 illustrates an exemplary ID setting process according to various embodiments.
44 illustrates an exemplary default key generator process in accordance with various embodiments.
45 illustrates an exemplary ghostPassword keyboard interface according to various embodiments.
46 illustrates an exemplary user name input process according to various embodiments.
47 illustrates an example of defining a moment interface according to various embodiments.
48 shows an exemplary browser ghost password entry process.
49 illustrates an exemplary ghost password management process according to various embodiments.
50 illustrates an exemplary flow of a ghost password element according to various embodiments.

Although specific embodiments have been described, these are presented by way of example only, and are not intended to limit the scope of protection. The methods and systems described herein can be implemented in a variety of different forms. In addition, various omissions, substitutions, and changes in the form of the exemplary method and system described herein may be made without departing from the scope of protection.

1 is a network diagram illustrating a network environment 100 according to various embodiments. Referring to FIG. 1, a user device 110 communicates with a media platform 120. User device 110 may be any device capable of communicating with or causing communication with media platform 120 via a wired or wireless connection. For example, the user device 110 may be a wired or wireless communication device including, but not limited to, a smart phone, a wearable device, a tablet PC, a laptop, a desktop PC, a personal entertainment system, and an embedded processing system. have.

The user device 110 may communicate with the media platform 120 through the communication network 130. In various cases, communication network 130 represents one or more wired and/or wireless connections. For example, the communication network 130 may include, for example, a wired and/or wireless local area network (LAN), a wired and/or wireless wide area network (WAN), and any combination thereof, but is not limited thereto.

Media platform 120 may be communicatively coupled with local data storage 125. Further, the media platform 120 may further communicate with a plurality of remote and/or third party data sources including, but not limited to, the first data source 140 and the second data source 150, for example.

Association model

Semantic indexing

In various embodiments, the media platform 120 may associate a media object with semantic information including, but not limited to, attributes, relationships, and classifications. Semantic information may be inherited from one or more other objects (ie, including other media objects) each providing a separate set of attributes, relationships and/or classifications.

For example, a media object (eg, a photo) could depict a smiling Bill Gates. Media objects can inherit all the attributes of Bill Gates as a person and the relationships Bill Gates has with others (e.g., age, entrepreneur, influencer, billionaire, philanthropist, father, family, technologist, American, etc.). have. A smile means that Bill Gates appeared in the picture and was in a good mood.

John Smith may be interested in the content of a media object (eg, a photo) depicting Bill Gates. Thus, a media object can inherit all the attributes of John Smith as a person and the relationship John Smith has with others, even if John Smith is not depicted in the photograph.

As another example, a media object (eg, a video) can depict a vehicle belonging to Alice after an accident. The media object can inherit all the attributes of Alice's car (eg, make, model, year, mileage, and maintenance records) that can be viewed through a third-party source (eg, Carfax®). The media object may further inherit all of the attributes for Alice, including, but not limited to, Alice's driving record, job activity, and personal information. In addition, a media object may inherit all of the attributes for a particular incident (eg, categorized as minor or large incident).

In various embodiments, the media platform 120 may enable a media object to be searched through any one of corresponding semantic information. For example, a media object depicting Bill Gates can be found by searching for a picture of a smiling 50-year-old man. Similarly, images of Alice's car accident can be found through a search for images of cars owned by a woman involved in the accident.

It should be understood that the media object can be any kind of computer storage file including, but not limited to, text and multimedia (eg, photo, video) files, for example.

2 illustrates semantic indexing according to various embodiments. Referring to FIG. 2, a media object (eg, a picture of a vehicle) may be connected to a plurality of semantic information including, but not limited to, manual association 2 and automatic association 4. In various embodiments, the media platform 120 may create and add an automatic association 2 based on geographic location information included in metadata of a media object. Based on manual association (2) and automatic association (4), a media object can be a number of media objects including, but not limited to, ownership (e.g., Bob Smith) and location (e.g., Jordan Middle School parking lot). Other relationships can be inherited.

Automatic associative model

In various embodiments, automatic associations may be added to media objects. For example, you can create automatic associations based on geographic location and/or timestamps. Automatic associations can also be created and added to media objects based on current events (eg, fairs, holidays, personal birthdays, etc.) and weather (eg, rain, snow, storm). In some embodiments, if the certainty of the automatic association is below a certain threshold, an automatic association may be provided to the user for confirmation.

Augmented semantic information

In various embodiments, the media platform 120 may add new semantic information to a media object, and through this, may search for a media object based on the new semantic information. For example, if Bob Smith retired, the media object depicting Bob Smith's vehicle could be searched for'cars owned by retired people'.

The media object may be further searched based on new semantic information added to an object having an existing association with the media object. For example, if Bob Smith's son Charlie Smith is added as an object and Charlie Smith is a Jordanian middle school student, then the media object depicting Bob Smith's vehicle can also be searched for'cars owned by the student's parents'. .

The system for semantic indexing includes a media platform. In various embodiments, the media platform receives a first media object associated with a first set of semantic information; It may include one or more hardware processors configured to associate the first media object with a second media object associated with a second set of semantic information.

According to an exemplary implementation, the first media object inherits a second set of semantic information associated with the second media object. Each of the first and second semantic information sets may include at least one of an attribute, a relationship, and a classification. One or more hardware processors may be configured to automatically generate additional semantic information and to associate the automatically generated semantic information with the first media object. The one or more hardware processors may be configured to automatically generate additional semantic information based at least in part on one or more of a geographic location and a timestamp associated with the first media object.

According to another exemplary implementation, the one or more hardware processors are configured to receive additional semantic information from the user and to associate the additional semantic received from the user with the first media object. For example, the user may provide additional semantic information at least partially by displaying an association between the first media object and the third media object related to the third semantic information set. In another example, the user provides additional semantic information, at least in part, by displaying an association between the first media object and one or more attributes, relationships, and classifications.

Human-centered association interface

In various embodiments, media platform 120 provides a user interface (UI) that allows a user to quickly attach information to a media object.

User interface stamping

The media platform 120 allows a user to allocate semantic information to media content using the stamping UI 300. In various embodiments, the stamping UI 300 allows a user to allocate semantic information to multiple media objects with one click. The stamping UI 300 provides a stamp area 4 that displays a category, source, or value of semantic information to be added. In one exemplary implementation, the user clicks on a target media object in the list 2, and the stamp area 4 recommends a semantic information list based on the analysis of the target media object. Analysis of the target media object may include object recognition and metadata analysis, as well as examining semantic information of other media items related to the target media item. The user may have a choice to edit the recommended semantic information list in the stamp area 4 or may accept the entire recommended set. For example, the user can stamp the target media object together with the semantic information set shown in the stamp area 4 with one click.

The stamp area 4 may contain individual associations 6 that can be added separately from other associations. The stamp area 4 may further include an associated template 8. The stamp association template 8 may include a semantic category group configured based on a commonly used association. When the stamp-related template 8 is selected, a set of semantic information for each category of the template is allocated. For example, the home stamp association template associates the target media object with other media objects related to the house, adds semantic information about the geographic location of the house, and also adds a relationship with a person who is part of the home category. Can be done. The stamp association template 8 can be pre-configured to be used repeatedly for commonly used associations.

According to an exemplary embodiment, a system for semantic indexing includes: indexing a media object of a media platform to identify semantic information of each media object; It may include a media platform having one or more processing devices configured to associate a plurality of media objects based on matching semantic information. In an example, the processing device searches an index of the media object for semantic information common to the stamp template, and based on the search, the processing device classifies the corresponding media object based on the common semantic information; Displays classified media objects so that users can curate. In some embodiments, the processing device generates an association stamp template including common semantic information to apply association to one or more media objects. For example, curation may include stamp templates or one or more other associated applications. The stamp template can be edited to modify the semantic information of the association and include multiple associations. In some embodiments, the system includes an interface with stamp templates that allows a user to curate media objects of a media platform by applying associations from the selected stamp template to multiple media objects based on a single selection. In an example, the processing device receives additional semantic information from user input to associate with one or more media objects.

Stamping workflow

4 is a flow diagram illustrating a stamping process 400 in accordance with various embodiments. Referring to FIG. 4, in various embodiments, process 400 may be performed by media platform 120.

In various embodiments, process 400 is performed when a data collection mode is selected. As such, the media platform 120 may collect additional information on at least some specific associations from the stamp. That is, objects having common semantic information can be identified and the common semantic information can be grouped as association. A template may be formed by grouping common semantic information. Grouping of semantic information is attached through the stamping operation, and the actual media object is defined in the system. Associations can also form relationships between objects with common semantic information that can propagate modifications.

For example, if a user clicks on a media object shown on screen display 6, screen display 8 appears in response to determining that the stamp contains a food item association. In the screen display 8, the UI makes it possible to collect information about food. The screen display 8 differs depending on the object template type. When the user activates the save function, a new object is created (or saved) and an association is assigned to that object.

Alternatively, if the data collection mode is not selected, the user can click on the media object and another object in the stamp area 4 is associated with the media object.

In various embodiments, the two workflows are part of the overload approach of media platform 120. The media platform 120 may gradually collect information as specified by the user. Advantageously, the user is encouraged to make an effort to enter additional information due to the improved ability to search based on that information using visual search.

Association selection interface

In various embodiments, the user may identify the content of the media object, associate semantic information with the media object, and select an association type. When a user associates the content of another media object with the target media object, the target media object inherits another association with the content of the other media object. Advantageously, the user can efficiently identify the content and associate it with the media object with high accuracy. 4 shows a flow diagram for an exemplary data collection process for associating a world object with media content.

5 shows a process 500 for adding an association to a stamp according to various embodiments. Referring to FIG. 5, the process 500 may be performed by the media platform 120. On the screen display 2, the user can inspect the media object to be curated. From the stamp view, the user can see the association to be added. In the stamp view, the user can select a photo (eg, click, touch, etc.) and stamp it on a media object, and all connections of the stamp are added to the photo. Alternatively, the user can choose to add a new connection to the stamp.

In response to the user clicking the stamping mode button, the menu proceeds to a screen display 4 that allows the user to select from different processes to add a new association. For example, in some embodiments, the user can add a new association by selecting the object type via the screen display 6. When the user selects an object type, the UI is switched to the screen display 8 (ie, an object selection screen) to display a list of searchable objects according to the selected object type. Users can search for an object to add. In some embodiments, the screen display 8 includes a stamp icon to provide easy navigation.

When a specific object is selected, an association selection screen display 10 is provided to allow a user to additionally select an association for the object. The user has the option to cancel and return without completing the associated task. Alternatively, the user may select one or more associations in the association selection screen display 10 to complete the stamping operation. Subsequent to or alternatively to the association selection screen 10, the UI provides additional confirmation controls (eg, save, cancel, etc.) to complete the association as depicted on the screen display 12.

6 illustrates an object selector 600 according to various embodiments. Users can select an object type and start searching for various objects to connect to. The UI 600 displays an associated stamp displaying an existing stamp in order to communicate with a user whose object has already been loaded into the stamp.

Association selection

When the user clicks the menu button to start the association editing mode, the user can choose to add associations to the stamp in various ways. For example, the user can select from a list of recent connections. In another example, the user can select from previously configured association templates. Each template is an association group entered by the user. For example, the template could include "baby Jim playing", "Project X" or "receipt for Project Y." In some embodiments, the media platform 120 allows a user to define and edit a fixed number of templates. With a limited number of configurable templates, users can easily access preconfigured templates without a template management system.

Alternatively, the user can select a world object type that allows the user to navigate the object type to select an association. In addition, users can select from common associations, such as general associations configured by application authors. For example, an application that uses thumbnails can use temporary photo associations.

Associated Picker Flow

To simplify selection of the type of association between world objects and media objects, media platform 120 provides an optimized association selector that allows a user to pick one or more associations. Fig. 7A shows an association selector 700 according to an exemplary embodiment.

For example, to select one association, the user may click an association button or checkbox (eg, select) to complete the association selection. To select multiple associations, the user can click a checkbox (for example, select), and this association dialog acts as a multiple selection dialog. Fig. 7B shows single and multiple association selections according to an exemplary embodiment.

Association search

The media platform 120 allows a user to add associations incrementally. Moreover, the media platform 120 allows users to quickly group artifacts with common parameters for faster association. In various embodiments, media platform 120 connects the associative process to the search process through a multi-mode user interface. Using a multi-mode user interface, users can quickly switch between association and search and vice versa. With this quick transition, the following scenarios are possible:

While the user is adding an association, the user can switch to the search mode to limit or filter the number of available media objects. Thus, the user searches for a simple attribute such as a timestamp or semantic information that has already been added.

While the user is searching, the user may determine that some semantic information is missing. The user can then quickly switch to the stamping mode through the first level menu and add more associations to the media object.

The interface allows the user to add as much information as desired in an incremental manner, which reduces the perceived amount of work and effort. Moreover, the UI allows users to start searching with the newly added semantic information, so they can immediately recognize the benefits of the newly added information.

Hierarchical Visual Facet Search

In various embodiments, the media platform 120 provides a visual interface that allows a user to quickly view the criteria that the presented media meets. The selection criteria may be displayed in a specific area. In addition, the interface may have a normal mode and a minimum mode to provide more space to the user. 8 illustrates a visual interface 800 according to various embodiments. In minimal mode, the user is in read-only mode and cannot interact with the various selectors.

In various embodiments, presenting the query as a group of selectors simplifies the concept to the end user. In addition, the position of the selectors relative to each other is important and the user can rearrange them.

9 illustrates a visual browsing menu 900 according to various embodiments. Referring to FIG. 9, in various embodiments, when the user clicks the center menu button in the normal mode, the media platform 120 provides a first-level menu view 2. This menu allows the user to browse two level hierarchical facet classifications. When the menu button is clicked once, the first-stage facet category is displayed so that the user can navigate to the second-stage menu view (4). In the second level menu view 4, the user is presented with information on various media objects. Users can view media objects from March to August by checking for multiple media objects from multiple years (except for example 2008, 2010 and 2012, 2011) and filtering. At the same time, the interface displays a selector used to pick data.

Selector

10 shows selection criteria 1000 according to various years. Referring to FIG. 10, a selection criterion 1000 may be established by a user. As shown in Fig. 10, there may be several selectors 2 for each facet. The selector 2 can designate a facet value used for search. The selection criterion 1000 may include the NOT criterion 4 and MUST criterion 6, and both may be single value selectors. The selection criterion 1000 may further include a map location criterion 8 and a value range criterion 10 which may both be OR criteria (ie, at least one of the map location criterion 8 and the value range criterion 10). Must be true). Advantageously, the user can quickly navigate and understand the criteria used in the search.

Add new search selector

11 illustrates a process 1100 of adding a new search selector according to various embodiments. Referring to FIG. 11, the user clicks the open menu 2, and the menu guides the user to the faceted browsing mode 4. If the user decides to search using a specific facet, the user can drag that facet to the selector area. The dragging operation is important because the user can place the selector over existing selectors. By dragging and dropping the selector, a dialog box (6) is displayed, allowing the user to further edit the selection. The dialog box 6 may display a different UI for each facet type. The purpose of displaying a large interface is to give the computer application designer more space to present various options to the user. In addition, the large interface relieves the user from having to deal with a small space. In the dialog box 6, the user can assign AND, OR and MUST criteria to different facet values or value ranges. When the user clicks Accept, the newly added selector is displayed on the screen display 8.

Faceted navigation

12 illustrates a faceted navigation interface 1200 according to various embodiments.

The user may navigate between facets by selecting one of the first-level facet categories. For example, the first level facet category can be divided into several (eg, 5) basic groups. The group depends on the media object to be searched for, for example, who, what, when, where and how. When the user clicks one of the facet categories of the first level, the second stage facet category is displayed to simplify the user's search. For example, under the first stage facet category "When", there may be several second stage facet categories including, but not limited to, specific dates, weather conditions, event types, and event names. In one exemplary embodiment, each top-level category has a different color to help highlight and identify the category.

When the user clicks on the second stage facet category, the interface displays available facets and facet values in the user's media object. In some embodiments, a single facet with an unlimited value may be included in the second stage facet category. Advantageously, the faceted navigation interface provides a quick way to view the metadata of a media object. Through the faceted navigation interface, users can explore media collections based on meta-information caused by curiosity. The user may be further encouraged to identify the missing meta-information (for example, the user can switch to stamping mode and add the missing information). In addition, the faceted navigation interface provides a unified interface for users to initiate media requests that are not available in the media library. For example, if a user finds that a particular media object (eg, last year's user cousin's birthday photo) is missing from the media library, he can use that interface to initiate a request for the missing media object.

Faceted display section

13 illustrates a facet display unit 1300 according to various embodiments. Referring to FIG. 13, the facet display unit 1300 displays facets of a specific facet category regardless of whether the facets have a corresponding value. By displaying facets with no values, the user can see what is missing for the collection of media objects the user is currently viewing. For example, the user may observe that no value is provided for the occupation facet or the gender facet from the facet display unit 1300.

The system can display the facets in response to determining that there are no values associated with the facets. If you provide a facet with an empty value or no search hit, background search notifies the user that the facet has determined that it is not useful in the data set. The empty facet may inform the user that semantic data is missing from one or more data objects or that the data object is unavailable. The user can then find the media object and add the missing metadata or get the unusable media object.

Facet expression

In various embodiments, the media platform 120 presents facets to the user in different ways based on the type associated with the facet value. Advantageously, displaying facets based on the type associated with the facet value improves communication with the user about the available data types.

Discrete values are individual values. There are two types of discrete values: limited discrete values and unlimited discrete values. A limited discrete value (eg gender) is a value with a limited number of possible values. Facets with limited discrete values are part of the second level facet category. In contrast, unlimited discrete values (eg, people, events) are represented as separate second level facet categories.

Range values are values that can be grouped into ranges. Some range values are contiguous ranges containing an unlimited number of possible values (eg, timestamps). Continuous range values are always expressed as a range or group of ranges. Conversely, an integer range consists of discrete values (eg, days of the week). Integer ranges can be expressed as groups of discrete values. The map value may be displayed in a special map presentation in which the map value is a special value.

14 illustrates facet representation according to various embodiments. As shown in FIG. 14, the facet representation may include a representation of a limited discrete facet 12, an integer range facet 4, and an unlimited discrete facet 14. Unlimited discrete facets 14 may require an entire subcategory to indicate where different facet representations (eg, limited discrete facets 12 and integer range facets 4) can be grouped together.

Hierarchical facets for media content

In various embodiments, facets may be organized in a two-level hierarchy. Hierarchies are stored in data structures (eg, trees). Each leaf node in the tree can point to an index of an individual facet value. As such, the media platform 120 may process different facet layers based on a user. For example, the facet hierarchy may be set based on user-level expertise or interests.

For example, a layer can contain:

1) Who?

a. name

b. relation

c. Attention

d. job

2) When?

a. date

b. event

c. Event type

d. Weather conditions

3) Where?

a. Place name

b. map

c. Place type

4) How?

a. photographer

b. Camera type

c. Media type

5) What?

a. Object

b. Object type

Appendix B further shows hierarchical faceted search engines according to various types.

According to various embodiments, a system for performing hierarchical visual facet search for one or more media objects includes a media platform. In various embodiments, the media platform provides a user device with a selector user interface (UI) adapted to receive a plurality of selection criteria; Providing a first level menu including a plurality of first level selection criteria including first selection criteria to a user device; Receive, from a user device, an indication to add at least the first selection criteria to a selector (UI); Providing a user device with a second level menu including a plurality of second level selection criteria corresponding to the first selection criteria, wherein the plurality of second level selection criteria includes a second selection criteria; Receive, from a user device, an indication to add at least the second selection criterion to a selector (UI); Add a first selection criterion and a second selection criterion to a selector (UI); It may include one or more hardware processors configured to perform a search to identify one or more media objects that satisfy the first selection criterion and the second selection criterion based at least in part on the content of the selector UI.

In an exemplary implementation, the first selection criterion may include an ID criterion, a location criterion, and the second selection criterion may include one of the name, relationship, interest, and occupation of an individual associated with the media object. For example, the first selection criterion may include a time criterion and the second selection criterion includes one of a date, event, event type, and weather condition associated with the media object. In another example, the first selection criterion may include a location criterion and the second selection criterion may include one of a map coordinate, a location name, and a location type associated with the media object. In another example, the first selection criterion may include a method or means criterion, and the second selection criterion may include one of a photographer, a camera type, and a media type associated with the media object. In another example, the first selection criterion may include an identification criterion, and the second selection criterion may include one of an object and an object type associated with the media object.

Hybrid in-memory facet engine

In various embodiments, a faceted search engine may be deployed on user device 110. As such, the faceted search engine can rely on an in-memory search index that can be loaded on demand based on the facets being searched. The search index is built on top of the actual object property values.

Facets are a way to search for one or more media objects. Facets can be associated with various facet values. Moreover, each facet corresponds to a specific field of data being indexed. Fields have data types and expected values. To perform a search, the faceted search engine can build an index of all values in each file. Advantageously, faceted search engines can run all queries very quickly. In contrast, traditional faceted search engines rely on precomputed views of fixed queries and cannot handle dynamic compound queries.

In an exemplary embodiment, the system performs a hybrid in-memory facet search for one or more digital objects. The system stores an index for a data set created using one or more indexing processes, wherein the index includes a mapping of values to an identifier for each digital object in the data set; Receive an update to the index; Store index updates using a timestamp separate from the stored index; It may include a media platform having one or more processing devices configured to apply index updates to the index in response to a request for a stored index. In an example, the processing device determines characteristics associated with the data set; An index is a process based on the characteristics of a data set, and the index includes the mapping of values to identifiers for each digital object in the data set.

For example, the data set may include discrete data and the index includes multiple arrays for each digital object including at least one of the discrete data and classified values of the identifier. For example, the identifiers may be grouped into ordered groups. In an exemplary embodiment, the data set contains continuous data and each digital object is mapped to a unique timestamp.

When the mapping of values to the identifier for each digital object in the data set includes overlapping values, the processing unit determines a confidence factor associated with each value for each digital object and ranks the identifier based on the confidence factor. I can.

The request for the stored index may be a search request having a query criterion, and the processing device generates a selector object matching the query criterion to an identifier of the index; Calculate the number of unmodified identifiers associated with each query criterion based on the selector object; You can run a search starting with the query criteria associated with the most unmodified identifier of the query criteria.

Type of index

To index data, faceted search engines can use different types of indexes depending on the type and characteristics of the data being indexed. Each index type is written differently and has a specific purpose.

The index may be a base index or a derived index. Each index type is implemented in a different way, but every index has the ability to map the actual value to a specific row of the indexing media object.

Primary index

The base index is calculated directly from the primitive value of the world object. Each type of primitive value is treated differently, so different types of base indexes can be calculated depending on the characteristics of the primitive value.

15 illustrates a limited discrete index 1500 according to various embodiments. Referring to FIG. 15, the limited discrete index 1500 is a basic type of index including discrete and unlimited. The limited discrete index 1500 is always maintained in memory (eg, of the user device 110) and is placed whenever a limited discrete index 1500 is needed. In various embodiments, the constrained discrete index 1500 may be implemented in two arrangements. The first array may hold a value indexed in an ascending sort order and a start position of the second index. The second arrangement can hold media row identifiers grouped according to their original values and in sorted order in each group. Advantageously, storing the limited discrete index 1500 requires a small amount of memory.

Large discrete indices contain a very large number of discrete values.

The continuous index contains an unlimited number of possible values with a value (eg, timestamp) and an almost one-to-one mapping. For example, almost all photos can have different timestamps. Continuous range values can be processed with a special index structure. For example, a continuous index can be processed using a normal B-tree similar to a database index.

The map index contains geographic location data. The geographic location data may be three-dimensional data (eg, longitude, latitude and altitude) that is processed as a whole. In some embodiments, a database engine (eg, SQLite®) may be used to process the map index.

Derived index

Derived indices are based on other indices (eg, base or derived) and provide classification and/or implications. In various embodiments, the derived index may be a simple derived index or a fuzzy derived index.

16 illustrates a simple derivation index 1600 according to various embodiments. Referring to FIG. 16, the index value of the simple derivation index 1600 is based on a value indexed by another index. For example, an age group may be indexed with a plurality of facet values including, but not limited to, babies, infants, children, teenagers, young adults, adults, middle-aged and elderly people. The actual indexed value is derived from the raw age value. For example, an infant is a person between 2 and 5 years of age. In various embodiments, the simple derived index 1600 is created using an array holding the value and the corresponding primitive value in the base index. As such, the simple derived index 1600 occupies a very limited space in memory and can easily accommodate changes to the base index.

17 illustrates a fuzzy derivation index 1700 according to various embodiments. Referring to FIG. 17, the fuzzy derivation index 1700 may include multiple index values overlapping the original value. Therefore, each mapping has a certainty factor associated with it. For example, if there is a fuzzy index in the age group, a person who is 1.8 years old could be a baby and an infant. However, the person is more likely to be an infant. As such, the confidence factor for infants is 90%, while the confidence factor for babies is 15%. The confidence factor is chosen to be appropriate for each value. The advantage of such fuzzy indexing is that you can find the same information in different ways and use certainty factors to rank the search results.

Index life cycle

Indexes can be created in memory by iterating over the raw data. Indexes are kept in memory in the most compact form. Conversely, raw data is accessed differently in orthogonal operations. In order to keep the memory usage small, raw data can be repeated in batches when the volume of raw data is large. Because the data is loaded in sorted order, the process can involve multiple iteration passes (e.g., one pass for each index). Thus, if the amount of raw data is small, the raw data can be loaded into memory at once and sorted during index creation.

When the index is created in the most concise form, the index is stored in the file in that form. In the case of an index with more than one array, the individual arrays are stored in the same file in a specific order, e.g., an array of index data before the row id array.

You can also update the index, including adding or deleting values to the index. In various embodiments, if the index is already in memory, the update is applied to the index and when the update is complete, the index is stored on disk.

If the index is on disk, the update is added to the update file for that particular index. The update file contains all updates sorted by timestamp. Indexes are not uploaded for update. Instead of loading the index to apply one or more updates, the faceted search engine saves the changes to apply to the index. When an index is required for a search, the index is loaded from disk into memory, the stored update is loaded and applied to the index, and the index is stored in memory and can be used for search.

Advantageously, the update process reduces unnecessary computations performed each time the index is updated. The update process preserves the computational power of unused indexes between updates. When the index is called (eg, performing a search), the index is updated.

Query model

The faceted search engine gives users the power and control of the corporate search interface, but in an easy-to-understand manner without a steep learning curve. To perform the query, the user selects one or more facet values and indicates whether the result should, cannot, or should not have the selected facet value. For example, suppose a user is looking for a media object taken on a holiday other than Thanksgiving and wants to depict a shoe, dress, bag, or sunglasses. Users can define the following query:

MUST: Holiday type event

NOT: Thanksgiving event

OR: shoe appearance object

OR: Dress appearance object

OR: Invalid appearance object.

The user can also specify a complex query for a picture of the user's son around the house only in the fall while his wife was taking pictures. The query can be specified as follows:

MUST: The appearance of a human son

MUST: Man Wife Photographer

MUST: home location

MUST: Autumn event

NOT: inside the house

NOT: Everyone appears

Query mechanism

For each criterion defined by the user, the facet search engine can create a selector object that operates on the facet index. The purpose of the selector is to match the criteria with the original media object identifier. Also, the number of possible media object identifiers matching a given criterion can be returned, and possible matches are returned in turn. The faceted search engine sorts the index in ascending order based on possible matches. In this way, the index with the least number of matches is executed first, and the faceted search engine iterates through the index list to calculate a value that matches all criteria.

MustSelectors []

CanSelectors []

NotSelectors []

mustSelectorsIdx = 0

canSelectorsIdx = 0

notSelectorsIdx = 0

For each user criteria

If criteria is MUST

MustSelectors[mustSelectorsIdx] = new Selector(criteria)

mustSelectorsIdx++

else If criteria is CAN

CanSelectors[canSelectorsIdx] = new Selector(criteria)

canSelectorsIdx++

else If criteria is NOT

NotSelectors [notSelectorsIdx] = new Selector(criteria)

notSelectorsIdx++

For each MustSelectors

calculate the number of resulting row ids

Sort the MustSelectors selectors by the number of resulting row ids in ascending order

Result = MustSelectors[0].rowIds

For each mustSeletor in MustSelectors<1 to mustSelectorsIdx-1>

Result = Result AND mustSeletor.rowIds

For each NotSelectors

calculate number of possible row ids

Sort the NotSelectors by the number of row ids in ascending order

For each notSeletor in NotSelectors <0 to notSelectorsIdx -1>

Result = Result NOT notSeletor.rowIds

ORResult = CanSelectors[0].rowId

For each canSeletor in CanSelectors <1 to canSelectorsIdx-1>

ORResult = ORResult OR canSeletor.rowIds

Result = Result AND ORResult

Advantageously, multiple query algorithms can be executed simultaneously. Thus, each selector is independent and avoids race conditions. Selectors are additionally executed in the strictest to least strict order (eg, AND followed by NOT followed by OR). Selector order provides the ability to skip execution of less stringent selectors if the result set is empty.

Also, the algorithm can be optimized as follows:

Performing the AND and NOT parts of a query using criteria that have an index in memory.

If the result has a row ID,

Perform queries against criteria with unmodified indexes.

If the result has a row ID,

Query the rest of the criteria.

This optimization reduces the need for unnecessary loading of the index if we know that the query result contains 0 records.

Selection behavior

A selection operation is performed to select matching row identifiers for specific selection criteria. Each index type implements a selector in a specific way that corresponds to the structure of a specific index type.

Discrete index

The discrete index can be numeric or non-numeric. Numerical discrete indexes provide mathematical operations including, but not limited to, range selection, greater than, less than, for example.

Less than X:

-Binary search the facet value array to find the position of the maximum value at an index less than X. The position of the maximum value indicates the position of the original record ID array. Then, you can collect the previous row ID before the maximum value position.

· Greater than X:

-Binary search the facet value index to find the location of the minimum value greater than X. The position of the minimum value indicates the position of the original record ID array. Then, you can collect the row ID after the position of the minimum value.

·X like:

-Binary search the facet value index to find the position where the value equals X. Positions with the same value indicate positions in the original record ID array. The raw record ID array has a starting position. The end position is determined at the next index of the facet value index.

The derived value index may also be numeric or non-numeric, where the numeric derived value index may support mathematical operations.

Less than X:

-Binary search the facet value array to find the position of the maximum value at an index less than X. The maximum position is indicated in the base index value arraRaw record ID array. Then, the row ID before the maximum value position is collected.

· Greater than X:

-Binary search the facet value index to find the location of the minimum value greater than X. The position of the minimum value indicates the position of the original record ID array. Then, the row ID after the minimum value position is collected.

·X like:

-Binary search the facet value index to find the position where the value equals X. Positions with the same value indicate positions in the original record ID array. The raw record ID array has a starting position. The end position is taken at the next index of the facet value index.

Query basic operations

Query operations (eg, AND, NOT, and OR) may be performed in various embodiments. For large arrays, a compressed bit vector array is used to store row identifiers (e.g., a lowering array) and binary bitmasks are used to perform operations suitable for these data structures. For small arrays, the following algorithm is applied:

The AND operation works on two arrays of row identifiers, rowIDs1 and rowID2. The results are included in the resultsID. The algorithm for the AND operation operates in linear time (i.e. O(n)) and includes:

Sort rowIDs1 and rowIDs2 in an ascending order. Using radix sort

Assign rowIDs 1 to the array of fewer elements

Pointer1 = 0

Pointer2 = 0

ResultsPointer = 0

while Pointer1< number of count rowIDs1 && Pointer2< number of count rowIDs2

if rowIDs1[Pointer1] == rowIDs2[Pointer2]

resultingIDs[ResultsPointer] = rowIDs1[Pointer1]

ResultsPointer++

Pointer1++

Pointer2++

else if rowIDs1[Pointer1]> rowIDs2[Pointer2]

Pointer2++

else

Pointer1++

The NOT operation is similar to the set subtraction operation. A set of row IDs is subtracted from the result. The algorithm for the NOT operation also works in linear time (i.e. O(n)) and includes:

Sort rowIDs1 and rowIDs2 in an ascending order. Using radix sort

Pointer1 = 0

Pointer2 = 0

ResultsPointer = 0

while Pointer1< number of count rowIDs1 && Pointer2< number of count rowIDs2

if rowIDs1[Pointer1] == rowIDs2[Pointer2]

Pointer1++

Pointer2++

else if rowIDs1[Pointer1]> rowIDs2[Pointer2]

Pointer2++

else

resultingIDs[ResultsPointer] = rowIDs1[Pointer1]

ResultsPointer++

Pointer1++

while Pointer1< number of count rowIDs1

resultingIDs[ResultsPointer] = rowIDs1[Pointer1]

ResultsPointer++

Pointer1++

The OR operation works on two arrays of row identifiers, rowIDs1 and rowID2; The result is called resultsID. The results are included in the resultsID. The algorithm for the AND operation operates in linear time (i.e. O(n)) and includes:

Sort rowIDs1 and rowIDs2 in an ascending order. Using radix sort

Assign rowIDs 1 to the array of fewer elements

Pointer1 = 0

Pointer2 = 0

ResultsPointer = 0

while Pointer1< number of count rowIDs1 && Pointer2< number of count rowIDs2

if rowIDs1[Pointer1] == rowIDs2[Pointer2]

resultingIDs[ResultsPointer] = rowIDs1[Pointer1]

ResultsPointer++

Pointer1++

Pointer2++

else if rowIDs1[Pointer1]> rowIDs2[Pointer2]

if resultingIDs[ResultsPointer] <rowIDs2[Pointer2]

resultingIDs[ResultsPointer] = rowIDs2[Pointer2]

ResultsPointer++

Pointer2++

else

if resultingIDs[ResultsPointer] <rowIDs1[Pointer1]

resultingIDs[ResultsPointer] = rowIDs1[Pointer1]

ResultsPointer++

Pointer1++

Multimode user interface

In various embodiments, the media platform 120 supports a multi-mode UI that can adapt to various major types of activities. For example, the UI may be in a visual search mode, an object connection mode, or an object manipulation mode. Thus, the UI is an endless set of options so you can focus on your main activities without cluttering the screen. In various embodiments, the commands are placed in substantially the same relative location or area on the screen, so that the user can access the commands into memory. The UI may be controlled by a mode switching control that informs the user of which mode the UI is in and allows the user to quickly switch to another mode.

In an exemplary embodiment, a system for interacting with a media platform comprises: a media platform having a processing device configured to control a multi-mode application of the media platform by providing a user interface (UI) to a user device, the user The interface includes a multi-mode navigation area, each mode navigation area is associated with a mode of a multi-mode application, and each mode navigation area includes a specific set of functions for controlling a mode associated with the mode navigation area. The UI displays a navigation area for an active mode based on detecting device activity, and the UI includes a global navigation menu for switching to an inactive mode and suppresses functions associated with the inactive mode.

For example, each mode navigation area may include a specific set of functions for controlling a mode associated with the mode navigation area; When the navigation area is displayed, a specific set of functions is maintained in the mode navigation area. In an exemplary embodiment, the global navigation menu may be controlled by a user's gesture input. In order to display a specific set of functions, the UI may include one or more expandable sub-mode navigation areas. One or more expandable sub-mode navigation areas may be displayed based on detected device activity, while the UI suppresses functions associated with inactive sub-modes.

The UI may include an active mode indicator in the global navigation menu. In some embodiments, the processing device detects device activity by tracking the user's activity pattern and suggests the next mode by highlighting a shortcut in the global navigation menu. The multi-mode application may include at least one of a visual search mode, an object manipulation mode, or a data input mode.

Multimode control switch

18 illustrates a multi-mode control switch 1800 according to various embodiments. Referring to FIG. 18, a multi-mode control switch 1800 may be displayed on a user device 110 (eg, a smart phone). The multi-mode control switch 1800 displays the current mode "visual search" as well as additional modes that the user can switch. Alternatively, the user can use a swipe gesture (eg, on a touch screen) to switch between different modes or display additional modes.

Multimode overview

19A-C illustrate a multi-mode UI according to various embodiments. 19A-C, the multi-mode UI may be used to associate semantic information with a media object and search for the media object. The stamping mode provides an interface for determining association with media objects. The UI is structured to enable users to efficiently search for media objects that can be associated in a very effective way that allows them to perform batch association. The stamping UI controls are related to related functions. For search, the user can switch to the visual search mode (right). The multi-mode control switch, as described in FIG. 18, enables quick navigation between modes. In search mode, the visual search UI provides control options focused on searching the media library.

Advantageously, the multi-mode UI prevents the user from guessing which actions can be used for different activities (eg, search, stamping, sharing, etc.). The multi-mode UI efficiently categorizes and displays activities related to each activity mode. UI commands are placed in the same location or area of the screen for a given mode. Thus, the multi-mode UI reduces the cognitive requirements for individual users without compromising the functionality of the application.

Masking access control

In various embodiments, media platform 120 controls access to stored data objects (eg, media objects stored in data store 125) in a manner that does not require a user account. Conversely, access is controlled based on automatic or manual data object protection rules orthogonal to user account mechanisms. Each data protection rule selects a specific data object to be protected and can be turned on or off to make the object inaccessible or accessible respectively. By combining the data protection rule states, an effective data masking layer can be calculated. The data masking layer determines whether a given data object can be accessed. Presenting data protection as simple data selection rules simplifies complex access control mechanisms.

Data access calculation

20 illustrates data access calculations according to various embodiments. In various embodiments, the media platform 120 may calculate data object visibility by performing an effective data masking calculation. You can combine access control rules to create a masking mechanism. Each rule identifies an inaccessible (eg hidden) data object. When multiple rules are combined, data objects not covered by any one rule are visible to the user.

In some embodiments, media platform 120 implements a masking mechanism by generating a lock count and attaches the lock count to each data object. When an access control rule is activated, the rule identifies the data object associated with the rule, determines a corresponding lock count (e.g., by one), and limits access to the associated data object. When a user performs a search, the system restricts access to data objects with lock counts greater than zero (eg, hiding data objects from search results or preventing access to data objects). Conversely, a data object with a lock count of 0 is displayed and can be accessed by the user. In some embodiments, media platform 120 may implement a masking mechanism by performing a check as to whether the selected data object is associated with any active access control rules.

Access control workflow

21 shows a process 2100 for enforcing access control in accordance with various embodiments.

Referring to FIG. 21, a first UI requesting a user to reset a user ID before editing a rule is provided (1). A second UI is provided so that the user can create a new access control rule or edit an existing rule (2). The second UI is a dynamic UI based on metadata available for picking a related tag. Also, the 2nd UI negotiates two services. First, the second UI may negotiate with an ontology-based tag search engine service to support an access control editor interface to facilitate the addition of related tags by the user. For example, a user may want to pick tags for cities in Italy, and an ontology-based tag search engine helps users find them efficiently.

Second, the second UI may negotiate with an appropriate meta data recommendation engine indicating what kind of meta data is available for a specific object type. For example, a video may have a duration as metadata, but a text document may have a word or character count. The engine helps narrow the selection so that users can easily edit it.

After the user confirms the final version of the access control rules, the access control rules are packaged for efficient storage and transport (3). The rules are ready to be executed by the media platform 120. The access control rules are then stored (for example, in existing rules and databased rules).

The effect of the rule is pre-calculated for efficient enforcement at run time, and these effects are stored with each data object (4).

Alternatively, the access control rule may be a symbolic link access control rule, which is a simple group of hand-selected data objects. In one embodiment, symbolic links are used to identify files in the locked file system. Therefore, when the symbolic link group is locked, the actual file is also locked. Data control locks implemented using symbolic links are separate from the organizational structure.

The access control rule may also be a meta data access control rule in which the access control rule is based on meta data instead of a tag.

The access control rule may also be a keyword based tag access control rule in which the tag is a keyword matching tag rather than an ontology based tag.

The access control rule may also be an ontology based on tag access control rules.

Access control rule state change

The access control rule state can be changed manually or automatically.

22 illustrates an automatic change of an access control rule state according to various embodiments. Referring to FIG. 22, an external system can operate in conjunction with a masking access control system. The external system is responsible for which access control rules are effective and which access control rules are not. For example, the operating system may control access control rules implemented on a physical storage system controller (eg, a hard drive or solid state drive (SSD) controller). In this case, the operating system can add an extra level of protection that works in conjunction with the base operating system level. As shown in Fig. 22, in state 1, the external system sends a command to the described system instructing the command to turn off one or more rules. After changing the state of the corresponding rule, the system checks the new state of the access control rule and responds in state 2.

23 illustrates a manual change to an access control rule state according to various embodiments. Referring to FIG. 23, in a manual mode, a user controls the state of an access control rule. When the user wants to change the access control rule state, the system provides a UI through which the user can edit the access control rule. For example, when a user is requested to authenticate with a login screen to confirm a user ID, the login screen may take a password form, a photo code form, a lock key form, or other authentication form. A UI listing the access control rules and the status of each access control rule (eg, enable, disable, enable, disable, etc.) may be displayed. When the user clicks on one of the rules, a UI is displayed to the user to change the status of the rule (eg, enable, disable, enable, disable, etc.).

In accordance with various embodiments, a system that enforces restrictive access control to a set of media objects includes a single device. The single device is configured to: determine, based at least in part on the first access control rule, to block access to at least one first media object included in the media object set; Determine to block access to at least one second media object included in the media object set based at least in part on the second access control rule; It may be configured to present to a user of a single device at least a third media object that is included in the set of media objects but is not a first media object and a second media object. The apparatus may be configured to provide a third media object other than the first media object and the second media object based at least in part on a lock count associated with each of the first media object, the second media object, and the third media object. .

Limited access control in independent and distributed multi-system environments

In some embodiments, the data is distributed across multiple independent systems, including, but not limited to, user device 110, data store 125, first data source 140 and second data source 150. Can be. Media platform 120 may synchronize access control rules on separate systems in a separate higher priority synchronization channel than for data synchronization. In addition, data object metadata can also use separate synchronization mechanisms and/or channels, so that each system can apply rules separately from the centralized system.

synchronization

In a multi-system ecosystem, different systems (e.g., data store 125, first data source 140 and second data source 150) are connected together so that users can access the same access control rules for any one system. Make them reusable. The media platform 120 synchronizes an access control rule, a data object, and metadata for the data object so that each system can operate independently. Multiple synchronization networks can operate independently. For example, an access control rule synchronization network can operate in a substantially real-time manner at a high priority level. Metadata synchronization networks can also operate at high priority levels. The data object synchronization network may be a third independent network.

According to an exemplary implementation, the system may enforce restrictive access control on the set of digital objects accessible by the user's first device and the second device. The system includes a user's first device configured to detect an update associated with a first system access control rule, wherein the first system access control rule applies to at least one first digital object included in a set of digital objects on the first device. Block access to; Determine to block access to at least a second digital object included in the set of digital objects on the second device based, at least in part, on the update to the first system access control rule; To provide the second device with an update associated with the first system access control rule to maintain limited access control to the set of digital objects on the second device.

According to an exemplary implementation, a system for enforcing restricted access control to a set of media objects includes multiple devices for a single user. The first device further comprises: determining, based at least in part on the first access control rule, to block access to at least one first media object included in the media object set; Determine to block access to at least one second media object included in the media object set based at least in part on the second access control rule; It may be configured to provide a user of the first device with at least a third media object included in the set of media objects but not the first media object and the second media object. The system may include a user's second device, the first access control rule may include a general rule applicable to the first device and the second device, and the second access control rule may be applied to the first device. It may include device specific rules that are not applicable to the second device.

According to an exemplary implementation, the first device and the second device are configured to participate in a browsing session in which a user of the second device browses a set of media objects through the first device. For example, a first device and a second device may be configured to perform a browsing session based on a third access control rule applicable to a browsing session between a first device and a second device for a user. The third access control rule may block access to the third media object included in the media object set. For example, the first device may be configured to provide at least one second media object to a user of the second device, but not the first media object and the third media object.

Central server

In some embodiments, synchronization may occur with a central server or cloud acting as a Maest. All changes are first transmitted to a central server or cloud before the changes are propagated to other systems. 24 illustrates central server-based synchronization according to various embodiments.

Peer-to-peer

In some embodiments, the peer-to-peer paradigm is applied to synchronize multiple systems. For example, peer-to-peer synchronization can use independent versioning to keep track of the latest updates. 25 illustrates peer-to-peer synchronization according to various embodiments.

Hierarchical synchronization network

In some embodiments, some systems may act as a local synchronization server that coordinates state between local devices. The local synchronization server is responsible for communication with the centralized server. For example, if there is no mobile network, the WiFi hotspot can host a server that communicates with a central server as well as coordinates synchronization between various systems connected to the hotspot. 26 illustrates hierarchical synchronization according to various embodiments.

Data access calculation

In various embodiments, data object visibility is calculated by the access control rule state cascade and affects the data masking calculation.

Access control rule state cascade

In order to support access control rule distribution and peer-to-peer browsing, multiple layers can be defined in which access control rules can be turned on and off. Such layers may be, for example, a general purpose layer (i.e. for the entire ecosystem), a system or device layer (i.e. for each individual device or system), an application layer (i.e. for a system implemented at the platform level) and a session layer (i.e. Peer-to-peer or for temporary change), but is not limited thereto.

In various embodiments, access control rules may be turned on and off at each layer. To compute the state of each rule, rule states are cascaded from the least specific (i.e. general) layer to the most specific (i.e. session) layer. The state of each rule is calculated by allowing the rule state of a more general layer to redefine the rule state of a more specific layer. 27 illustrates an access control rule cascade according to various embodiments.

Effective data masking computing

Effective data masking calculations are performed in a manner similar to that described above, but the process is repeated for each target session.

Peer-to-peer browsing session using access control

In the peer-to-peer browsing mode, other systems can browse data objects stored in the host device in a temporary manner while maintaining access control rules. For each satellite system requesting hosted browsing, the system can create a browsing session and change the state of the access control rules for a particular browsing session. Session rules may be included in the rule state cascade calculation as described above. In some embodiments, peer-to-peer browsing transforms the host device into a temporary server for data browsing. 28 illustrates a peer-to-peer browsing session according to various embodiments.

Start browsing session

29 shows a process 2900 for initiating a peer-to-peer browsing session in accordance with various embodiments. In order for the user to initiate a peer-to-peer browsing session, the user is presented with a set of guests that the user can invite to browse their device (1). The user selects the desired guest and then continues checking the access control rules. The valid status of the access control rules for the new session is displayed to the user, allowing the user to change the valid status of each rule for that particular session (2). Users can enable and disable each access control rule for a specific guest session (3). The user is presented with a UI for viewing the currently active browsing sessions with guests participating in each session (4). The user can add a guest, remove a guest, and/or terminate a session through the UI.

According to an example implementation, the system may enforce limited access control for a user while browsing another user's device.

Visual access code

In various embodiments, access may be controlled through a visual access code mechanism that makes it easier for users to remember while providing improved security by increasing the possible combinations. The visual access code mechanism is provided through a UI with two entry levels. In the first entry step, the system administrator is asked to select a picture from a series of pictures or images that can be pre-configured. The set of photos can be the same for all users or can be different for each user. Also, the set of photos may be the same for all devices or may differ from device to device.

30 illustrates a process 3000 of constructing a visual access code according to various embodiments. The user may be provided with an image (eg, a photo) in step 1. In step 2, the user creates a subset of the hotspot groups (e.g., 4, 5, etc.)in the possible (e.g., 16, 25, 36, etc.) hotspot groups (e.g., clicks, touches, gestures, etc.). By) being asked to choose. For example, a user may select a subset of 4 hotspots from 16 hotspots displayed in a photo by touching the hotspots in a random order. In some embodiments, the image may have an overlay or marking that makes the hotspot visible and helps the user select and invoke the selected hotspot. The visual access code consists of a photo index associated with the photo selected in step 1 and a subset of the hotspot values selected in step 2. The picture index and coordinates associated with the subset of hotspots may be stored as an encrypted digest.

The user's visual memory is used to store and retrieve registered images and hotspots using the image's visual cue. Visual memory is a form of memory that preserves some characteristics of the senses associated with the visual experience. Visual memory describes the relationship between cognitive processing and the encoding, storage and retrieval of the resulting neural representation. Visual memory occurs over a range of time in eye movements to visually navigate to previously visited locations. Visual access codes containing a subset of the hotspots of registered images can be stored longer and more easily recalled to provide authentication. In addition. Because the available hotspots vary from photo to photo, you are less likely to choose a trivial visual access code than a trivial alphanumeric password (eg, "1111", "1234", "password", etc.). The user can memorize visual information resembling an object, place, animal, or person as a mental image of the visual access code. The user can recall visual access codes as patterns in long-term visual memory using different areas of the prefrontal and anterior cortex.

According to various embodiments, a system for processing a visual access code is configured to provide, to a user of a first device or service, a plurality of images; Receive, from the user, a selection of a first image from among the plurality of images; Receive, from a user, a first selection of at least one of a plurality of hotspots included in the first image; And a first device configured to generate a visual access code based at least in part on the selection of the first image and the first hotspot.

According to an exemplary implementation, the registration process of the visual access process includes a first selection from an image grid (e.g., photo, picture, complex shape, image, etc.) and a series of hotspot locations of the first selected image (e.g., Pixel location, screen coordinates, overlay point, etc.). For example, the user may be provided with a grid of photos depicting various landmarks, and may select a photo depicting the landmark from the grid. Then, from the selected landmark photo, the user can select a series of hotspot locations in the selected photo. For example, the selected hot spot location may be a location on a picture or image corresponding to another part such as a landmark, a background, or a border of the picture. The features depicted in the picture or image serve the user as a visual cue corresponding to the hotspot location selected by the visual cue. Thus, features depicted in photos can be more easily stored in the user's memory than conventional alphanumeric combinations.

The system can efficiently store user selections of a set of photos and hotspots during the registration process as described in more detail with reference to FIGS. 32-41. For example, each of the plurality of photos may be associated with a corresponding index number. For example, each index number may contain a globally unique picture identifier. In one embodiment, the first phase selection of an image from an image grid may include additional pages of multiple image grids. The user can scroll through several pages of the image grid to identify the image registered for the first phase image selection. Each image in multiple grids of images may contain an index number based on a picture identifier that is unique across each image.

Each image may include a number of predefined hotspots for the user to select a subset for the second phase set of hotspots. According to an exemplary implementation, each of the plurality of hotspots is associated with a corresponding hotspot identifier that may be encryptedly stored along with an image index number. According to another exemplary implementation, each of the plurality of hot spots may be associated with a 2D coordinate of a corresponding pixel in the photo.

According to one embodiment, the visual access code may be implemented by, for example, an authentication service on the target device or service. For example, a website could replace the existing alphanumeric login form with a two-step method of entering a visual access code to authenticate the user. After the user registers the visual access code with the authentication service, the authentication service (e.g., mobile device or website) securely stores the user's visual access code to match subsequent inputs of the visual access code with the stored registered access code. You can use a variety of techniques. For example, after the visual access code of user registration, the authentication service may convert the identifier associated with the first step and the coordinates associated with the second step into a text string, and encrypt the corresponding string and store it. Then, when the user visits the destination again and enters the visual access code, the authentication service can decrypt the stored string to verify the user's permission to access the destination.

According to another embodiment, the visual access code may be implemented by a client-side Visual Authorization Interface (VAI) that receives a visual access code from a user and outputs an alphanumeric password to various destinations. In this embodiment, the VAI includes an algorithm to regenerate the alphanumeric password based on the destination. As an example, a user can access a web site using a conventional alphanumeric login form with VAI. To use VAI with a destination, users use VAI to set or register an alphanumeric password.

The visual access code system may further include a system and method for entering a visual access code via an on-screen virtual input mechanism or a visual authentication interface (VAI). VAI acts as a client-side visual password entry software that does not require support from other applications or websites. VAI provides the user with a user interface to visually enter a password, and the software then encodes these visual access codes into regular alphanumeric characters suitable for current websites and applications. The system does not store passwords anywhere, and consistently generates them every time a user enters a visual access code.

For example, the virtual keyboard may be a dedicated VAI for entering a visual access code. VAI can perform client-side authentication for visual password input through a visual access code process. VAI provides the user with an interface to enter a visual access code regardless of the device hardware. In other words, VAI provides compatibility for secure authentication that does not require hardware such as fingerprint readers, and maintains the integrity of visual access codes regardless of locally stored passwords.

Users can go to their own password reset form to launch VAI and fill in the destination's password form with an alphanumeric password. As described later, the VAI consistently regenerates the alphanumeric password of the destination based on the visual access code input by the user. Also, if the same visual access code is entered into a VAI of a different destination, the VAI generates a different alphanumeric password. Accordingly, the VAI can authenticate the user using a visual access code compatible with the conventional destination login method. After the user registers for a destination using VAI, the destination stores the output of the VAI (eg, an alphanumeric password). The output of the VAI is used as the target-side authentication key, while the visual access code is the client-side authentication key.

For example, after the destination records the output of the VAI, the user can revisit the destination to start the VAI and enter the user's visual access code, and the VAI outputs a password that matches the password previously stored at the destination. In various embodiments, the output of the VAI is a unique identifier of the user, selection of the first image, a unique identifier of the first image, an image blending algorithm, selection of a hotspot, movement of the hotspot coordinates, and/or using one or more one-way encryption algorithms. It can be based on a hexadecimal digest.

After the user registers a series of hotspots or hotspot sets of the selected image, the user may be presented with a visual access code process for user authentication to the device or service. For example, a user may navigate the access interface of a device or service, and several photos or images may be presented during the first step of the visual access process. The user must retrieve the correct one previously selected during the registration process from among the several pictures presented. For example, the user may be presented with a grid of pictures depicting various famous landmarks. The user's registered image may be grouped into a plurality of identical photos generated during the registration process, or may be grouped into photos different from the image presented during the registration process. The user first selects a registered image from a plurality of images. For example, the image selected by the user may be matched with an index number of a photo identifier that is unique throughout the photo.

31 shows an exemplary lock code management interface for use with a visual access code. The lock code management interface 3100 allows a user to manage visual access codes, configure visual access code preferences, assign user profiles, and the like. In an exemplary embodiment, the lock code management interface allows a user to configure different visual access codes based on application categories such as media applications, financial applications, business applications, and the like. The master user can configure multiple visual access codes for multiple sub-users of a service or device. For example, parents can configure guest visual access codes to allow children to access gaming applications. In another example, the spouse may configure the partner's visual access code to allow the spouse's partner to access the financial account, but not the spouse's social media or messaging account.

Fig. 32 shows an exemplary flow 3200 for configuring a visual access code according to an exemplary embodiment. The process begins with collecting a unique identifier from the user. Process 3200 uses one-way cryptographic encoding to generate a consistent set of images for the user, as further described with reference to FIG. 33. Workflow 3200 proceeds to determine if the user wants to set a master password, as further described with reference to FIG. 36.

33 illustrates an exemplary process 3300 for mapping a user identifier to photo selection mapping according to various embodiments. Process 3300 may begin with the user providing a unique identifier. Process 3300 may determine a one-way encryption code and generate a unique user hexadecimal digest, as further described with reference to FIGS. 35-36. The unique user hexadecimal digest can then be used to create a unique photo and hotspot listing, as described in more detail with reference to FIGS. 40-41. The process 3300 then presents the list of photos to the user to register the password, as described in more detail with reference to FIGS. 37-39.

Compact encoding

In some embodiments, the visual access code may be encoded using compact encoding. For example, in compression encoding, each picture can have an index from 0 to 8 and each hotspot can have an index from 0 to 15. There may be no fixed correlation between the index assigned to the hotspot and the location of the hotspot in the picture. The correlation between the index assigned to the hotspot and the location of the hotspot in the picture varies depending on the picture. In one embodiment, the index value associated with the hotspot is randomly assigned. Thus, random index assignment to the hotspot creates a secure access password.

In the example above, compact encoding produces 16 possible values. Four of these values are chosen multiple times in random order, resulting in 3,876 possibilities. Because there are an additional 9 different photos. The number of possible combinations increases to 9×3,876 = 34,884. This is more than three times the number of possibilities offered by conventional 4-digit cryptography.

Position encoding

In some embodiments, the visual access code may be generated using positioning encoding based on the coordinates of each selected hotspot. In the case of the same picture, the coordinates of each hotspot may be fixed, but the coordinates cannot be moved for each picture. Table 1 shows how a simple hotspot index encoding maps to a coordinate index as described in the previous section. For example, the hot spot index 2 corresponds to the values 140 and 59 of photo 1 and the values 89 and 147 of photo 2. With compressed encoding, the value 2 is shared between photos, but the coordinate values for the same hotspot are not shared between photos. Also, the number of digits stored is also increased (for example, to 8 instead of 4 values). Thus, positional encoding creates more possibilities and makes the corresponding visual access code more difficult to break.

In some embodiments, the visual access code may depend on the size and/or resolution of the photo. For example, in a 500×500 pixel photo, each hotspot may generate codes from 0 to 499 on the horizontal axis and 0 to 499 on the vertical axis. As such, 4 hotspots are equal to 8 digits, providing 1.02432860e+17 possibilities. Multiplying this number further by the number of pictures (e.g. 9), for a password that is much stronger than a conventional 8-character long alphanumeric case sensitive password with special characters (e.g. 2.02095455e+11 possibilities). Yields ~9e + 17 possibilities of

Hotspot index Photo 1 coordinates Photo 2 coordinates 0 (0,0) (100,100) One (40,50) (24,135) 2 (140,59) (89,147) 3 (240,15) (29,225) 4 (370,50) (54,135) 5 (140,150) (214,335) 6 (78,150) (334,235) 7 (67,20) (344,185) 8 (80,500) (124,195) 9 (90,310) (249,435) 10 (140,240) (214,235) 11 (400,150) (314,135) 12 (230,60) (245,135) 13 (312,70) (124,235) 14 (32,80) (274,535) 15 (42,98) (214,335)

Location encoding with unique photo identifier

In some embodiments, all photos may be associated with a globally unique identifier. As such, password storage is system or user specific. Users cannot choose the same password for two different systems. For example, the password of (0,100,101,200,201,300,301,400,401) encoded using location coding corresponds to the first picture (i.e., picture 0). However, by incorporating a unique photo identifier, a password of (38A52BE4-9352-453E-AF97 5C3B448652F 0,100,101,200,201,300,301,400,401) is generated, and '38A52BE4-9352-453E-AF97-5C3B448652F0' is a unique photo identifier throughout the photo. In various embodiments, the globally unique identifier may be a value of length that is difficult to guess (eg, a 16-character long number).

34 illustrates an exemplary unique user hexadecimal digest in accordance with various embodiments. The hexadecimal digest may include a photo selection index, a filter blending algorithm identifier, a final password mapping algorithm identifier, a photo filter bitmap, and a hot spot identifier. For example, a unique user hexadecimal digest may start with 8 bytes designated for the photo selection index followed by a 1 byte filter blending algorithm identifier followed by an indicator for the final password mapping algorithm. In an exemplary embodiment, the photo filter bitmap may consist of 27 bytes. According to an exemplary embodiment, the unique user hexadecimal digest may include 25 bytes to indicate the hotspot shifting index.

35 shows an exemplary registration process for assigning a visual access code according to an exemplary embodiment. The registration process 3500 may begin with a user navigating to a destination requiring authentication. The virtual input (eg, VAI) method can detect conventional login forms that require a username and password. After the user has entered a user name in a conventional login form, the registration process 3500 method may provide a registration interface for assigning a visual access code to a destination using a hexadecimal digest. As described above, the user may proceed through the VAI by selecting the first image and a series of hotspots to register a new visual access code for the destination.

Based on the provided username and destination identifier, the VAI can regenerate the password based on the hexadecimal digest to match the stored password with the destination.

The registration process 3500 may proceed with completing a conventional login form with a password based on a hexadecimal digest. For example, registration process 3500 may generate an alphanumeric password using one-way encryption encoding and seed the password with the destination identifier. The password seed is used to supply a one-way encryption algorithm before generating an alphanumeric password, as described with reference to FIG. 36. In the seed, there is a direct one-to-one mapping between the seed and the hotspot selected by the user. This process generates a password seeded with the application/website destination name prior to one-way encryption so that different application/website destinations have different passwords even if the same visual password is used. Thus, the access code process consistently replicates password generation based on user input, so the user system does not need to store passwords for each site.

According to an exemplary embodiment, the algorithm for setting the visual access code may include:

Encode the phrase with one-way cryptography i.e. sha-512

let uniqueUserHexDigest = onewayCryptography(user phrase)

save uniqueUserHexDigest to hostsystem Keystore

Use the cryptography to generate the list of photos

let basePhotoIndex = getByteAtIndex(0,8, uniqueUserHexDigest)

Repeat index i 0 to(number of Photos to use for key)-1

Let photoIndex =(basePhotoIndex + i) modulus(total number of photos in the system)

let photo = getPhotoWithIndex(photoIndex)

let blendingAlgorithmIndex= getByteAtIndex(8, uniqueUserHexDigest)

let photoFilterBitmap = getNumberOfBytesFromPosition(27,12, uniqueUserHexDigest)

let photoBlendingFilter = generateFullPhotoFromBitmap(photoFilterBitmap)

let finalPhoto = blendPhotoWithFilterUsingAlgorithm(photo, photoBlendingFilter, blendingAlgorithmIndex)

let hotspots = getPhotoHotSpotsForIndex(photoIndex)

let h=0

let shiftedHotSpots = Array of size of 25

for each hotspot

let hotspotShift = getByteAtIndex(h+38, uniqueUserHexDigest)

let adjustedHotSpot = adjustHotSpotCenterByShift(hotspots[h], hotspotShift)

shiftedHotSpots.add(adjustedHotSpot)

Upon visiting the destination, the VAI may determine an associated visual access code and present a virtual input method to the user to authenticate the user. In response to successful visual access code authentication through the virtual input method, the system can fill in an existing login form with an alphanumeric password assigned to complete destination authentication.

36 shows an exemplary process 3600 for encoding a visual access code with a password according to an exemplary embodiment. The process 3600 may generate a cryptographic seed by inputting the SHA-512 picture of the hotspot location classified by the X-axis and the application/website destination identifier. Process 3600 generates a hexadecimal digest that is used to map the alphanumeric password using the encrypted password with a one-way encryption algorithm. In an exemplary embodiment, the resulting password will generate a secure ASCII password that may contain uppercase and lowercase English alphabets and numbers as well as special characters. According to an exemplary embodiment, the algorithm may include:

input passwordHexDigest

let passwordBytes = getNumberOfBytesFromPosition(0,16, passwordHexDigest)

let finalPassword = ""

For each byte in passwordBytes

If( byte == 45 OR

(byte >=48 AND byte <=57) OR

(byte >=65 AND byte <=90) OR

(byte >=97 AND byte <=122))(

// take the value as is

finalPassword.append(byte)

continue to next byte

}

let modByte = byte modulus 63

if(modByte == 0){

modByte += 45

} else if(byte >=1 AND byte <=11){

modByte += 47

} else if(byte >=12 AND byte <=37){

modByte+= 53

}else{

modByte+= 59

}

finalPassword.append(modByte)

37 illustrates an exemplary mobile interface in accordance with various embodiments. To set up a visual access code for the virtual input method, the user starts by selecting a unique passphrase, such as first name, mother's pre-marriage last name, date of birth or favorite location. The two-stage system creates a set of images so that the user can select a registered image. According to an exemplary embodiment, the algorithm may include:

Let photoHexDigest = generateOneWayCryptographyFrom(photoBitmpa)

Let selectedHotspotsXY = ""

For each selected hotspot

Let hotspotXY=getXYForHotspot

selectedHotspotsXY.append(hotspotXY)

End

Let siteOrAppId = collectCurrentSiteOrAppId

Let finalPasswordSeed = concat(photoHexDigest, selectedHotspotsXY, siteOrAppId)

Let passwordHexDigest = generateOneWayCryptographyFrom(finalPasswordSeed)

Let passordMappingAlgorithmIndex=

getByteAtIndex(9, uniqueUserHexDigest)

let asciiPassword= generateAsciiPasswordWithAlgorithm(passwordHexDigest, passordMappingAlgorithmIndex)

Fig. 38 shows an exemplary implementation of entering a visual access code according to an exemplary embodiment. In step 1, the user navigates to a destination website or application that requires authentication or a login screen. The VAI can detect the destination's authentication form and retrieve the visual access code associated with the destination identifier. VAI provides the user with a set of images that include images previously registered by the user for the destination. When the user selects an image that matches the registered image, VAI proceeds to step 2 and presents the user hot spot of the registered image. For example, the registered image may be an image of a house including 16 possible hot spots.

When the user selects a series of hotspots that match the previously registered hotspots, the visual input method proceeds to step 3. For example, the user may identify four hotspots during the registration process by touching different locations on the image corresponding to different parts of the depicted house that match the hotspot selected by the user. In step 3, the virtual input method enters the alphanumeric password stored with the visual access code into the authentication form of the destination. In some embodiments, the virtual input method may present a confirmation message that the user has successfully entered the visual access code. Users can proceed to complete the login by clicking on the destination authentication form without typing an alphanumeric password.

39A-C illustrate exemplary implementations of a virtual input method in a web site according to various embodiments. 39A shows a first step of a virtual input method for presenting to a user several images including previously registered images. The user can click or touch the registered image to check the previously registered image. In response to the user selecting an image that matches the registered image, the registered image may be presented and instructed to identify a series of hotspots of the registered image.

In Fig. 39B, the user is presented with a hotspot selection screen for the virtual input method. The interface can provide the user with multiple hotspots for the selected image. In response to the user selecting the series of hotspots with registered images that match the registered series of hotspots, the visual access code process may authenticate the user to the device or service. For example, the user may click or touch four locations of a photo corresponding to the hotspot location selected by the user during the registration process. For example, the visual access process determines whether the location selected by the user meets the corresponding index number stored during the registration process. According to another implementation example, the visual access process determines whether the location selected by the user satisfies the two-dimensional coordinates of the corresponding pixel in the photo in the registration process.

In an exemplary implementation, to satisfy the second step of the visual access code process for user authentication, a series of hotspots on the registered image may be identified in the same order as the hotspots selected during the registration process. In another example implementation, the series of hotspots on the registered image may be identified in any order to satisfy the second step of the visual access code process for user authentication. Because the permutation of location and visual cues is greatly increased compared to traditional alphanumeric combinations, the user first remembering the correct image and then identifying a series of hotspot locations in the correct image may be sufficient to authenticate the user.

In response to the user selecting an image that does not match the registered image, the user is presented with a non-matching image and may be instructed to identify a series of hotspots of the non-matching image. To authenticate the user, the visual access code process may provide or suppress feedback to the user regarding a first selection of images from a plurality of images. Thus, the unauthorized user may not be notified whether the first selection of images or the second identification of a series of hotspots does not satisfy the visual access code process. Repeated attempts to guess multiple image combinations from image groups and hotspot locations can be detected as brute force attacks.

When the correct hotspot is selected, the virtual input method proceeds to FIG. 39C to present a confirmation message to the user and input a final alphanumeric password in the login form of the destination.

To generate a secure visual access code for each user, the process can use a modified image that is unique for each user that appears to be visually unidentifiable. In one embodiment, the process may include moving the center point of the original image to modify the coordinates of the hotspot and add a blended texture that protects the image data.

40 illustrates an image blending process according to various embodiments. Mixing each user's picture provides each user with a different password that simply cannot be detected by simply observing the user's image selection. In one embodiment, the system combines the texture masking with the original photo via a blending algorithm to generate a modified photo to generate a secure visual access code. Examples of photo blending algorithms may include:

Let resultingPhoto = copyPhoto(originalPhotoSize)

For each x in 0 to photoWidth

For each y in 0 to photoHeight

resultingPhotoPixelAt(x,y) = resultingPhotoPixelAt(x,y) + setTransparencyTo(photoBlendingFilterPixelAt(x,y),20%)

End

End

In an exemplary embodiment, the system selects a blending algorithm and photo filter bitmap based on data stored in a unique user hexadecimal digest. For example, the hexadecimal digest may contain a value representing a simple overlap blending algorithm to create a modified photo. Multiple blending and password mapping algorithms improve the security of the access code.

41 illustrates an exemplary implementation of hotspot location movement according to various embodiments. You can use the same visual image by moving the center point of the original photo coordinates to change the hot spot location value of the image and create a different password for each user while maintaining the visual shape of the image.

An exemplary hotspot movement algorithm may include:

Input hotspotShiftingIndex

For each hotspot

hotspot.x =(hotspot.x-2) +(hotspotShiftingIndex remainder 5)

hotspot.y =(hotspot.y-2) +(hotspotShiftingIndex modulus 5)

End

For example, based on the unique user phrase provided during the registration step, the hexadecimal digest may provide a movement value for moving the center of the hotspot to distinguish the hotspot coordinates for the image for the user. For example, when moving based on the movement value 9 in the unique user hexadecimal digest, the original hot spot center having the location coordinates 30 and 50 moves the hot spot center to the location coordinates 301 and 49.

According to another exemplary implementation, a multiple authentication system (MFA) supports a variety of password entry mechanisms (e.g., alphanumeric, visual, voice, etc.) that can be used to authenticate users to access multiple application/website destinations. . MFA does not store passwords in local or network memory (e.g. password vaults), and the Internet to several different service providers (e.g., third-party applications, cloud services, websites, etc. that contain user authentication). It includes a real-time password generator that generates unique and secure passwords regardless of the connection.

MFA generates a unique and complex password, registers the password with a service provider, and associates it with a personalized recall tool that users can access. For example, personalized recall tools are generally vulnerable to hacking from information that is publicly available or secretly obtained by the user (e.g., social media, public records, confidential information, etc.) and/or It can be based on the user's own personal memory rather than a recovery question.

MFA is compatible and extensible with personal and corporate systems (e.g. financial institutions, universities, health insurance agencies, hospitals, etc.), reducing the direct costs associated with password resets, as well as public relations liability due to legal and security breaches. MFA avoids traditional password problems such as several unique password reminders, synchronization of password storage between devices, availability of cloud-based solutions, and immutable biometric passwords that can be fatal to leaks and abuses.

According to an exemplary aspect, MFA is an authentication mechanism on the user device that is separate from the authentication module of the service provider (eg, application, web site, digital destination, etc.). The multiple authentication system allows users to generate identifier codes and access codes based on personalized recall tools that users can access. Multiple authentication systems use an identifier code and an access code to generate a unique target key for each service provider. For example, the access code can be in alphanumeric, pattern and/or visual format. Multiple authentication systems can operate on user devices without common storage (eg, without password storage) and do not require an Internet connection to access each service provider's unique target key.

According to the implementation, the identifier code is securely stored once on each user device and the access code is provided by the user upon authentication with each service provider (eg, application, website, digital destination, etc.). The same identifier code and access code are associated with several different unique target keys. Each unique target key is used to register with a different application/website destination. The application/website destination generally grants access rights by matching the target key received at login with the target key registered to the user by the destination. Thus, a user of a device with a stored identifier code can log in to multiple destinations by entering a single access code each time they log in, and each destination receives a different destination key.

The target key for the application/website destination is not stored on the user side (e.g. user device, password vault, user's cloud storage, hardware key, etc.), but based on a multi-factor authentication system using identifier codes and access codes. Is regenerated. That is, the target key is consistently generated based on the access code and the identifier code that are not stored on the user side.

The identifier code can be securely stored on the user device. Identifier codes can be subject to strong minimum requirements (e.g. length requirement, complexity test, duplicate detection, etc.) and memorable information that can be easily remembered by the user (e.g. favorite movie quotes or life's crucial Moment).

For example, the identifier may be an alphanumeric passphrase or sentence, a visual access code (eg, selecting an image or hotspot through VAI), a pattern, and the like. The identifier code can be entered once into the device and is protected from being displayed or transmitted on the device. For example, the identifier code may be stored in a secure format (e.g., encryption, hashing, etc.) and the device's security mechanisms (e.g., secure hardware space, protected memory, read-only memory, write once read many secure storage), etc.) can be used.

The access code is provided to the device by the user whenever the service provider requests a secure login. A single access code can be entered in a variety of formats (eg, alphanumeric, visual, audio, etc.) to access different service providers and is not transmitted outside the device.

In an exemplary implementation, other user-centric authentication mechanisms such as text passwords, visual passwords, patterns, biometrics, user actions, etc. receive an access code that is used with an identifier code to consistently generate a target key. Access codes may require strong security requirements (e.g. length requirements, complexity testing, duplicate detection, etc.) and can be efficiently entered into the device.

In an exemplary implementation, the access code is at least one of an alphanumeric string, a visual access sequence, a safety lock, a pattern, a gesture, a hidden lock, and a dice word, and the access code is biometric information (e.g., fingerprint, face scan, retina scan, etc.). ) To further encrypt. The biometric information is stored in the device to provide an additional local security layer that can regenerate the target key through fast authentication, and there is no need to transmit the biometric information to an external service or component. Subsequent login requests from the application or service may prompt the user to provide biometric information to decrypt the access code to regenerate the target key.

In addition, the user may be asked to enter the access code from time to time (eg, randomly, countdown clock, number of logins every x times, etc.) to ensure that the access code is remembered. An additional login request from an application or service may prompt the user to confirm the access code by entering an access code in addition to biometric information.

Since the access code is entered more often than the identifier code, the access code can be configured to be entered more efficiently than the identifier code. For example, the access code can be an alphanumeric password, a visual access code (eg, selecting an image or hotspot via VAI), a pattern, etc. and must be different from the identifier code.

The target key is provided to the service provider (eg, via transmission, by filling the password in the destination's password format, etc.). For example, MFA can use strong heuristics to correctly identify a website's login field, use accessibility APIs (e.g., Open YOLO protocol), etc.

As described above, the registration process (e.g., registration process 3500) can generate an alphanumeric password using one-way encryption encoding and seed the password with the destination identifier. The MFA process generates a target key seeded with the application/website target name, identifier code, and access code. Thus, the access code process consistently replicates password generation based on user input, so the user system does not need to store passwords for each site.

No network connection or external resources are required to generate the target key on the configured device. Thus, according to an alternative example, by completing the setup process with an identifier code and an access code, an authorized export mechanism (e.g., matrix barcode, short-range communication, etc.) on the configured device, or by using a networking transmission technique as understood by those skilled in the art. You can finish configuring additional devices by completing the setup process. The example implementation is configured to prohibit network transmission of the identifier code and/or access code for the integrity of the MFA.

An exemplary MFA method includes storing, by the device, an identifier code from the user in secure storage of the device. In response to receiving a login request from the application or service, the user is prompted to provide an access code, and the device generates a target key based on the identifier code and access code. The target key is registered with the application or service to authenticate the user.

In response to receiving a subsequent login request from the application or service: the user is prompted to provide an access code, and the MFA regenerates the same target key for the application or service based on the stored identifier code and received access code. The target key can be consistently regenerated based on the stored identifier code and access code for accessing the application or service.

In addition, in response to receiving a login request from another application or service, the user is prompted to provide the same access code, and the MFA generates a different target key based on the identifier code and access code. Different target keys are registered with different applications or services to authenticate users, and different target keys can be consistently regenerated based on stored identifier codes and access codes for accessing other applications or services.

42 is a block diagram illustrating a wired or wireless system 550 according to various embodiments. 1 and 21, system 550 can be used to implement media platform 120. In various embodiments, system 550 may be a conventional personal computer, computer server, personal digital assistant, smart phone, tablet computer, or any other processor capable device capable of wired or wireless data communication. As will be apparent to those skilled in the art, other computer systems and/or architectures may also be used.

System 550 preferably includes one or more processors, such as processor 560. A coprocessor that manages input/output, a coprocessor that performs floating-point mathematical operations, a special-purpose microprocessor with an architecture suitable for fast execution of signal processing algorithms (e.g., digital signal processors), dependent on the main processing system Additional processors may be provided, such as slave processors (eg, backend processors), additional microprocessors or controllers for dual or multiprocessor systems, or coprocessors. Such a coprocessor may be a separate processor or may be integrated with the processor 560.

Processor 560 is preferably coupled to communication bus 555. Communication bus 555 may include a data channel to facilitate information transfer between storage devices of system 550 and other peripheral components. The communication bus 555 may further provide a set of signals used for communication with the processor 560 including a data bus, an address bus, and a control bus (not shown). Communication bus 555 may be, for example, an industry standard architecture ("ISA"), an extended industry standard architecture ("EISA"), a microchannel architecture ("MCA"), a peripheral component interconnect ("PCI") local bus or Any standard or non-standard bus architecture such as IEEE 488 Universal Interface Bus ("GPIB"), a bus architecture that is compatible with standards published by the Institute of Electrical and Electronics Engineers ("IEEE") including IEEE 696/S-100, etc. It may include.

System 550 preferably includes main memory 565 and may also include auxiliary memory 570. Main memory 565 provides storage of instructions and data for programs executed in processor 560. Main memory 565 is generally a semiconductor based memory such as dynamic random access memory ("DRAM") and/or static random access memory ("SRAM"), including read-only memory ("ROM"). Other semiconductor based memory types include, for example, synchronous dynamic random access memory ("SDRAM"), Rambus dynamic random access memory ("RDRAM"), ferroelectric random access memory ("FRAM"), and the like.

Secondary memory 570 optionally includes internal memory 575 and/or removable media 580, for example a floppy disk drive, magnetic tape drive, compact disk ("CD") drive, digital versatile disk ("DVD"). It may include a drive or the like. Removable medium 580 is read and/or written in a well-known manner. The removable storage medium 580 may be, for example, a floppy disk, magnetic tape, CD, DVD, SD card, or the like.

The removable storage medium 580 is a non-transitory computer-readable medium storing computer executable code (ie, software) and/or data. Computer software or data stored on removable storage medium 580 is read into system 550 for execution by processor 560.

Alternatively, auxiliary memory 570 may include other similar means for causing computer programs or other data or instructions to be loaded into system 550. Such means may include, for example, an external storage medium 595 and a communication interface 590. Examples of the external storage medium 595 may include an external hard disk drive or an external optical drive, or an external magneto-optical drive.

Other examples of auxiliary memory 570 are programmable read-only memory ("PROM"), removable programmable read-only memory ("EPROM"), electrically erasable read-only memory ("EEPROM"), or flash memory (EEPROM). Block-oriented memory, similar to), and may include a semiconductor-based memory. Also included is a removable medium 580 and a communication interface, which allows software and data to be transferred from an external storage medium 595 to the system 550.

System 550 may also include an input/output (“I/O”) interface 585. The I/O interface 585 facilitates input/output between external devices. For example, the I/O interface 585 may receive input from a keyboard or mouse and provide output to a display. The I/O interface 585 may equally facilitate input and output from various alternative types of human and machine interface devices.

System 550 may also include a communication interface 590. The communication interface 590 allows software and data to be transferred between the system 550 and an external device (eg, a printer, network, information source, etc.). For example, computer software or executable code may be transmitted from a network server to the system 550 via a communication interface 590. Examples of communication interface 590 include modems, network interface cards ("NICs"), wireless data cards, communication ports, PCMCIA slots and cards, infrared interfaces, and IEEE 1394 firewire, to name a few.

Communication interface 590 is preferably an Ethernet IEEE 802 standard, fiber channel, digital subscriber line ("DSL"), asynchronous digital subscriber line ("ADSL"), frame relay, asynchronous transmission mode ("ATM"), integrated digital Promulgated service networks ("ISDN"), personal communication services ("PCS"), Transmission Control Protocol/Internet Protocol ("TCP/IP"), Serial Line Internet Protocol/Point-to-Point Protocol ("SLIP/PPP"), etc. It implements industry protocol standards, but can also implement user-defined or non-standard interface protocols.

Software and data transmitted through communication interface 590 are generally in the form of electrical communication signals 605. The electrical communication signal 605 is preferably provided to the communication interface 590 via a communication channel 600. In one embodiment, the communication channel 600 may be a wired or wireless network or various other communication links. The communication channel 600 carries a telecommunication signal 605 and includes a variety of wired or cable, fiber optics, conventional telephone lines, cell phone links, wireless data communication links, radio frequency ("RF") links, or infrared links, to name a few. It can be implemented using wired or wireless communication means.

Computer executable code (ie, computer program or software) is stored in main memory 565 and/or auxiliary memory 570. The computer program may also be received via communication interface 590 and stored in main memory 565 and/or auxiliary memory 570. When executed, such a computer program enables the system 550 to perform various functions of the present invention as described above.

As used herein, the term "computer-readable medium" is used to refer to any non-transitory computer-readable storage medium used to provide computer executable code (e.g., software and computer programs) to system 550. do. Examples of media include main memory 565, secondary memory 570 (including internal memory 575, removable media 580 and external storage media 595), and communication interfaces (including network information servers or other network devices). 590 and any peripheral devices communicatively coupled. This non-transitory computer-readable medium is a means for providing the system 550 with executable code, programming instructions, and software.

In one embodiment implemented using software, the software may be stored on a computer-readable medium and loaded into the system 550 via a removable medium 580, an I/O interface 585, or a communication interface 590. . In this embodiment the software is loaded into the system 550 in the form of an electrical communication signal 605. The software, when executed by the processor 560, preferably causes the processor 560 to perform the features and functions of the invention described above in this specification.

System 550 also includes optional wireless communication components that facilitate wireless communication over voice and data networks. Wireless communication components include an antenna system 610, a wireless system 615, and a baseband system 620. In system 550, radio frequency ("RF") signals are wirelessly transmitted and received by antenna system 610 under the management of wireless system 615.

In one embodiment, the antenna system 610 may include one or more antennas and one or more multiplexers (not shown) that perform a switching function to provide a transmission/reception signal path to the antenna system 610. In the receive path, the received RF signal may be coupled to a low noise amplifier (not shown) that amplifies the RF signal received from the multiplexer and transmits the amplified signal to the wireless system 615.

In alternative embodiments, the wireless system 615 may include one or more radios configured to communicate over various frequencies. In one embodiment, the wireless system 615 may combine a demodulator (not shown) and a modulator (not shown) in one integrated circuit (“IC”). The demodulator and modulator can also be separate components. In the inflow path, the demodulator removes the RF carrier signal, leaving the baseband received audio signal transmitted from the wireless system 615 to the baseband system 620.

If the received signal includes audio information, the baseband system 620 decodes the signal and converts it to an analog signal. Then, the signal is amplified and transmitted to the speaker. Baseband system 620 also receives analog audio signals from the microphone. These analog audio signals are converted to digital signals and encoded by the baseband system 620. Baseband system 620 also codes digital signals for transmission and generates a baseband transmitted audio signal that is sent to the modulator portion of wireless system 615. The baseband transmitted audio signal is mixed with an RF carrier signal to produce an RF transmitted signal that can be sent to the antenna system and passed through a power amplifier (not shown). The power amplifier amplifies the RF transmission signal and sends the signal to the antenna system 610 where the signal is converted to an antenna port for transmission.

Baseband system 620 is also communicatively coupled with processor 560. The processor 560 accesses one or more data storage areas, including, but not limited to, the main memory 565 and the auxiliary memory 570, for example. Processor 560 is preferably configured to execute instructions (ie, computer programs or software) that may be stored in main memory 565 or auxiliary memory 570. The computer program may also be received from the baseband processor 610 and stored in the main memory 565 or the auxiliary memory 570, or may be executed upon receipt. When executed, such a computer program enables the system 550 to perform various functions of the present invention as described above. For example, the main memory 565 may include various software modules (not shown) executable by the processor 560.

The various embodiments may also be implemented primarily in hardware using components such as, for example, application specific integrated circuits (“ASIC”) or field programmable gate arrays (“FPGA”). Implementations of a hardware state machine capable of performing the functions described herein will also be apparent to those skilled in the art. Various embodiments can also be implemented using a combination of hardware and software.

In addition, those skilled in the art will appreciate that the various illustrative logical blocks, modules, circuits, and method steps described in connection with the foregoing drawings and embodiments disclosed herein can often be implemented in electronic hardware, computer software, or a combination of the two. will be. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described generally in terms of functionality. Whether these functions are implemented in hardware or software depends on the specific application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. Also, functional groups within modules, blocks, circuits or steps are for ease of description. Certain functions or steps may be moved from one module, block or circuit to another module, block or circuit without departing from the invention.

Moreover, various exemplary logic blocks, modules, and methods described in connection with those disclosed herein include general purpose processors, digital signal processors (“DSPs”), ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete It may be implemented or performed as hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor can be a microprocessor, but in the other way the processor can be any processor, controller, microcontroller, or state machine. The processor may also be implemented in a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors associated with a DSP core, or any other such configuration.

Additionally, steps of a method or algorithm described in connection with the disclosure herein may be implemented directly in hardware, a software module executed by a processor, or a combination of the two. The software module may reside in any other type of storage medium including RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM or network storage medium. Exemplary storage media may be coupled to a processor, such that the processor may read information from and write information to the storage medium. Alternatively, the storage medium can be integrated into the processor. Processors and storage media can also reside on the ASIC.

43 illustrates an exemplary ID setting process 4300 according to various embodiments. The ghostPassword ID is set to allow users to collectively interact with multiple username/password combinations across multiple server services or applications. Users can centrally manage username/password combinations with a single memorable ID using the critical moment. Interfaces 4310, 4320, 4330 and 4340 illustrate an exemplary ID setting process for setting an ID. For example, you can set an ID by entering a unique phrase used as an ID card. Users can easily remember or pick something that remains memorable. Interfaces can provide examples of critical moments. For example, the decisive moment could be a bad restaurant. If the user is prompted for something like this: "I was being asked to order from a Mule restaurant, and before speaking to Mule, the onlooker always said hey." Other memorable examples include events, movie quotes, and favorite places. It is stored in a secure area of the phone and is protected by hardware.

44 illustrates an exemplary default key generator process in accordance with various embodiments. The password interface may include a keyboard add-on to generate ghostPassword. For example, the key generator may include a keyboard for entering a master password that creates a unique key for each service or application. The key generator keyboard can be integrated with applications installed on the phone by default. At 4420, you can select the type of key generator. Whenever the user creates a password, the user will enter a key. Keys can be generated by providing users with various key input mechanisms. On the 4430, you can choose a key generator type, such as simple alphanumeric. At 4440, it is possible to select a key generator type for the visual password.

45 illustrates an exemplary ghostPassword keyboard interface according to various embodiments. At 4510, the user can switch between the ghost keyboard key generator and the normal keyboard mode. At 4520, the ghost keyboard includes an icon representing an app or service currently in use. At 4530, the user may enter or select a username currently used to log in to the application or service. At 4540, the user may enter the master ghostPassword associated with the ID level. At 4550, the ghost cryptographic key generator authenticates the master ghostPassword and determines a unique secure password personalized for the selected application or service. The unique security password for the application is registered with the application and the ghost cryptographic keyboard continuously regenerates the security password based on steps 4520, 4530, 4540 matching the previously registered security password. At 4560, the generated secure password is filled in the password input field of the selected application or service with the selected user name. Key Generator Cryptographic Keyboard allows users to respond to a Master Goz password to generate personalized secure passwords from multiple applications or services. Thus, the user does not need to memorize or memorize each individualized security password.

46 illustrates an exemplary user name input process according to various embodiments. In an example implementation, a user can set and store a username associated with a registered application or service. At 4610, the mobile device can automatically detect whether the accessibility service has been activated on the device and prompt the user to activate the service or continue to use the clipboard to detect the URL. At 4620-4640, the user can enter a username, email address, or phone number using a password version integer. At 4650, the exemplary interface provides the user with an email address with a previously stored username, eg, a revision count.

47 illustrates an exemplary critical moment interface in accordance with various embodiments. Interface 4700 represents an exemplary critical moment input mode. At 4710, the user may enter a critical moment with a hint. At 4720, the user can use the hotspot to select a visually critical moment.

48 illustrates an exemplary browser ghostPassword input process according to various embodiments. Process 4800 illustrates an example login process using an ID with an application registered in browser mode. For example, the user may be presented with the user name at 4820. In response to choosing a username, at 4830, the user is presented with a first visual password screen. The user selects a previously registered image among the gallery using the visual password keyboard, and at 4840, the user selects a previously registered hotspot from the appropriate image. When the hotspot is properly selected, ghostPassword generates a secure password associated with the application, fills in the secure password information and username on the application login screen, and sends the secure password and username to the application without requiring the user to enter a secure password. At 4850, the selected application authenticates the user and provides user access to the application.

49 illustrates an exemplary ghostPassword management process according to various embodiments. Process 4900 shows an exemplary management screen for use with various embodiments. For example, the interface 4910 may include login, settings, help, information, or ID so that the user can manage or update the registration security password and application control. At 4920, the user may select a previously registered account. At 4930, a user can access a secure password associated with a particular application or service. At 4940, the user can change the settings associated with ghostPassword, for example to export, import, change or reset passwords. Interface 4950 illustrates an exemplary export mode for exporting with an ID and/or secure password. The ghostPassword management interface allows you to export via a variety of communication means, such as QR code, near field communication, sharing options or other communication options.

50 illustrates an exemplary flow of a ghostPassword element according to various embodiments.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles described herein can be applied to other embodiments without departing from the spirit or scope of the present invention. Accordingly, the specification and drawings presented herein are to be understood as representing the presently preferred embodiments of the present invention and therefore representative of the subject broadly contemplated by the present invention. In addition, it is further understood that the scope of the present invention fully encompasses other embodiments that may be apparent to those skilled in the art and thus the scope of the present invention is not limited.

Claims (12)

Storing, by the device, an identifier code from the user in secure storage of the device;
In response to receiving a login request from an application or service:
Prompting the user to provide an access code;
Generating a target key based on the identifier code and the access code;
Registering the target key with the application or service to authenticate the user;
In response to receiving a subsequent login request from the application or service:
Prompting the user to provide an access code; And
And regenerating the target key based on the stored identifier code and the received access code. The method of claim 1,
In response to receiving a login request from another application or service:
Prompting the user to provide an access code;
Generating another target key based on the identifier code and the access code; And
Registering a different target key with another application or service to authenticate the user,
A different target key is different from the target key, and how different target keys can be regenerated based on stored identifier codes and access codes. The method of claim 1,
How identifier codes are stored as hash files. The method of claim 1,
How the identifier code is not transmitted from the device. The method of claim 1,
A method in which an identifier code is stored by reading and writing to multiple secure storage devices of the device. The method of claim 1,
How identifier codes are stored as hash files. The method of claim 1,
The access code is at least one of an alphanumeric code, visual access code, safe lock, pattern, gesture, hidden lock, die word, fingerprint, retina scan, face scan, or other biometric information entry. The method of claim 1,
The access code is at least one of an alphanumeric character string, a visual access sequence, a safe lock, a pattern, a gesture, a hidden lock, and a dice word, and the access code is further encrypted through biometric information, and the biometric information is stored in a device. The method of claim 8,
A method of decrypting an access code to regenerate a target key by prompting the user to provide biometric information on a subsequent login request from an application or service. The method of claim 9,
A method of prompting a user to provide an access code by entering an access code in addition to biometric information for further subsequent login requests from an application or service. The method of claim 1,
How subsequent login requests are detected by the device using heuristics. Memory; And
A user device authentication system comprising a processor operably coupled to the memory,
The processor is configured to generate a target key based on an identifier code and an access code in response to a user login request from a service provider,
The target key is registered as the service provider of the user, the identifier code is securely stored in the memory, the user is prompted to enter the access code, and the blending algorithm consistently regenerates the target key of the service provider. Authentication system.

Images