US20190087832A1 - Digital credential field data mapping - Google Patents
Digital credential field data mapping Download PDFInfo
- Publication number
- US20190087832A1 US20190087832A1 US15/993,555 US201815993555A US2019087832A1 US 20190087832 A1 US20190087832 A1 US 20190087832A1 US 201815993555 A US201815993555 A US 201815993555A US 2019087832 A1 US2019087832 A1 US 2019087832A1
- Authority
- US
- United States
- Prior art keywords
- credential
- field data
- digital
- user
- data objects
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/018—Certifying business or products
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G06F17/30371—
-
- G06F17/30377—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/31—User authentication
- G06F21/316—User authentication by observing the pattern of computer usage, e.g. typical user behaviour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/31—User authentication
- G06F21/32—User authentication using biometric data, e.g. fingerprints, iris scans or voiceprints
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/186—Templates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06311—Scheduling, planning or task assignment for a person or group
- G06Q10/063112—Skill-based matching of a person or a group to a task
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06398—Performance of employee with respect to a job function
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/067—Enterprise or organisation modelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/105—Human resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
- G06Q50/2057—Career enhancement or continuing education service
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B7/00—Electrically-operated teaching apparatus or devices working with questions and answers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/083—Network architectures or network communication protocols for network security for authentication of entities using passwords
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/0861—Network architectures or network communication protocols for network security for authentication of entities using biometrical features, e.g. fingerprint, retina-scan
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/0876—Network architectures or network communication protocols for network security for authentication of entities based on the identity of the terminal or configuration, e.g. MAC address, hardware or software configuration or device fingerprint
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
- H04L63/107—Network architectures or network communication protocols for network security for controlling access to devices or network resources wherein the security policies are location-dependent, e.g. entities privileges depend on current location or allowing specific operations only from locally connected terminals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/131—Protocols for games, networked simulations or virtual reality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/30—Profiles
- H04L67/306—User profiles
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/535—Tracking the activity of the user
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/38—Services specially adapted for particular environments, situations or purposes for collecting sensor information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06395—Quality analysis or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Definitions
- a digital credential platform server may receive requests from client devices identifying a digital credential receiver. In response to such requests, the digital credential platform server may retrieve data corresponding to a set of digital credential objects received by the digital credential receiver. The digital credential platform server also may retrieve one or more verified evaluation records, for example, from a secure third-party provider server, associated with the digital credential receiver.
- Verified evaluation records may include, for instance, results of personality or interest evaluations administered to the digital credential receiver, results of automated evaluations of the receiver's digital records of interactions, and the like, in order to further assess compatibility of the digital credential receiver with particular fields. Additionally, in some embodiments, the digital credential platform server may retrieve and/or determine a career phase associated with the digital credential receiver, which may be relevant to the mapping analysis.
- the digital credential platform server may analyze the various data sources associated with the digital credential receiver, in order to determine and calculate correlation scores between the digital credential receiver and various field data objects.
- the digital credential platform server may use a combination of analyses and/or comparisons between the receiver data and related data retrieved from a plurality of different field data objects. For example, the digital credential objects (or digital badges) earned by the receiver may be compared to the corresponding capabilities, skills, certifications, etc., associated with different fields in a field data structure. Additionally, the receiver's verified evaluation records may be compared to the corresponding interest or personality data associated with the different fields.
- the current career phase of the digital credential receiver may be compared to corresponding career phase data associated with the different fields.
- the platform server may calculate correlation scores for a plurality of different field data objects, and designate/transmit a subset of the fields that correlate most closely to the digital credential receiver.
- the platform server may store and maintain expiration dates and/or half-lives for various types of data associated with digital credential receivers, such as the receiver's digital credentials, the receiver's verified evaluation data, and the receiver's career phase data. Expiration dates and/or half-lives may be used as a multiplier to weight the correlation calculations of credential receivers to field data objects.
- reverse analyses and calculations may be performed, in which a plurality of different digital credential receivers are selected and scored based on their correlation to a particular field data object representing an occupation or an individual job listing.
- various embodiments described herein may include digital credentials issued to receivers based on the detection of specific interests and/or personality traits within the users, specific DNA traits and/or health-based traits, and/or for various other types or combinations of traits that may be detected for particular digital credential receivers, and these digital credentials may further be used in the correlation analyses.
- Still other aspects described herein may relate to a secure digital certification platform and/or service, to provide digital credential certification, verification, and security, in order to address problems associated with anonymous and/or unverified digital credentials and/or credential receivers.
- FIG. 1 is a block diagram showing illustrating an example of a content distribution network.
- FIG. 2 is a block diagram illustrating a computer server and computing environment within a content distribution network.
- FIG. 3 is a block diagram illustrating an embodiment of one or more data store servers within a content distribution network.
- FIG. 4 is a block diagram illustrating an embodiment of one or more content management servers within a content distribution network.
- FIG. 5 is a block diagram illustrating the physical and logical components of a special-purpose computer device within a content distribution network.
- FIG. 6 is a block diagram illustrating an example computing environment for generating, managing, and tracking digital credential templates and digital credentials, according to one or more embodiments of the disclosure.
- FIG. 7 is an diagram illustrating an example computing environment for executing and monitoring physical simulations within a digital credential system, according to one or more embodiments of the disclosure.
- FIG. 8 is a flow diagram illustrating an example process of executing and monitoring physical simulations for generation of digital credentials, according to one or more embodiments of the disclosure.
- FIG. 9A is an diagram illustrating a computer terminal-based system for sensor-based monitoring and generation of digital credentials, according to one or more embodiments of the disclosure.
- FIG. 9B is an diagram illustrating a physical environment-based system for sensor-based monitoring, and generation of digital credentials, according to one or more embodiments of the disclosure.
- FIG. 10 is a flow diagram illustrating an example process of generating and issuing digital credentials in a sensor-monitored environment, according to one or more embodiments of the disclosure.
- FIG. 11 is an diagram illustrating an example computing environment for analyzing sensor-based activity monitoring within a digital credential system, according to one or more embodiments of the disclosure.
- FIG. 12 is a flow diagram illustrating an example process of generating digital credentials and tracking the corresponding activities in a sensor-monitored environment, according to one or more embodiments of the disclosure.
- FIG. 13 is a flow diagram illustrating an example process of analyzing activities in a sensor-monitored environment to determine digital credential expiration and/or recertification times, according to one or more embodiments of the disclosure.
- FIG. 14 is an diagram illustrating an example computing environment for generating and analyzing digital credentials using received sensor monitoring data, according to one or more embodiments of the disclosure.
- FIG. 15 is a flow diagram illustrating an example process of generating and storing digital credentials with associated data from sensor-monitored environments, according to one or more embodiments of the disclosure.
- FIGS. 16A-16B are flow diagrams illustrating example processes of retrieving sensor data associated with issued digital credentials, and generating additional and/or updated digital credentials based on the retrieved sensor data, according to one or more embodiments of the disclosure.
- FIGS. 17A-17B are diagrams illustrating facial recognition and analysis functionality performed during digital credential generation and analyses processes within sensor-monitored environments, according to one or more embodiments of the disclosure.
- FIG. 18 is a flow diagram illustrating an example process of generating and storing digital credentials with associated user feedback data from sensor-monitored environments, according to one or more embodiments of the disclosure.
- FIG. 19 is a block diagram illustrating an example system for analyzing and mapping digital credential receivers to field data objects, according to one or more embodiments of the disclosure.
- FIG. 20 is a flow diagram illustrating an example process of generating notifications based on determined correlations between digital credential receivers and field data objects, according to one or more embodiments of the disclosure.
- a digital credential platform server may receive requests from client devices identifying a digital credential receiver. In response to such requests, the digital credential platform server may retrieve data corresponding to a set of digital credential objects received by the digital credential receiver. The digital credential platform server also may retrieve one or more verified evaluation records, for example, from a secure third-party provider server, associated with the digital credential receiver.
- Verified evaluation records may include, for instance, results of personality or interest evaluations administered to the digital credential receiver, results of automated evaluations of the receiver's digital records of interactions, and the like, in order to further assess compatibility of the digital credential receiver with particular fields. Additionally, in some embodiments, the digital credential platform server may retrieve and/or determine a career phase associated with the digital credential receiver, which may be relevant to the mapping analysis.
- the digital credential platform server may analyze the various data sources associated with the digital credential receiver, in order to determine and calculate correlation scores between the digital credential receiver and various field data objects.
- the digital credential platform server may use a combination of analyses and/or comparisons between the receiver data and related data retrieved from a plurality of different field data objects. For example, the digital credential objects (or digital badges) earned by the receiver may be compared to the corresponding capabilities, skills, certifications, etc., associated with different fields in a field data structure. Additionally, the receiver's verified evaluation records may be compared to the corresponding interest or personality data associated with the different fields.
- the current career phase of the digital credential receiver may be compared to corresponding career phase data associated with the different fields.
- the platform server may calculate correlation scores for a plurality of different field data objects, and designate/transmit a subset of the fields that correlate most closely to the digital credential receiver.
- the platform server may store and maintain expiration dates and/or half-lives for various types of data associated with digital credential receivers, such as the receiver's digital credentials, the receiver's verified evaluation data, and the receiver's career phase data. Expiration dates and/or half-lives may be used as a multiplier to weight the correlation calculations of credential receivers to field data objects.
- reverse analyses and calculations may be performed, in which a plurality of different digital credential receivers are selected and scored based on their correlation to a particular field data object representing an occupation or an individual job listing.
- various embodiments described herein may include digital credentials issued to receivers based on the detection of specific interests and/or personality traits within the users, specific DNA traits and/or health-based traits, and/or for various other types or combinations of traits that may be detected for particular digital credential receivers, and these digital credentials may further be used in the correlation analyses.
- Still other aspects described herein may relate to a secure digital certification platform and/or service, to provide digital credential certification, verification, and security, in order to address problems associated with anonymous and/or unverified digital credentials and/or credential receivers.
- Content distribution network 100 may include one or more content management servers 102 .
- content management servers 102 may be any desired type of server including, for example, a rack server, a tower server, a miniature server, a blade server, a mini rack server, a mobile server, an ultra-dense server, a super server, or the like, and may include various hardware components, for example, a motherboard, a processing units, memory systems, hard drives, network interfaces, power supplies, etc.
- Content management server 102 may include one or more server farms, clusters, or any other appropriate arrangement and/or combination or computer servers. Content management server 102 may act according to stored instructions located in a memory subsystem of the server 102 , and may run an operating system, including any commercially available server operating system and/or any other operating systems discussed herein.
- the content distribution network 100 may include one or more data store servers 104 , such as database servers and file-based storage systems.
- Data stores 104 may comprise stored data relevant to the functions of the content distribution network 100 . Illustrative examples of data stores 104 that may be maintained in certain embodiments of the content distribution network 100 are described below in reference to FIG. 3 .
- multiple data stores may reside on a single server 104 , either using the same storage components of server 104 or using different physical storage components to assure data security and integrity between data stores.
- each data store may have a separate dedicated data store server 104 .
- Content distribution network 100 also may include one or more user devices 106 and/or supervisor devices 110 .
- User devices 106 and supervisor devices 110 may display content received via the content distribution network 100 , and may support various types of user interactions with the content.
- User devices 106 and supervisor devices 110 may include mobile devices such as smartphones, tablet computers, personal digital assistants, and wearable computing devices. Such mobile devices may run a variety of mobile operating systems, and may be enabled for Internet, e-mail, short message service (SMS), Bluetooth®, mobile radio-frequency identification (M-RFID), and/or other communication protocols.
- Other user devices 106 and supervisor devices 110 may be general purpose personal computers or special-purpose computing devices including, by way of example, personal computers, laptop computers, workstation computers, projection devices, and interactive room display systems. Additionally, user devices 106 and supervisor devices 110 may be any other electronic devices, such as thin-client computers, Internet-enabled gaming systems, business or home appliances, and/or personal messaging devices, capable of communicating over network(s) 120 .
- user devices 106 and supervisor devices 110 may correspond to different types of specialized devices, for example, student devices and teacher devices in an educational network, employee devices and presentation devices in a company network, different gaming devices in a gaming network, etc.
- user devices 106 and supervisor devices 110 may operate in the same physical location 107 , such as a classroom or conference room.
- the devices may contain components that support direct communications with other nearby devices, such as a wireless transceivers and wireless communications interfaces, Ethernet sockets or other Local Area Network (LAN) interfaces, etc.
- LAN Local Area Network
- the user devices 106 and supervisor devices 110 need not be used at the same location 107 , but may be used in remote geographic locations in which each user device 106 and supervisor device 110 may use security features and/or specialized hardware (e.g., hardware-accelerated SSL and HTTPS, WS-Security, firewalls, etc.) to communicate with the content management server 102 and/or other remotely located user devices 106 .
- security features and/or specialized hardware e.g., hardware-accelerated SSL and HTTPS, WS-Security, firewalls, etc.
- different user devices 106 and supervisor devices 110 may be assigned different designated roles, such as presenter devices, teacher devices, administrator devices, or the like, and in such cases the different devices may be provided with additional hardware and/or software components to provide content and support user capabilities not available to the other devices.
- the content distribution network 100 also may include a privacy server 108 that maintains private user information at the privacy server 108 while using applications or services hosted on other servers.
- the privacy server 108 may be used to maintain private data of a user within one jurisdiction even though the user is accessing an application hosted on a server (e.g., the content management server 102 ) located outside the jurisdiction.
- the privacy server 108 may intercept communications between a user device 106 or supervisor device 110 and other devices that include private user information.
- the privacy server 108 may create a token or identifier that does not disclose the private information and may use the token or identifier when communicating with the other servers and systems, instead of using the user's private information.
- the content management server 102 may be in communication with one or more additional servers, such as a content server 112 , a user data server 112 , and/or an administrator server 116 .
- Each of these servers may include some or all of the same physical and logical components as the content management server(s) 102 , and in some cases, the hardware and software components of these servers 112 - 116 may be incorporated into the content management server(s) 102 , rather than being implemented as separate computer servers.
- Content server 112 may include hardware and software components to generate, store, and maintain the content resources for distribution to user devices 106 and other devices in the network 100 .
- content server 112 may include data stores of training materials, presentations, interactive programs and simulations, course models, course outlines, and various training interfaces that correspond to different materials and/or different types of user devices 106 .
- a content server 112 may include media content files such as music, movies, television programming, games, and advertisements.
- User data server 114 may include hardware and software components that store and process data for multiple users relating to each user's activities and usage of the content distribution network 100 .
- the content management server 102 may record and track each user's system usage, including their user device 106 , content resources accessed, and interactions with other user devices 106 .
- This data may be stored and processed by the user data server 114 , to support user tracking and analysis features.
- the user data server 114 may store and analyze each user's training materials viewed, presentations attended, courses completed, interactions, evaluation results, and the like.
- the user data server 114 may also include a repository for user-generated material, such as evaluations and tests completed by users, and documents and assignments prepared by users.
- the user data server 114 may store and process resource access data for multiple users (e.g., content titles accessed, access times, data usage amounts, gaming histories, user devices and device types, etc.).
- Administrator server 116 may include hardware and software components to initiate various administrative functions at the content management server 102 and other components within the content distribution network 100 .
- the administrator server 116 may monitor device status and performance for the various servers, data stores, and/or user devices 106 in the content distribution network 100 .
- the administrator server 116 may add or remove devices from the network 100 , and perform device maintenance such as providing software updates to the devices in the network 100 .
- Various administrative tools on the administrator server 116 may allow authorized users to set user access permissions to various content resources, monitor resource usage by users and devices 106 , and perform analyses and generate reports on specific network users and/or devices (e.g., resource usage tracking reports, training evaluations, etc.).
- the content distribution network 100 may include one or more communication networks 120 . Although only a single network 120 is identified in FIG. 1 , the content distribution network 100 may include any number of different communication networks between any of the computer servers and devices shown in FIG. 1 and/or other devices described herein. Communication networks 120 may enable communication between the various computing devices, servers, and other components of the content distribution network 100 . As discussed below, various implementations of content distribution networks 100 may employ different types of networks 120 , for example, computer networks, telecommunications networks, wireless networks, and/or any combination of these and/or other networks.
- an illustrative distributed computing environment 200 including a computer server 202 , four client computing devices 206 , and other components that may implement certain embodiments and features described herein.
- the server 202 may correspond to the content management server 102 discussed above in FIG. 1
- the client computing devices 206 may correspond to the user devices 106 .
- the computing environment 200 illustrated in FIG. 2 may correspond to any other combination of devices and servers configured to implement a client-server model or other distributed computing architecture.
- Client devices 206 may be configured to receive and execute client applications over one or more networks 220 . Such client applications may be web browser based applications and/or standalone software applications, such as mobile device applications. Server 202 may be communicatively coupled with the client devices 206 via one or more communication networks 220 . Client devices 206 may receive client applications from server 202 or from other application providers (e.g., public or private application stores). Server 202 may be configured to run one or more server software applications or services, for example, web-based or cloud-based services, to support content distribution and interaction with client devices 206 . Users operating client devices 206 may in turn utilize one or more client applications (e.g., virtual client applications) to interact with server 202 to utilize the services provided by these components.
- client applications e.g., virtual client applications
- Various different subsystems and/or components 204 may be implemented on server 202 . Users operating the client devices 206 may initiate one or more client applications to use services provided by these subsystems and components.
- the subsystems and components within the server 202 and client devices 206 may be implemented in hardware, firmware, software, or combinations thereof.
- Various different system configurations are possible in different distributed computing systems 200 and content distribution networks 100 .
- the embodiment shown in FIG. 2 is thus one example of a distributed computing system and is not intended to be limiting.
- exemplary computing environment 200 is shown with four client computing devices 206 , any number of client computing devices may be supported. Other devices, such as specialized sensor devices, etc., may interact with client devices 206 and/or server 202 .
- various security and integration components 208 may be used to transmit, receive, and manage communications between the server 202 and user devices 206 over one or more communication networks 220 .
- the security and integration components 208 may include separate servers, such as web servers and/or authentication servers, and/or specialized networking components, such as firewalls, routers, gateways, load balancers, and the like.
- the security and integration components 208 may correspond to a set of dedicated hardware and/or software operating at the same physical location and under the control of same entities as server 202 .
- components 208 may include one or more dedicated web servers and network hardware in a datacenter or a cloud infrastructure.
- the security and integration components 208 may correspond to separate hardware and software components which may be operated at a separate physical location and/or by a separate entity.
- Security and integration components 208 may implement various security features for data transmission and storage, such as authenticating users and restricting access to unknown or unauthorized users.
- security and integration components 208 may provide, for example, a file-based integration scheme or a service-based integration scheme for transmitting data between the various devices in the content distribution network 100 .
- Security and integration components 208 also may use secure data transmission protocols and/or encryption for data transfers, for example, File Transfer Protocol (FTP), Secure File Transfer Protocol (SFTP), and/or Pretty Good Privacy (PGP) encryption.
- FTP File Transfer Protocol
- SFTP Secure File Transfer Protocol
- PGP Pretty Good Privacy
- one or more web services may be implemented within the security and integration components 208 and/or elsewhere within the content distribution network 100 .
- Such web services including cross-domain and/or cross-platform web services, may be developed for enterprise use in accordance with various web service standards, such as RESTful web services (i.e., services based on the Representation State Transfer (REST) architectural style and constraints), and/or web services designed in accordance with the Web Service Interoperability (WS-I) guidelines.
- Some web services may use the Secure Sockets Layer (SSL) or Transport Layer Security (TLS) protocol to provide secure connections between the server 202 and user devices 206 .
- SSL or TLS may use HTTP or HTTPS to provide authentication and confidentiality.
- web services may be implemented using REST over HTTPS with the OAuth open standard for authentication, or using the WS-Security standard which provides for secure SOAP messages using XML, encryption.
- the security and integration components 208 may include specialized hardware for providing secure web services.
- security and integration components 208 may include secure network appliances having built-in features such as hardware-accelerated SSL and HTTPS, WS-Security, and firewalls.
- Such specialized hardware may be installed and configured in front of any web servers, so that any external devices may communicate directly with the specialized hardware.
- Communication network(s) 220 may be any type of network familiar to those skilled in the art that can support data communications using any of a variety of commercially-available protocols, including without limitation, TCP/IP (transmission control protocol/Internet protocol), SNA (systems network architecture), IPX (Internet packet exchange), Secure Sockets Layer (SSL) or Transport Layer Security (TLS) protocols, Hyper Text Transfer Protocol (HTTP) and Secure Hyper Text Transfer Protocol (HTTPS), Bluetooth®, Near Field Communication (NFC), and the like.
- network(s) 220 may be local area networks (LAN), such as one based on Ethernet, Token-Ring and/or the like.
- Network(s) 220 also may be wide-area networks, such as the Internet.
- Networks 220 may include telecommunication networks such as a public switched telephone networks (PSTNs), or virtual networks such as an intranet or an extranet.
- PSTNs public switched telephone networks
- Infrared and wireless networks e.g., using the Institute of Electrical and Electronics (IEFF) 802.11 protocol suite or other wireless protocols
- IEFF Institute of Electrical and Electronics 802.11 protocol suite or other wireless protocols
- Computing environment 200 also may include one or more data stores 210 and/or back-end servers 212 .
- the data stores 210 may correspond to data store server(s) 104 discussed above in FIG. 1
- back-end servers 212 may correspond to the various back-end servers 112 - 116 .
- Data stores 210 and servers 212 may reside in the same datacenter or may operate at a remote location from server 202 .
- one or more data stores 210 may reside on a non-transitory storage medium within the server 202 .
- Other data stores 210 and back-end servers 212 may be remote from server 202 and configured to communicate with server 202 via one or more networks 220 .
- data stores 210 and back-end servers 212 may reside in a storage-area network (SAN), or may use storage-as-a-service (STaaS) architectural model.
- SAN storage-area network
- STaaS storage-as-a-service
- data stores 301 - 309 may reside in storage on a single computer server 104 (or a single server farm or cluster) under the control of a single entity, or may reside on separate servers operated by different entities and/or at remote locations.
- data stores 301 - 309 may be accessed by the content management server 102 and/or other devices and servers within the network 100 (e.g., user devices 106 , supervisor devices 110 , administrator servers 116 , etc.). Access to one or more of the data stores 301 - 309 may be limited or denied based on the processes, user credentials, and/or devices attempting to interact with the data store.
- data stores 301 - 309 are illustrative and non-limiting.
- Data stores server architecture, design, and the execution of specific data stores 301 - 309 may depend on the context, size, and functional requirements of a content distribution network 100 .
- data store server(s) 104 may be implemented in data store server(s) 104 to store trainee and/or student data, trainer and/or professor data, training module data and content descriptions, training results, evaluation data, and the like.
- data stores server(s) 104 may be implemented in data stores server(s) 104 to store listings of available content titles and descriptions, content title usage statistics, subscriber profiles, account data, payment data, network usage statistics, etc.
- a user profile data store 301 may include information relating to the end users within the content distribution network 100 . This information may include user characteristics such as the user names, access credentials (e.g., login and passwords), user preferences, and information relating to any previous user interactions within the content distribution network 100 (e.g., requested content, posted content, content modules completed, training scores or evaluations, other associated users, etc.).
- user characteristics such as the user names, access credentials (e.g., login and passwords), user preferences, and information relating to any previous user interactions within the content distribution network 100 (e.g., requested content, posted content, content modules completed, training scores or evaluations, other associated users, etc.).
- An accounts data store 302 may generate and store account data for different users in various roles within the content distribution network 100 .
- accounts may be created in an accounts data store 302 for individual end users, supervisors, administrator users, and entities such as companies or educational institutions.
- Account data may include account types, current account status, account characteristics, and any parameters, limits, restrictions associated with the accounts.
- a content library data store 303 may include information describing the individual content items (or content resources) available via the content distribution network 100 .
- the library data store 303 may include metadata, properties, and other characteristics associated with the content resources stored in the content server 112 .
- Such data may identify one or more aspects or content attributes of the associated content resources, for example, subject matter, access level, or skill level of the content resources, license attributes of the content resources (e.g., any limitations and/or restrictions on the licensable use and/or distribution of the content resource), price attributes of the content resources (e.g., a price and/or price structure for determining a payment amount for use or distribution of the content resource), rating attributes for the content resources (e.g., data indicating the evaluation or effectiveness of the content resource), and the like.
- the library data store 303 may be configured to allow updating of content metadata or properties, and to allow the addition and/or removal of information relating to the content resources.
- content relationships may be implemented as graph structures, which may be stored in the library data store 303 or in an additional store for use by selection algorithms along with the other metadata.
- a pricing data store 304 may include pricing information and/or pricing structures for determining payment amounts for providing access to the content distribution network 100 and/or the individual content resources within the network 100 .
- pricing may be determined based on a user's access to the content distribution network 100 , for example, a time-based subscription fee, or pricing based on network usage and.
- pricing may be tied to specific content resources. Certain content resources may have associated pricing information, whereas other pricing determinations may be based on the resources accessed, the profiles and/or accounts of the user, and the desired level of access (e.g., duration of access, network speed, etc.).
- the pricing data store 304 may include information relating to compilation pricing for groups of content resources, such as group prices and/or price structures for groupings of resources.
- a license data store 305 may include information relating to licenses and/or licensing of the content resources within the content distribution network 100 .
- the license data store 305 may identify licenses and licensing terms for individual content resources and/or compilations of content resources in the content server 112 , the rights holders for the content resources, and/or common or large-scale right holder information such as contact information for rights holders of content not included in the content server 112 .
- a content access data store 306 may include access rights and security information for the content distribution network 100 and specific content resources.
- the content access data store 306 may include login information (e.g., user identifiers, logins, passwords, etc.) that can be verified during user login attempts to the network 100 .
- the content access data store 306 also may be used to store assigned user roles and/or user levels of access.
- a user's access level may correspond to the sets of content resources and/or the client or server applications that the user is permitted to access. Certain users may be permitted or denied access to certain applications and resources based on their subscription level, training program, course/grade level, etc.
- Certain users may have supervisory access over one or more end users, allowing the supervisor to access all or portions of the end user's content, activities, evaluations, etc. Additionally, certain users may have administrative access over some users and/or some applications in the content management network 100 , allowing such users to add and remove user accounts, modify user access permissions, perform maintenance updates on software and servers, etc.
- a source data store 307 may include information relating to the source of the content resources available via the content distribution network. For example, a source data store 307 may identify the authors and originating devices of content resources, previous pieces of data and/or groups of data originating from the same authors or originating devices, and the like.
- An evaluation data store 308 may include information used to direct the evaluation of users and content resources in the content management network 100 .
- the evaluation data store 308 may contain, for example, the analysis criteria and the analysis guidelines for evaluating users (e.g., trainees/students, gaming users, media content consumers, etc.) and/or for evaluating the content resources in the network 100 .
- the evaluation data store 308 also may include information relating to evaluation processing tasks, for example, the identification of users and user devices 106 that have received certain content resources or accessed certain applications, the status of evaluations or evaluation histories for content resources, users, or applications, and the like.
- Evaluation criteria may be stored in the evaluation data store 308 including data and/or instructions in the form of one or several electronic rubrics or scoring guides for use in the evaluation of the content, users, or applications.
- the evaluation data store 308 also may include past evaluations and/or evaluation analyses for users, content, and applications, including relative rankings, characterizations, explanations, and the like.
- data store server(s) 104 may include one or more external data aggregators 309 .
- External data aggregators 309 may include third-party data sources accessible to the content management network 100 , but not maintained by the content management network 100 .
- External data aggregators 309 may include any electronic information source relating to the users, content resources, or applications of the content distribution network 100 .
- external data aggregators 309 may be third-party data stores containing demographic data, education related data, consumer sales data, health related data, and the like.
- Illustrative external data aggregators 309 may include, for example, social networking web servers, public records data stores, learning management systems, educational institution servers, business servers, consumer sales data stores, medical record data stores, etc. Data retrieved from various external data aggregators 309 may be used to verify and update user account information, suggest user content, and perform user and content evaluations.
- content management server(s) 102 may include various server hardware and software components that manage the content resources within the content distribution network 100 and provide interactive and adaptive content to users on various user devices 106 .
- content management server(s) 102 may provide instructions to and receive information from the other devices within the content distribution network 100 , in order to manage and transmit content resources, user data, and server or client applications executing within the network 100 .
- a content management server 102 may include a content customization system 402 .
- the content customization system 402 may be implemented using dedicated hardware within the content distribution network 100 (e.g., a content customization server 402 ), or using designated hardware and software resources within a shared content management server 102 .
- the content customization system 402 may adjust the selection and adaptive capabilities of content resources to match the needs and desires of the users receiving the content.
- the content customization system 402 may query various data stores and servers 104 to retrieve user information, such as user preferences and characteristics (e.g., from a user profile data store 301 ), user access restrictions to content recourses (e.g., from a content access data store 306 ), previous user results and content evaluations (e.g., from an evaluation data store 308 ), and the like. Based on the retrieved information from data stores 104 and other data sources, the content customization system 402 may modify content resources for individual users.
- user preferences and characteristics e.g., from a user profile data store 301
- user access restrictions to content recourses e.g., from a content access data store 306
- previous user results and content evaluations e.g., from an evaluation data store 308
- the content customization system 402 may modify content resources for individual users.
- a content management server 102 also may include a user management system 404 .
- the user management system 404 may be implemented using dedicated hardware within the content distribution network 100 (e.g., a user management server 404 ), or using designated hardware and software resources within a shared content management server 102 .
- the user management system 404 may monitor the progress of users through various types of content resources and groups, such as media compilations, courses or curriculums in training or educational contexts, interactive gaming environments, and the like.
- the user management system 404 may query one or more databases and/or data store servers 104 to retrieve user data such as associated content compilations or programs, content completion status, user goals, results, and the like.
- a content management server 102 also may include an evaluation system 406 .
- the evaluation system 406 may be implemented using dedicated hardware within the content distribution network 100 (e.g., an evaluation server 406 ), or using designated hardware and software resources within a shared content management server 102 .
- the evaluation system 406 may be configured to receive and analyze information from user devices 106 . For example, various ratings of content resources submitted by users may be compiled and analyzed, and then stored in a data store (e.g., a content library data store 303 and/or evaluation data store 308 ) associated with the content.
- the evaluation server 406 may analyze the information to determine the effectiveness or appropriateness of content resources with, for example, a subject matter, an age group, a skill level, or the like.
- the evaluation system 406 may provide updates to the content customization system 402 or the user management system 404 , with the attributes of one or more content resources or groups of resources within the network 100 .
- the evaluation system 406 also may receive and analyze user evaluation data from user devices 106 , supervisor devices 110 , and administrator servers 116 , etc. For instance, evaluation system 406 may receive, aggregate, and analyze user evaluation data for different types of users (e.g., end users, supervisors, administrators, etc.) in different contexts (e.g., media consumer ratings, trainee or student comprehension levels, teacher effectiveness levels, gamer skill levels, etc.).
- a content management server 102 also may include a content delivery system 408 .
- the content delivery system 408 may be implemented using dedicated hardware within the content distribution network 100 (e.g., a content delivery server 408 ), or using designated hardware and software resources within a shared content management server 102 .
- the content delivery system 408 may receive content resources from the content customization system 402 and/or from the user management system 404 , and provide the resources to user devices 106 .
- the content delivery system 408 may determine the appropriate presentation format for the content resources based on the user characteristics and preferences, and/or the device capabilities of user devices 106 . If needed, the content delivery system 408 may convert the content resources to the appropriate presentation format and/or compress the content before transmission. In some embodiments, the content delivery system 408 may also determine the appropriate transmission media and communication protocols for transmission of the content resources.
- the content delivery system 408 may include specialized security and integration hardware 410 , along with corresponding software components to implement the appropriate security features content transmission and storage, to provide the supported network and client access models, and to support the performance and scalability requirements of the network 100 .
- the security and integration layer 410 may include some or all of the security and integration components 208 discussed above in FIG. 2 , and may control the transmission of content resources and other data, as well as the receipt of requests and content interactions, to and from the user devices 106 , supervisor devices 110 , administrative servers 116 , and other devices in the network 100 .
- the system 500 may correspond to any of the computing devices or servers of the content distribution network 100 described above, or any other computing devices described herein.
- computer system 500 includes processing units 504 that communicate with a number of peripheral subsystems via a bus subsystem 502 .
- peripheral subsystems include, for example, a storage subsystem 510 , an I/O subsystem 526 , and a communications subsystem 532 .
- Bus subsystem 502 provides a mechanism for letting the various components and subsystems of computer system 500 communicate with each other as intended. Although bus subsystem 502 is shown schematically as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 502 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. Such architectures may include, for example, an Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus, which can be implemented as a Mezzanine bus manufactured to the IEEE P1386.1 standard.
- ISA Industry Standard Architecture
- MCA Micro Channel Architecture
- EISA Enhanced ISA
- VESA Video Electronics Standards Association
- PCI Peripheral Component Interconnect
- Processing unit 504 which may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of computer system 500 .
- processors including single core and/or multicore processors, may be included in processing unit 504 .
- processing unit 504 may be implemented as one or more independent processing units 506 and/or 508 with single or multicore processors and processor caches included in each processing unit.
- processing unit 504 may also be implemented as a quad-core processing unit or larger multicore designs (e.g., hexa-core processors, octo-core processors, ten-core processors, or greater.
- Processing unit 504 may execute a variety of software processes embodied in program code, and may maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can be resident in processor(s) 504 and/or in storage subsystem 510 .
- computer system 500 may include one or more specialized processors, such as digital signal processors (DSPs), outboard processors, graphics processors, application-specific processors, and/or the like.
- DSPs digital signal processors
- outboard processors such as graphics processors, application-specific processors, and/or the like.
- I/O subsystem 526 may include device controllers 528 for one or more user interface input devices and/or user interface output devices 530 .
- User interface input and output devices 530 may be integral with the computer system 500 (e.g., integrated audio/video systems, and/or touchscreen displays), or may be separate peripheral devices which are attachable/detachable from the computer system 500 .
- Input devices 530 may include a keyboard, pointing devices such as a mouse or trackball, a touchpad or touch screen incorporated into a display, a scroll wheel, a click wheel, a dial, a button, a switch, a keypad, audio input devices with voice command recognition systems, microphones, and other types of input devices.
- Input devices 530 may also include three dimensional (3D) mice, joysticks or pointing sticks, gamepads and graphic tablets, and audio/visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode reader 3D scanners, 3D printers, laser rangefinders, and eye gaze tracking devices.
- Additional input devices 530 may include, for example, motion sensing and/or gesture recognition devices that enable users to control and interact with an input device through a natural user interface using gestures and spoken commands, eye gesture recognition devices that detect eye activity from users and transform the eye gestures as input into an input device, voice recognition sensing devices that enable users to interact with voice recognition systems through voice commands, medical imaging input devices, MIDI keyboards, digital musical instruments, and the like.
- Output devices 530 may include one or more display subsystems, indicator lights, or non-visual displays such as audio output devices, etc.
- Display subsystems may include, for example, cathode ray tube (CRT) displays, flat-panel devices, such as those using a liquid crystal display (LCD) or plasma display, light-emitting diode (LED) displays, projection devices, touch screens, and the like.
- CTR cathode ray tube
- LCD liquid crystal display
- LED light-emitting diode
- output devices 530 may include, without limitation, a variety of display devices that visually convey text, graphics and audio/video information such as monitors, printers, speakers, headphones, automotive navigation systems, plotters, voice output devices, and modems.
- Computer system 500 may comprise one or more storage subsystems 510 , comprising hardware and software components used for storing data and program instructions, such as system memory 518 and computer-readable storage media 516 .
- the system memory 518 and/or computer-readable storage media 516 may store program instructions that are loadable and executable on processing units 504 , as well as data generated during the execution of these programs.
- system memory 318 may be stored in volatile memory (such as random access memory (RAM) 512 ) and/or in non-volatile storage drives 514 (such as read-only memory (ROM), flash memory, etc.)
- RAM random access memory
- ROM read-only memory
- system memory 518 may include multiple different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
- SRAM static random access memory
- DRAM dynamic random access memory
- a basic input/output system (BIOS), containing the basic routines that help to transfer information between elements within computer system 500 , such as during start-up, may typically be stored in the non-volatile storage drives 514 .
- system memory 518 may include application programs 520 , such as client applications, Web browsers, mid-tier applications, server applications, etc., program data 522 , and an operating system 524 .
- Storage subsystem 510 also may provide one or more tangible computer-readable storage media 516 for storing the basic programming and data constructs that provide the functionality of some embodiments.
- Software programs, code modules, instructions that when executed by a processor provide the functionality described herein may be stored in storage subsystem 510 . These software modules or instructions may be executed by processing units 504 .
- Storage subsystem 510 may also provide a repository for storing data used in accordance with the present invention
- Storage subsystem 300 may also include a computer-readable storage media reader that can further be connected to computer-readable storage media 516 .
- computer-readable storage media 516 may comprehensively represent remote, local, fixed, and/or removable storage devices plus storage media for temporarily and/or more permanently containing, storing, transmitting, and retrieving computer-readable information.
- Computer-readable storage media 516 containing program code, or portions of program code may include any appropriate media known or used in the art, including storage media and communication media, such as but not limited to, volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage and/or transmission of information.
- This can include tangible computer-readable storage media such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other tangible computer readable media.
- This can also include nontangible computer-readable media, such as data signals, data transmissions, or any other medium which can be used to transmit the desired information and which can be accessed by computer system 500 .
- computer-readable storage media 516 may include a hard disk drive that reads from or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive that reads from or writes to a removable, nonvolatile magnetic disk, and an optical disk drive that reads from or writes to a removable, nonvolatile optical disk such as a CD ROM, DVD, and Blu-Ray® disk, or other optical media.
- Computer-readable storage media 516 may include, but is not limited to, Zip® drives, flash memory cards, universal serial bus (USB) flash drives, secure digital (SD) cards, DVD disks, digital video tape, and the like.
- Computer-readable storage media 516 may also include, solid-state drives (SSD) based on non-volatile memory such as flash-memory based SSDs, enterprise flash drives, solid state ROM, and the like, SSDs based on volatile memory such as solid state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory based SSDs.
- SSD solid-state drives
- volatile memory such as solid state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory based SSDs.
- MRAM magnetoresistive RAM
- hybrid SSDs that use a combination of DRAM and flash memory based SSDs.
- the disk drives and their associated computer-readable media may provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for computer system 500 .
- Communications subsystem 532 may provide a communication interface from computer system 500 and external computing devices via one or more communication networks, including local area networks (LANs), wide area networks (WANs) (e.g., the Internet), and various wireless telecommunications networks.
- the communications subsystem 532 may include, for example, one or more network interface controllers (NICs) 534 , such as Ethernet cards, Asynchronous Transfer Mode NICs, Token Ring NICs, and the like, as well as one or more wireless communications interfaces 536 , such as wireless network interface controllers (WNICs), wireless network adapters, and the like.
- NICs network interface controllers
- WNICs wireless network interface controllers
- the communications subsystem 532 may include one or more modems (telephone, satellite, cable, ISDN), synchronous or asynchronous digital subscriber line (DSL) units, FireWire® interfaces, USB® interfaces, and the like.
- Communications subsystem 536 also may include radio frequency (RF) transceiver components for accessing wireless voice and/or data networks (e.g., using cellular telephone technology, advanced data network technology, such as 3G, 4G or EDGE (enhanced data rates for global evolution), WiFi (IEEE 802.11 family standards, or other mobile communication technologies, or any combination thereof), global positioning system (GPS) receiver components, and/or other components.
- RF radio frequency
- the various physical components of the communications subsystem 532 may be detachable components coupled to the computer system 500 via a computer network, a FireWire® bus, or the like, and/or may be physically integrated onto a motherboard of the computer system 500 .
- Communications subsystem 532 also may be implemented in whole or in part by software.
- communications subsystem 532 may also receive input communication in the form of structured and/or unstructured data feeds, event streams, event updates, and the like, on behalf of one or more users who may use or access computer system 500 .
- communications subsystem 532 may be configured to receive data feeds in real-time from users of social networks and/or other communication services, web feeds such as Rich Site Summary (RSS) feeds, and/or real-time updates from one or more third party information sources (e.g., data aggregators 309 ).
- RSS Rich Site Summary
- communications subsystem 532 may be configured to receive data in the form of continuous data streams, which may include event streams of real-time events and/or event updates (e.g., sensor data applications, financial tickers, network performance measuring tools, clickstream analysis tools, automobile traffic monitoring, etc.). Communications subsystem 532 may output such structured and/or unstructured data feeds, event streams, event updates, and the like to one or more data stores 104 that may be in communication with one or more streaming data source computers coupled to computer system 500 .
- event streams of real-time events and/or event updates e.g., sensor data applications, financial tickers, network performance measuring tools, clickstream analysis tools, automobile traffic monitoring, etc.
- Communications subsystem 532 may output such structured and/or unstructured data feeds, event streams, event updates, and the like to one or more data stores 104 that may be in communication with one or more streaming data source computers coupled to computer system 500 .
- a digital credential management system 600 may include a digital credential platform server 610 configured to communicate with various other digital credential systems 620 - 680 .
- the digital credential platform server 610 may receive and store digital credential templates from various digital credential template owner systems 620 .
- Systems 620 may correspond to the computer servers and/or devices of educational institutions or professional training organizations, which may have the primary responsibility for defining a digital credential template and controlling the content and requirements for users to receive a digital credential from the organization.
- the digital credential management system 600 may include one or more digital credential issuer systems 630 . As discussed below, each issuer system 630 may communicate with the platform server to request and receive access to issue digital credentials based on specific digital credential templates. The platform server 610 may process template access requests from the credential issuer systems 630 , permitting or denying a specific system 630 to generate (or issue) a digital credential based on a specific digital credential template.
- a digital credential template may refer to an electronic document or data structure storing a general (e.g., non-user specific) template or description of a specific type of digital credential that may be issued to an individual.
- Digital credential templates may include, for example, a description of the skills, proficiencies, and/or achievements that the digital credential represents. This description may take the form of diploma data, certification data, and/or license data, including the parent organization (i.e., the digital credential template owner) responsible for creating and defining the digital credential template.
- Examples of digital credential templates may include templates for various technology certifications, licensure exams, professional tests, training course completion certificates, and the like.
- a digital credential may refer to an instance of an electronic document or data structure, generated for a specific individual (i.e., the credential receiver), and based on a digital credential template.
- a digital credential document or data structure may be based on a corresponding digital credential template, but may be customized and populated with user-specific information such as individual identification data (e.g., name, email address, and other user identifiers), credential issuance data (e.g., issue date, geographic location of issuance, authorized issuer of the credential, etc.), and links or embedded data that contain the specific user's supporting documentation or evidence relating to the credential.
- individual identification data e.g., name, email address, and other user identifiers
- credential issuance data e.g., issue date, geographic location of issuance, authorized issuer of the credential, etc.
- links or embedded data that contain the specific user's supporting documentation or evidence relating to the credential.
- the system 600 also may include a digital credential receiver system 640 and a digital credential endorser system 650 .
- the digital credential receiver system 640 may be a computing device associated with a credential receiver (or credential earner), for example, an individual user of an electronic learning system, professional training system, online certification course, etc.
- credential receivers may access the platform server 610 via systems 640 to accept or reject newly issued digital credentials, review and update their own set of previously earned digital credentials, as well as to publish (or share) their digital credentials via communication applications or publishing platforms such as social media systems.
- Digital credential endorser system 650 may be a computing system associated with an endorsing entity, such as an educational institution, business, or technical organization that has chosen to review and endorse a specific digital credential template.
- the platform server 610 may receive and track the endorsements received from systems 650 , and may associate the endorsements with the user-specific digital credentials issued based on the endorsed templates.
- the digital credential management system 600 in this example includes a number of external client devices 660 and external digital credential publishers 670 .
- External client devices 660 may correspond to computing systems of third-party users that may interact with the platform server 610 to initiate various functionality or retrieve data relating to templates and/digital credentials managed by the platform 610 .
- a client device 660 may query the platform server 610 for data metrics and/or analyses relating to a subset of digital credentials stored in the digital credential data store 615 .
- the third-party systems 660 also may provide data to the platform server 610 that may initiate updates to the templates and/digital credentials stored in the data store 615 .
- External digital credential publishers 670 may correspond to third-party systems configured to receive digital credential data from the platform 610 and publish (or present) the digital credential data to users.
- Examples of publishers 670 may include social media website and systems, digital badge wallets, and/or other specialized servers or applications configured to store and present views of digital badges to users.
- the generation and management of digital credentials, as well as the tracking and reporting of digital credential data may be performed within CDNs 100 , such as eLearning, professional training, and certification systems 100 .
- a content management server 102 or other CDN server e.g., 104 , 112 , 114 , or 116
- the content management server 102 or other servers of an eLearning CDN 100 may issue digital credentials to users, based on its own digital certificate templates and/or templates received from other systems or CDNs.
- an eLearning CDN 100 may be configured to include a digital credential platform server 610 to store and manage templates and digital credentials between separate systems within the CDN 100 .
- the content management server(s) 102 of a CDN 100 may incorporate one or more digital certificate template owner system(s) 620 , digital certificate issuer system(s) 630 , and/or digital certificate platform server(s) 610 .
- the various components and functionalities described herein for the platform server 610 , owner system 620 , and/or issuer system 630 all may be implemented within one or more content management servers 102 (and/or other servers) of an eLearning or professional training CDN 100 .
- a digital credential platform server 610 may be implemented using one or more computer servers, and other specialized hardware and software components, separately from any other CDN components such as content servers 112 , content management servers 102 , data store servers 104 , and the like.
- the digital credential platform server 610 may be configured to communicate directly with related systems 620 - 670 , or indirectly through content management servers 102 and/or other components and communications networks of the CDN 100 .
- each of the components and sub-components discussed in the example digital credential management system 600 may correspond to a single computer server or a complex computing system including a combination of computing devices, storage devices, network components, etc. Each of these components and their respective subcomponents may be implemented in hardware, software, or a combination thereof. Certain systems 620 - 670 may communicate directly with the platform server 610 , while other systems 620 - 670 may communicate with the platform server 610 indirectly via one or more intermediary network components (e.g., routers, gateways, firewalls, etc.) or other devices (e.g., content management servers 102 , content servers 112 , etc.).
- intermediary network components e.g., routers, gateways, firewalls, etc.
- other devices e.g., content management servers 102 , content servers 112 , etc.
- any of the network hardware components and network architecture designs may be implemented in various embodiments to support communication between the systems, servers, and devices in the digital credential management system 600 .
- different systems 620 - 670 may use different networks and networks types to communicate with the platform server 610 , including one or more telecommunications networks, cable networks, satellite networks, cellular networks and other wireless networks, and computer-based IP networks, and the like.
- certain components within the digital credential management system 600 may include special purpose hardware devices and/or special purpose software, such as those included in I/O subsystem 611 and memory 614 of the platform server 610 , as well as those within the memory of the other systems 620 - 670 , and the digital credential data store 615 maintained by the platform server 610 , discussed below.
- the various interactions between the platform server 610 and other systems 620 - 670 may be described below in terms of a client-server model, it should be understood that other computing environments and various combinations of servers and devices may be used to perform the functionality described herein in other embodiments.
- the requests/responses to determine the authorized issuers 630 for specific digital credential templates, the generation of digital credentials, and the retrieval and presentation of digital credential tracking and reporting data may be performed by a centralized web-based platform server 610 in collaboration with various client applications at the other systems 620 - 670 (e.g., web browser applications or standalone client software), in other cases these techniques may be performed entirely by a specialized digital credential platform server 610 , or entirely by one or more digital credential tools (e.g., software services) executing on any one of the systems 620 - 670 .
- digital credential tools e.g., software services
- a client-server model may be used as shown in system 600 , but different functional components and processing tasks may be allocated to the client-side or the sever-side in different embodiments.
- the digital credential data store 615 may be implemented as separate servers or storage systems in some cases, and may use independent hardware and software service components. However, in other implementations, some or all of the digital credential data store 615 may be incorporated into the platform server 610 (as shown in this example) and/or may be incorporated into various other systems 620 - 670 .
- each of the systems 620 - 670 that collaborate and communicate with the platform server 610 may be implemented as client computing systems, such desktop or laptop computers, smartphones, tablet computers, and other various types of computing devices, each of which may include some or all of the hardware, software, and networking components discussed above.
- client systems 620 - 670 may be implemented using any computing device with sufficient processing components, memory and software components, and I/O system components for interacting with users and supporting the desired set of communications with the platform server 610 , as described herein.
- client systems 620 - 670 may include the necessary hardware and software components to establish the network interfaces, security and authentication capabilities, and capabilities for transmitting/receiving digital credential templates and digital credentials, digital credential data requests/responses to the platform server 610 , etc.
- Each client system 620 - 670 may include an I/O subsystem, network interface controller, a processing unit, and memory configured to operate client software applications.
- the digital credential platform server 610 may be configured to receive and execute various programmatic and graphical interfaces for generating, managing, and tracking issued digital credentials, in collaboration with the various client systems 620 - 670 .
- each client system 620 - 670 may include an I/O subsystem 611 having hardware and software components to support a specific set of output capabilities (e.g., LCD display screen characteristics, screen size, color display, video driver, speakers, audio driver, graphics processor and drivers, etc.), and a specific set of input capabilities (e.g., keyboard, mouse, touchscreen, voice control, cameras, facial recognition, gesture recognition, etc.).
- output capabilities e.g., LCD display screen characteristics, screen size, color display, video driver, speakers, audio driver, graphics processor and drivers, etc.
- input capabilities e.g., keyboard, mouse, touchscreen, voice control, cameras, facial recognition, gesture recognition, etc.
- platform server 610 functionality may be compatible or incompatible with certain client systems 620 - 670 .
- certain types of digital credential generation and search functionality may require specific types of processors, graphics components, network components, or I/O components in order to be optimally designed and constructed using a client system 620 - 670 .
- the digital credential platform server 610 may generate and provide software interfaces (e.g., via a web-based application, or using other programmatic or graphical interface techniques) used by the various client systems 620 - 670 to perform the various digital credential management functionality described herein.
- the platform server 610 may access the underlying digital credential data store 615 perform the various functionality described herein.
- platform server 610 may include components such as network interface controllers 612 , processing units 613 , and memory 614 configured to store server software, handle authentication and security, and to store, analyze, and manage the digital credentials, templates, and credential tracking data stored within the digital credential data store 615 .
- the digital credential data store 615 may be implemented as separate dedicated data stores (e.g., databases, file-based storage, etc.) used for storing digital credential template objects, issued digital credentials, credential tracking data, and authorized user/role data.
- the platform server 610 and data store 615 may be implemented as separate software (and/or storage) components within a single computer server 610 in some examples, while in other examples may be implemented as separate computer servers/systems having separate dedicated processing units, storage devices, and/or network components.
- physical testing environments including “simulation laboratories” may use implemented to allow users to perform physical tasks (including mental and/or computer-based tasks) in a monitored environment.
- Such physical testing environments may use virtual reality and/or augmented reality in various cases.
- the simulation lab and/or the user may be monitored by various sensors during testing or certification processes, and the results may be analyzed to determine (at least in part) whether or not the user should be awarded a particular digital credential or certification.
- simulation labs may be implemented as testing environments for manual tasks, computer-based tasks, scenario training, etc., and various monitoring of the simulation lab environment during test may provide data metrics relating to successful completion of tasks, efficiency of task completion, user response times, user decision making behaviors, user biometrics and risk factors, etc. Further, as discussed below, certain simulation labs may provide the ability to change testing scenarios as well as environmental conditions (lighting noise, temperature, etc.) during testing.
- a basic testing environment 700 is shown to illustrate certain features and concepts that may be included in various embodiments.
- different devices and components may be included in the simulation environments 700 .
- simulation environments 700 for standardized testing and completion of computer-based tasks may be setup to simulate an office environment, for instance, with a computer, keyboard, monitor, desk and chair, etc.
- Other testing environments 700 designed for other badges and/or skills certifications may be configured differently.
- testing environments 700 may be configured as a driving simulator (e.g., having front and side display screens, an installed automobile seat with steering wheel, pedals, vehicle controls and gauges, simulated mirror displays, etc.), or a flight simulator (e.g., having front and side display screens, up and down fields of vision, a pilot seat with a center stick and/or other airplane controls and gauges, etc.).
- driving simulator e.g., having front and side display screens, an installed automobile seat with steering wheel, pedals, vehicle controls and gauges, simulated mirror displays, etc.
- a flight simulator e.g., having front and side display screens, up and down fields of vision, a pilot seat with a center stick and/or other airplane controls and gauges, etc.
- Other testing environments 700 might not require or have any display screens, for example testing environments 700 for CPR certification may include one or more CPR manikins and other accessories to test CPR scenarios.
- Additional testing environments 700 may be implemented for law enforcement use of force or defensive tactics scenarios (with or with display screens, with
- Still other testing environments 700 may be implemented for skills testing and verification on machine assembly tasks, and/or on machine use tasks.
- the machines in testing environments 700 in such scenarios may range from simple to complex, to allow users to any testable task on any machine, from bicycle assembly, to automobile maintenance, to semiconductor design, to electrical work, to laser fabrication, to welding.
- Other testing environments 700 may be implemented for skills testing and verification in performance of medical or dental procedures, and the like, and thus may resemble a hospital operating room or dentist office with a full complement of medical tools and devices necessary to perform the tasks to be verified.
- Still other testing environments 700 may be configured to test/verify skills with respect to sports or other physical activities, and thus the testing environments 700 may comprise a dance studio, gymnastics apparatus, golf driving range, or other sports equipment.
- testing environments 700 may require different sets of testing equipment, as well as different monitoring and environmental control features.
- test subjects may include mechanical devices (e.g., machines configured to assemble parts), artificial intelligences and/or other software programs configured to perform certain tasks, etc.
- the environment may have cameras 705 and sensors configured to monitor the performance and behavior of the user during the testing.
- a number of cameras 705 may be installed throughout the testing environment 700 to capture image/video data of the user from different angles during the testing/skills verification process.
- additional sensors may be deployed within the testing environment 700 , including microphones, light sensors, heat sensors, vibration sensors, and any other sensor type, depending on the type of testing/evaluation being performed. For instance, for testing of computer-based tasks, additional sensors such as mouse movement trackers, keystroke loggers, and user eye-tracking software may be used.
- movement sensors may be placed on the user and/or on any objects with which the user may interact during the testing scenario.
- certain embodiments may include biometric sensors and devices 710 configured to detect and track the user's biometric data during the testing process. Such biometric sensors and devices may measure the user's temperature, heartrate, blood pressure, respiration, skin conductivity, body movements, brainwave activities, etc.
- the physical testing environment 700 also may include various environmental controls that allow a test administrator to control the physical environmental conditions during a test or skills evaluation.
- Such environmental controls may include lights 715 that allow the test administrator to control the light levels, angles, and/or colors during a test.
- lighting control within the environment 700 may allow the test administrator to evaluate the user's ability to perform a driving maneuver or roadside maintenance task at night, etc.
- Additional environmental controls may include may include temperature controls, weather simulation (e.g., wind, rain, snow, sunshine, fog, etc.), speakers to provide background noise or distraction, olfactory control that provides scents/odors to simulate the smells that be present during a comparable real-life scenario, vibration control to simulate the activity, and so on.
- FIG. 8 a flow diagram is shown illustrating an example process of executing tests or simulations, as well as monitoring and analyzing the results of the tests or simulations.
- the steps in this process may be performed using various components of a simulation lab and/or other physical test (or simulation) environment 700 , described above.
- each of steps 801 - 810 may be performed by a computer server of a test administrator associated with a physical simulation environment 700 .
- physical simulation environments 700 might be configured to receive test content and configuration parameters, to execute the tests and monitor the execution, and then to transmit the test results and related observation data to a separate server (e.g., a digital credential platform server 610 ) for scoring and analysis.
- a separate server e.g., a digital credential platform server 610
- a computer server controlling the physical testing environment 700 may receive input relating to the test or skills evaluation scenario to be executed within the physical testing environment 700 .
- the server may receive data identifying the particular user designated to complete the test or skills evaluation scenario.
- the server may retrieve the test or scenario to be loaded/executed within the physical testing environment 700 .
- the test or scenario may include interactive user software (e.g., driving or flight simulator programs, law enforcement scenarios, etc.) and/or may include testing software or other software programs loaded onto a desktop, laptop, or tablet computer.
- the test or scenario may require the user to work with computer-aided design software, spreadsheet software, database development software, etc.
- the test or scenario may include audio and/or video files to be played via speakers and/or display screens within the physical testing environment 700 , such as instructional videos or audio/visual test questions.
- the test or scenario retrieved in step 803 also may be retrieved based on the identity of the particular user who will be completing the test or skills evaluation scenario.
- the server of the physical testing environment 700 may be configured to select the appropriate test, scenario, and/or simulation (e.g., a particular software scenario, skill level, etc.) based on the user's current set of badges or digital credentials, the user's skill level, and/or the user's performance history on previous tests or scenarios within the testing environment 700 .
- the server may vary scenarios/test questions so that a particular user does not receive the same test questions, scenarios, or other testing content that they have already completed (or completed within a particular recent time window).
- the server may determine and apply a set of environmental conditions within the physical testing environment 700 for the execution of the test or scenario.
- the physical testing environment 700 in some embodiments may be capable of setting various environment conditions such as lighting (e.g., to simulate different day or night, and/or different real-world working environments), temperature and weather conditions (e.g., to simulate outdoor scenarios, different seasons and locations), noise (e.g., to provide background noise, traffic noise, distractions, etc.) and other various environment conditions.
- the server may select and apply environmental conditions as part of the test or scenario selected in step 803 , or as a separate determination which is performed based on random chance or selected by a test administrator, etc.
- the environmental conditions may be selected randomly and changed for each testing session.
- user may select and/or save their preferred environmental conditions for different types of testing.
- the physical testing environment 700 may track and analyze the user's various testing or scenario performance metrics (e.g., accuracy, efficiency, safety, compliance, biometrics, etc.) under different environmental conditions, in order to determine the optimal environmental conditions for the particular user.
- user's may receive different badges or certifications (or may have different badge assigned characteristics or endorsements) based on their test or scenario performance in different environmental conditions.
- the computer server(s) associated with the physical testing environment 700 may execute the test or simulation scenario, during which the user's performance and any/all user reactions or responses may be monitored.
- the physical testing environment 700 may use cameras and any other sensors to monitor the user's actions.
- Such monitoring may include various aspects of the user's performance, such as answers to test questions selected via a testing computer terminal, or the user's interactions with physical objects (and/or other people) within the physical testing environment 700 .
- the user's answers and actions may be recorded by cameras and computer input devices, and additional user data may be collected using various other sensors such as microphones, biometric sensors, etc.
- the results for the test and/or simulation scenario completed by the user may be analyzed.
- the such analyses may be performed based not only on the user's responses to particular test questions or scenarios.
- the analysis in step 806 may include an evaluation of the user's other reactions or responses, such as speed and confidence of action (e.g., as determined by user comments, speed of response, facial expression analysis, body movement analysis, biometric data, etc.), efficiency, safety, decision making, and user biometrics.
- speed and confidence of action e.g., as determined by user comments, speed of response, facial expression analysis, body movement analysis, biometric data, etc.
- efficiency, safety, decision making, and user biometrics e.g., efficiency, safety, decision making, and user biometrics.
- One or more of these separate analyses may be performed in steps 807 - 810 , and each may be performed independently of the others, or may be combined into a single analysis.
- the goal of the simulation might be only to measure the user's biometric data, and the user's actual responses to the questions/scenarios may be irrelevant and need not be evaluated in step 807 .
- the opposite analysis may be applied, where only the accuracy of the user's responses or behaviors are measured and analyzed in step 807 , and the user's biometric data is irrelevant and thus the analysis in step 810 is not performed.
- the only relevant analysis to be performed may be a safety/decision making analysis in step 809 , while the efficiency analysis in step 808 need not be performed.
- the server may apply both a safety/decision making analysis in step 809 and an efficiency analysis in step 808 (e.g., to confirm that a driving maneuver or route was completed both safely and efficiently, to confirm that a suspect was subdued safely and quickly, to assure that a manufacturing assembly task was performed safely and efficiently, etc.).
- a safety/decision making analysis in step 809 and an efficiency analysis in step 808 (e.g., to confirm that a driving maneuver or route was completed both safely and efficiently, to confirm that a suspect was subdued safely and quickly, to assure that a manufacturing assembly task was performed safely and efficiently, etc.).
- various techniques may include evaluating a physical simulation by a digital credential generator system (e.g., 630 and/or 610 ).
- Such techniques may include monitoring, by a digital credential generator system, a physical simulation area using a plurality of sensors, during a physical simulation.
- the digital credential generator system may detect, using the plurality of sensors, various physical actions performed by a user during the physical simulation.
- the digital credential generator system may analyze data corresponding to the plurality of physical actions performed by the user during the physical simulation, and determine that the user corresponds to a particular credential receiver. The digital credential generator system then may determine whether the first credential receiver is eligible to receive a digital credential, by comparing the data corresponding to the analysis of the physical actions performed by the user during the physical simulation, to one or more digital credential requirements. Finally, the digital credential generator system may generate a first digital credential associated with the first credential receiver and with the digital credential requirements, in response to determining that the first credential receiver is eligible to receive the first digital credential.
- outputting a physical simulation may include outputting audio and/or video simulation components within the physical simulation area, manipulating physical objects (e.g., motorized objects) during a live-action simulation within the physical simulation area, and/or outputting virtual reality simulations via a virtual reality headset. Additionally, certain embodiments may include generating physical simulation environments within a physical simulation area, including, for example, simulating ambient light conditions within the physical simulation area, outputting one or more background noise conditions within the physical simulation area, monitoring and controlling the physical temperature using a heating and cooling system installed at the physical simulation area, outputting smells to the physical simulation area using a smell output device, and/or outputting vibratory effect within the physical simulation area, using a vibration system.
- generating physical simulation environments within a physical simulation area including, for example, simulating ambient light conditions within the physical simulation area, outputting one or more background noise conditions within the physical simulation area, monitoring and controlling the physical temperature using a heating and cooling system installed at the physical simulation area, outputting smells to the physical simulation area using a smell output device, and/
- the monitoring of a test/simulation may include monitoring physical actions/activities performed by the user using video recording devices and/or motions. Additionally or alternatively, the monitoring may be of computer-based tasks, using additional software-based sensors such as mouse movement trackers, keystroke loggers, and user eye-tracking software, etc.
- the processes used for testing/evaluating a user and determining that a user is eligible for a particular digital credential need not include a specific test, designated evaluation, or scored scenario training. Rather, the testing and badging determinations may be performed automatically during the user's normal course of on-the-job performance of tasks.
- the testing and credentialing of users may be based on observation of workers during their normal work activities. Cameras and other sensors may be installed and used to detect the completion of tasks and/or certain competencies of the users, and the data from these sensors may be evaluated to automatically determine when the user is eligible for a digital credential.
- the job testing and badging may be performed entirely transparently to the worker performance of their job duties, and need not require any delay or distraction from job performance, or any designated time or location needed to perform formal testing.
- the “work” environment of the user may be monitored with cameras and/or sensors capable of tracking the user's activities and performance.
- different types of digital credentials relate to different activities that may be performed in a variety of different work environments.
- FIG. 9A an example work environment 900 a is shown for a user completing computer-based tasks.
- the work environment 900 a may include a basic workstation, server, modem, printer, monitor, keyboard, etc., as well as desk and chair to allow the user to complete normal computer-based work activities.
- the user may be data entry specialist, computer programmer or design engineer, call center customer support operator, or may be performing any other computer-based job.
- sensors 905 and 910 may include cameras, network monitoring devices, keystroke loggers, mouse movement monitors, biometric devices and sensors, etc.
- Such software tools may operate as background processes on a computer terminal being monitored. Additional monitoring devices may be built into specific software programs with which the user is interacting, and may be able to determine the correctness, quality, and efficiency of the user's interaction with the particular software.
- monitoring features within the software application may be used to determine how quickly the user performed the task, how many attempts it took the user, how correct/accurate was the finished product, etc.
- the monitoring of the user's interaction with a particular software program need not involve any monitoring features within the software itself, but instead may include monitoring at the operating system or hardware layers, or monitoring that is entirely external to the workstation.
- external cameras 905 and other sensors may capture and analyze the user's interactions with the software application, and thus need not affect the operation of the software at all.
- FIG. 9B Another example work environment is shown in FIG. 9B .
- this example work environment 900 b the entire layout of workplace floor is shown and monitored by a series of cameras 905 and/or other sensors.
- the monitoring in this example may apply to works who do not perform only computer-based tasks, but whose work requires them to interact with physical objects within their workspace, and/or to move around the work environment 900 b to other workspaces. For instance, maintenance works, office mail delivery works, construction workers, electricians, plumbers, machine assembly or manufacturing works, etc., may be monitored with such systems.
- the work environment 900 b may include motion sensors, microphones and noise sensors, as wells as movement sensors and/or tracking devices that may be placed on specific physical objects within the environment.
- work environment 900 b may correspond to a shop floor, mechanic's garage, or manufacturing assembly plant, and the cameras 905 and other sensors may be used to confirm that workers are complying with safety requirements and/or health codes with respect with their work with machinery or hazardous materials, etc.
- work environment 900 b may be an office environment, and the cameras 905 and other sensors may be used to confirm that individual workers are working efficiently, in their assigned areas, etc., and that workers without assigned areas (e.g., cleaning, mail delivery, maintenance workers, etc.) are working efficiently and not skipping any portion of the floor 900 b.
- workers without assigned areas e.g., cleaning, mail delivery, maintenance workers, etc.
- FIG. 10 a flow diagram is shown illustrating an example process of automatically monitoring work activities and issuing digital credentials via “on-the-job” testing.
- the steps in this process may be performed by monitoring and credentialing computing devices operating within various types of work environments 900 , such as those described above.
- each of steps 1001 - 1006 may be performed by a computer server operating automatically and unassisted (or at the direction of an administrator) within a work environment 900 .
- work environments 900 might be configured only to monitor work activities and performance, and then to transmit the results and related observation data regarding various worker to a separate server (e.g., a digital credential platform server 610 ) for scoring, analysis, and the issuance of digital credentials.
- a separate server e.g., a digital credential platform server 610
- a computer server controlling the on-the-job badging system may activate the cameras, sensors, monitoring software, etc., within the workstation and/or work environment. As discussed above, this activation may include specific monitoring software to detect computer-based tasks, and/or location monitoring devices such as cameras, sensors, biometrics, etc., depending on the type of workers and work environments 900 being monitored.
- an on-the-job testing and credentialing system may be implemented as an “always on” system, in which the workstation/workplace monitoring is constantly recording and analyzing worker activities. Thus, step 1001 may be optional in such embodiments.
- workstation/workplace monitoring might only be activated at certain times and not others, for example, only during normal work hours, only on certain specific work days designated for work evaluation, etc.
- a system administrator and/or individual workers may activate or de-activate the workstation/workplace monitoring systems within their work environment at any time.
- Such systems need not be an invasion of privacy for any worker that does not choose for their work to be monitored and evaluated, but workers may choose to turn the monitoring systems on in order to be eligible for evaluation and earning of additional work related digital credentials and credentials.
- the workstation/workplace monitoring systems may capture the user's work-related activities and behaviors, including performing various computer-based tasks and non-computer-based tasks as discussed above.
- the user's working data as collected by the workstation/workplace monitoring systems and sensors may be analyzed by the server, in order to determine in step 1004 whether or not the user is eligible for one or more digital credentials or other credentials (e.g., professional certifications, etc.) based on their on-the job work activities.
- digital credentials or credentials may be made available to users in response to detecting that the user has successful completed one or more specialized work tasks, thus demonstrating that the user has obtained the particular skill associated with the digital credential.
- the server and/or the monitoring systems and sensors may also be configured to detect a certain level of efficiency by the user in performing the tasks, and/or may require that the user perform a certain task N number of times before the user is eligible for the digital credential or credential.
- step 1004 if the system determines that the user is eligible for one or more particular digital credentials ( 1004 :Yes), then in step 1005 the system may either issue the digital credential directly (e.g., if the workplace server is permitted to be digital credential issuer), and/or may initiate a communication session with a badging platform 610 and/or digital credential issuer 630 to request that a new digital credential is issued for the worker.
- the workplace server may provide the information identifying the worker (e.g., name, employee ID, digital credential system profile ID, etc.) to a digital credential platform 610 or issuer 630 , along with verification that the worker has completed the requirements to earn a particular digital credential.
- the servers operating at the workplace may be configured to capture evidence (e.g., video evidence, screen captures, facial/identity verification, etc.) and transmit the evidence to the digital credential-issuing authority, before the digital credential may be issued.
- the worker may be notified that they have received a digital credential based on their normal on-the-job activities.
- the worker may indicate interest in obtaining one or more particular digital credentials, and the workstation/workplace monitoring system may be configured to evaluate the worker with respect to the particular digital credentials or credentials that the worker has expressed interest in.
- a user may be informed that they are eligible for receiving a digital credential prior to the issuance of the digital credential in step 1005 , and the user may be allowed to accept or reject the digital credential. Additionally, in some cases, the user may receive status reports (e.g., daily, weekly, etc.) identifying which digital credentials the user is being monitored for, and the user's progress with respect to earning those digital credentials. This data may include indications to the worker that he/she may earn a particular digital credential after performing a task another N times, or performing the task N amount faster, or performing the task without making any errors or backtracking, etc.
- status reports e.g., daily, weekly, etc.
- various techniques may include generating digital credentials for particular credential receivers, based on monitoring of a physical environment using a plurality of sensors.
- a digital credential issuer (or generator) 630 and/or 610 may detect or receiver sensor data from a number of sensors corresponding to the user actions performed by a user within the physical environment.
- the digital credential generator may determine the operations that were/were not performed within the physical environment, based on the user actions detected.
- the digital credential issuer then may retrieve data from a credential receiver data store associated with the user/credential receiver, and determine one or more digital credential templates, based on the retrieved data associated with the first credential receiver. For each of the digital credential templates, the digital credential issuer/generator may retrieve criteria associated with the digital credential template, compare the operations performed by the user within the physical environment to the criteria associated with the digital credential template, and then determine whether or not a credential receiver is eligible to receive a digital credential based on the digital credential template. If the first credential receiver is eligible to receive a digital credential based on the digital credential template, the digital credential generator may generate a digital credential based on the digital credential template and user data associated with the first credential receiver.
- the generated (or issued digital credential) may be embedded with additional data such as the evaluation/simulation time, location, or the sensor system/physical environment within which the evaluation/simulation was performed.
- facial recognition data and/or biometric data may be collected from the user (credential receiver), and may be used to validate or authenticate the digital credential by verifying the user's identity.
- the monitoring may be done using physical movement tracking sensors such as video recorders and/or motion detectors, or may use software-based sensors such as network monitoring devices, keystroke loggers, mouse movement monitors, touch screen monitors.
- software-based tools may operate as background processes on a computer terminal being monitored, and/or may be built into specific software programs with which the user is interacting.
- post-credentialing monitoring of the user may involve analysis of user's physical work product (e.g., documents produced, parts/items created, etc.), or may be involve observations of the user (e.g., via a workstation/workplace monitoring system).
- a data store of digital credentials may be linked to particular skills, work-related, or activities. The user/worker may then be tracked to determine the number of such tasks performed, and/or the quality, efficiency, and/or competence of the user's performing those tasks, in order to determine to what extend the user/worker is “using” the digital credential.
- an example computing environment 1100 including a digital credential platform server 1110 , one or more workstation/workplace monitoring systems 1120 , and a credential-to-activity mapping data store 1130 .
- the digital credential platform server 1110 may be a badging server similar or identical to the server 610 discussed above.
- server 1110 may be configured as a digital credential repository and credentialing system, acting as a clearinghouse for digital credential owners, issuers, earners, endorsers, etc.
- Server 1110 may include a digital credential (or digital credential) data store configured to store badging information such as the details of the particular digital credentials earned by particular users. As noted above, such details may include the date on which a digital credential was issued to a user, and for certain digital credentials, an expiration date associated with the digital credential.
- system 1100 also includes a credential-to-activity mapping data store 1120 , which may be implemented as a separate external data store and/or may be integrated into the digital credential data store of server 1100 .
- the credential-to-activity mapping data store 1130 may include mappings of one or more tasks or activities associated with each digital credential type that a user may potentially earn.
- a digital credential relating to automotive maintenance for a particular make of car may have associated activities and tasks that include particular maintenance tasks (e.g., tune-ups, part replacements, etc.) for different model cars having the make.
- an operating system administrator-related digital credential may list, within data store 1130 , various system administrator tasks and that a user may perform on the particular operating system.
- the activities or tasks associated with a particular digital credential may correspond to the same set of activities or tasks that a user is required to perform to earn the particular digital credential, and as discussed below, these activities or tasks may serve as a metric to evaluate how much the user is “using” the digital credential.
- One or more workstation and/or workplace monitoring systems 1120 may provide user monitoring data to the server 1110 , to allow the sever 1110 to analyze the user's activities and determine to what extent the user is using the activities and abilities associated with their digital credentials.
- the workstation and/or workplace monitoring systems 1120 may be similar or identical to any of the workstation/workplace monitoring systems and sensors discussed above.
- workplace monitoring systems 1120 may collect records detailing the user's physical work product (e.g., documents produced, modified or accessed by the user, inventory or work order records indicating tasks performed by the user, etc.).
- workplace monitoring systems 1120 may include observation systems (e.g., workplace monitoring systems) including cameras and other sensors to track the user's activities and determine which specific tasks have been performed by the user.
- the monitoring and tracking of post-credentialing activities by the user may be used to analyze and provide digital credential or credential feedback data to various entities.
- FIG. 12 a flow diagram is shown illustrating an example process that may be used to determine whether a user has or has not used the activities associated with a particular digital credential that they have obtained, and then to aggregate and report that digital credential usage data to the relevant parties.
- a particular digital credential is issued to a user based on the user's successful completion of the badging requirements.
- the digital credential may be associated with a computer-based activity, non-computer-based activity, or any other set of digital credential requirements determined by a digital credential owner or issuer.
- the digital credential issuance in step 1201 may be the result of formal testing and/or certification processes, or may be based on on-the-job or other observational data collected for the user.
- the digital credential server 1110 and/or monitoring systems 1120 may monitor and track the activities of the credentialed user, including, for example, the workplace tasks performed by the user based on analyses of the various monitoring systems/sensor data installed at the user's workstation and/or workplace environment. As described above, determining what activities and tasks the credentialed user has performed, and when, may be performed using a variety of techniques. In some cases, determining what work-related tasks a user has performed, and what other activities they have been engaged in, may be done by analyses of written and electronic documents associated with the user or workplace. For instance, documents such as maintenance requests, work orders, customer tickets, purchase receipts, and the like may be analyzed to determine what activities or tasks the user has completed and when.
- a maintenance record listing the user as the assigned technician may be used in determination that the user has performed the specified task/activity at the time listed on the record.
- the user's electronic mail and other electronic documents may be searched and analyzed (e.g., using a keyword analysis and/or trained artificial intelligence) to determine what tasks the user has performed and/or what activities the user has demonstrated during the relevant time periods.
- a digital credential such as leadership, communication skills, advanced C software programming, jujitsu skill levels, and the like, it may be more difficult to quantify if when, and how often a user is using the particular skill or task associated with the digital credential.
- a set of tasks and/or activities associated with the digital credentials obtained by the specific user may be retrieved using the credential-activity mapping data store 1130 , and in step 1204 the retrieved tasks and/or activities may be compared to the tasks and activities that have been performed by the user subsequent to the digital credentials being earned (as determined in step 1202 ).
- the comparison in step 1204 may determine that in the six month since the user was issued a professional certification to perform a particular technical task, the user has performed that task on a weekly basis.
- the comparison in step 1204 may determine that the user has used that software program only once since receiving the digital credential two years ago. In this case, the system may conclude that the professional certification issued six months ago to the user has been of greater usefulness than the software digital credential issued two years ago (allowing for the possibility of career changes, prestige-driven digital credentials rather than functional digital credentials, etc.).
- step 1205 data from the comparison of step 1204 , i.e., data indicating the post-credentialing usage by the user of the digital credential-associated activities or tasks, may be aggregated and analyzed, and then transmitted to one or more of the relevant system components.
- any of several different components and roles associated with the credentialing platform 1110 may request and receive this information for their associated digital credentials and/or associated users.
- digital credential owners and/or digital credential issuers may request and receive from the platform server 1110 data regarding the post-issuance usage of the digital credentials they own or have issued.
- digital credential endorsers may request and receive from the platform server 1110 data regarding the post-issuance usage of the digital credentials they have endorsed.
- Digital credential earners the users themselves also may request reports from the platform server 1110 quantifying the post-credentialing usage (which may be expressed in terms of time, value, and/or dollar amounts) associated with their previously earned digital credentials.
- Employers and other organizations also may request such reports for their employees or organization members, in order to determine which digital credentials have been the most used and most useful to the organization.
- FIG. 13 another flow diagram is shown illustrating an related process involving determining whether a user has or has not used the activities associated with a particular digital credential that they have obtained, and then adjusting an expiration or re-certification date associated with the digital credential based on the user's usage of the digital credential skills.
- the steps in this example may be similar or identical to the corresponding steps in FIG. 12 , and in some embodiments, the analyses and transmission of the post-credentialing usage described in step 1205 may be performed in conjunction with the setting of an expiration or re-certification date for the digital credential as discussed below.
- Steps 1301 - 1304 may correspond to steps 1201 - 1204 in some cases, and may be performed using similar or identical techniques to those discussed above.
- a platform server 1110 and/or digital credential issuer may issue a digital credential associated with one or more activities or tasks to a particular user, recording the digital credential issuance data within the digital credential data store.
- the post-issuance activities of the particular user may be monitored, including monitoring of the user's work-related activities and tasks performed/completed, in order to determine the particular tasks and activities with which the user has been engaged following issuance of the digital credential.
- step 1303 the skills, activities, and tasks associated with the user's digital credential(s) are retrieved, and in step 1304 are compared to the post-issuance user tasks and activities determined for the user in step 1302 . Finally, in step 1305 , based on the comparison in step 1304 , the platform server 1110 may determine that an expiration date and/or recertification date associated with the user's digital credential should be adjusted based on the user's post-issuance activities.
- the system may determine that the user's license should expire at the earliest possible time (e.g., the expiration time as of when the digital credential was first issued).
- the system may determine that the user's license may be extended.
- the platform server 1110 may determine a new extended expiration or recertification time for the digital credential, update the user's digital credential record within the digital credential data store, and transmit notifications to the affected entities (e.g., the user, employer, digital credential issuer, digital credential owner, etc.) providing the new expiration date.
- the platform server 1110 may in other examples determine a new recertification course or procedure for the user, such as simple refresher course to allow the user to recertify quick than the longer complete recertification course used by other users with less post-credentialing digital credential usage.
- Additional aspects described herein relate to capturing and using “evidence” data in connection with user testing and credentialing systems, on-the-job evaluation and badging systems, and/or post-credential monitoring systems.
- evidence of the user's performance may be extracted and saved, for example, in a digital credential server along with an associated issued digital credential, or as part of a separate user portfolio of evidence.
- Evidence data may include, for example, audio and video of the user during a live simulation, or during a virtual reality or augment reality simulation, audio and keystroke data from the user during the testing processing, the user's reaction time and/or decision-making data during a split-second simulated scenario or relevant real-life event (e.g., a workplace accident, etc.), and/or any other sensor or biometric data collected during testing, credentialing, and/or monitoring.
- evidence data associated with a user may be saved with the user's digital credential and/or into a separate portfolio of evidence, which may be available to the user for review, and also may be provided upon request to potential employers for review during a review or hiring process. Such evidence data also may be applied to updated digital credential credentialing requirements, so that in some cases a user may simply resubmit their evidence portfolio instead of being required to recertify their digital credential when the test or credentialing standards are updated.
- a digital credential platform server 1410 in communication with a plurality of testing, credentialing, and/or monitoring systems 1421 - 1423 , and one or more external client devices 1460 .
- the digital credential platform server 1410 may be a digital credentialing server similar or identical to the server 610 discussed above.
- server 1410 may be configured as a digital credential repository and credentialing system, acting as a clearinghouse for digital credential owners, issuers, earners, endorsers, etc.
- Server 1410 may include a digital credential (or digital credential) data store configured to store digital credential information such as the details of the particular digital credentials earned by particular users.
- such details may identify the digital credential issuer and/or other testing/credential authorities responsible for administering testing or simulation scenarios as part of the digital credentialing process, and/or for pre-digital credential or post-digital credential monitoring of workstations/workplaces to detect and analyze user tasks performance and user skills/abilities
- the platform server 1410 may receive data from three testing/credentialing systems 1421 - 1423 .
- the simulation lab system 1421 may correspond to a simulation lab or other physical testing environment
- an on-the-job credentialing systems 1422 may include workstation/workplace monitoring systems and sensors to record and analyze the user's on-the-job performance, and may issue digital credentials in some cases without the need for any separate formal testing procedure
- post-credential monitoring systems 1423 may be configured to monitor users following the issuance of a digital credential, including tracking task performance data, skills usage, and the like, and comparing the data to the skills/tasks associated with the user's digital credentials.
- one or more systems 1421 - 1423 which perform user testing, credentialing, and/or monitoring, such as those systems discussed above, may capture and transmit “evidence data” of the user during a test, simulation, or during an on-the-job monitoring process.
- Evidence data may include, for example, video and/or audio of the user during a test, simulation (e.g., live, VR, or AR), collected by the sensors of a physical testing environment 700 .
- Additional evidence data may include user reaction time data, decision-making data, facial expression and body language data, keystroke and mouse movement data, and/or user biometric data.
- the evidence data may correspond to a time period just before, during, and just after a test, simulation, or a task or activity performed during on-the-job monitoring.
- the various evidence data collected by systems 1421 - 1423 may be transmitted to the platform server 1410 and stored in an evidence portfolio data store.
- the evidence data collected by the testing, credentialing and/or user monitoring systems may be associated with a particular user (or users) and with a particular digital credential (or digital credentials) that the user is in the process of earning or using (e.g., for post-credentialing monitoring).
- the evidence data may provide documented proof that the user actually completed the digital credential requirements, along with additional contextual evidence showing how the user performed during the testing, simulation, or monitoring.
- a flow diagram is shown illustrating an example process by which a testing system, simulator, credentialing systems, workstation/workplace monitoring system, and the like, may collect and preserve evidence data related to a user and a digital credential.
- a testing, credentialing, and/or monitoring system such as those described above may execute a test, simulation, or user monitoring process for a particular user in connection with a digital credential that the user is seeking or has already obtained.
- the particular types of tests may include, for example, live simulations and/or virtual or augmented reality simulations executed within a physical testing environment 700 .
- the testing in step 1501 may correspond to an on-the-job credentialing system that monitors and evaluates a user's workplace tasks and activities, or to a post-credentialing user monitoring system configured to determine whether the user is using their previously issued digital credentials.
- the system 1421 - 1423 may capture evidence data relating to the user.
- evidence data may include audio or video of the user, user reaction time data, decision-making data, facial expression data, body language data, the user's keystrokes and mouse movement data, particular software interaction data, and/or the user's biometric data.
- the evidence data may be encapsulated and transmitted to the platform server 1410 for storage within the user's evidence portfolio, and in step 1504 the platform server 1410 may store the evidence data files with data records associated with the user and the particular digital credential(s) to which the evidence applies.
- certain systems 1421 - 1423 may retain and store user evidence data locally, rather than the evidence data being stored in a central repository. Additionally, when the evidence data is transmitted, it may be compressed and edited as needed, and/or encrypted in order to assure data security and user privacy.
- the platform server 1410 may determine a subset of the user activities matching digital credential requirements associated with the digital credential, wherein other user activities might bare no relevance to the requirements of the digital credential. In such cases, the platform server may store only the corresponding subset of the evidence/sensor data for the user activities matching digital credential requirements, and might not store other evidence/sensor data corresponding to irrelevant user activities upon which digital credentials do not depend.
- FIGS. 16A and 16B two additional flow diagrams are shown illustrating example processes by which evidence data may be retrieved and/or accessed from a platform server 1410 or other data repository.
- individual evidence data files stored by the platform server 1410 may be associated with a particular user and/or with a particular digital credential or credential earned (or in process of earning) by the user.
- evidence data may be stored and made available to certain authorized entities. For instance, in step 1601 of FIG. 16A , the platform server 1410 may receive a request for some or all of the user's evidence portfolio.
- the platform server 1410 may perform authorization/authentication on the request to determine (1) whether the requestor is authorized to access the user's evidence data, and/or (2) whether the requested evidence is current and valid. One or both of these determinations may require explicit authorization from the user himself or herself, in order to (1) prevent any unwanted parties from accessing the user's evidence data, and (2) to prevent any old and obsolete from being accessed, even by authorized parties.
- step 1602 may include verifying the requestor's identity or role and comparing to an access control list or other permissions data associated with the evidence.
- step 1602 may include a real-time request sent by the platform server 1410 to a client device associated with the user, to allow the user the option to allow or reject the request.
- the request in step 1601 may specify one or more particular users and/or one or more particular digital credentials for which the associated evidence is to be retrieved, and thus authorization in step 1602 may be granted or denied for evidence relating to each possible combination of users and digital credentials.
- the requestor has been granted access to the requested evidence data, the corresponding evidence data files may be retrieved and forwarded to the requestor.
- the request in step 1601 may be from the user himself/herself, who wants to review and study the evidence from his/her previous tests, simulations, and monitoring data.
- the request in step 1601 may be from a current or potential employer, who has been authorized by the user to retrieve and view the user's evidence data associated with all work-relevant digital credentials, as part of a hiring process or review process.
- the user's evidence data may verify to the employer or potential employer that the user actually completed the digital credential requirements, and also may allow the employer or potential employer to observe the user's behaviors, responses, reactions first-hand, thus allowing them to evaluate the user's reaction time, efficiency, mental state, decision-making, etc., and other difficult to quantify characteristics.
- the user may authorize a digital credential issuer or digital credential owner to view the user's evidence files related to the digital credentials issued and owned by those entities.
- users may make some or all of their evidence data publicly available (e.g., on a file-by-file basis) and/or may actively post their evidence data as a multimedia file or data records within a digital credential profile page of the user that is maintained and published by the platform server 1410 .
- the platform server 1410 may provide the functionality to receive updated tests, digital credential requirements, credentialing data, etc., and to apply a user's previously stored evidence to the new testing or credentialing requirements. For instance, in step 1604 of FIG. 16B , the platform server 1410 may receive a request to apply previously stored evidence data within a user's portfolio to an updated testing/credentialing process.
- testing or credentialing authorities may periodically update digital credentialing requirements in order to improve the quality of the digital credential testing, to comply with new best industry practices, to make a digital credential more restrictive by increasing the required scores or efficiency, etc.
- certain testing or credentialing authorities may implement multiple different levels of the same digital credential, in which users are subjected to the same test, same simulation, same monitoring processes, etc., but different scoring ranges may equate to different levels of the digital credential that may be earned by the user.
- a set of new requirements for particular digital credential may be similar to the previous set of requirement, with the addition of a newly imposed time limit by which the test or simulated scenario must be completed.
- new digital credential requirements or digital credential levels may raise the minimum performance level during a test or simulation to a higher level, and/or may require additional steps or procedures during the test or simulation that were not required in the previous version of the digital credential requirements.
- the platform server 1410 may provide the service of receiving the updated digital credential requirements or new digital credential levels, and automatically evaluating the new digital credential requirements/levels using the user's evidence data that was collected with earning the previous version of the digital credential.
- the requestor may be authenticated and the requested data may be validated, and in step 1605 the user's evidence data may be applied the updated testing/credentialing process.
- the evaluation in step 1606 may include automated analysis of the user's evidence data to determine whether the user complied with the newly imposed time limit, the new minimum performance level, and/or performed the additional new steps or procedures during the user's previous digital credential testing. If so, the digital credential authority may allow the user to upgrade their digital credential automatically without having to retake the test or simulation, etc. If not, the user may be informed that they are required to retake the test or simulation (or in some cases they may receive a lower digital credential level). Either way, in step 1607 , the results of the evidence analysis and application to the new credentialing requirements may be output to the requestor.
- Another potential advantage in certain embodiments may include the protection of the user's evidence data itself.
- the platform server 1410 might perform the analysis and application of the user's previously stored evidence data to the new testing requirement, without ever allowing any other entity access to the evidence data.
- the platform server 1410 may perform the analysis and/or may provide the actual evidence data files to the requestor device, with the sufficient authorization from the user.
- the updated testing/credentialing process in step 1604 may correspond to a re-issuance of a digital credential, with the same or updated requirements, or may corresponding to a different digital credential having similar and/or overlapping issuance requirements.
- the platform server 1410 may receive an updated set of requirements for a digital credential previously issued to a credential receiver, may retrieve the stored set of sensor data corresponding to the relevant activities performed by the credential receiver in connection with the issuance, may compare the retrieved set of sensor data/activities to the updated set of digital credential requirements, and then may generate/issue an updated digital credential to the credential receiver, based on the comparison of the retrieved set of sensor/activity data to the updated set of digital credential requirements. Similar techniques may be performed to generate and issue digital credentials to receivers for entirely different digital credentials, rather than updated credentials, such as similar credentials and/or credentials having overlapping eligibility requirements.
- Additional aspects described herein relate to capturing and using user biometric data, physical user cues, and the like, in connection with user testing and credentialing systems, on-the-job evaluation and badging systems, and/or post-credential monitoring systems.
- data identifying particular physical user cues and/or user biometric data may be collected during testing/simulation/monitoring processes and saved, for example, in a digital credential platform server along with an associated issued digital credential and/or the associated user.
- Physical user cues may include, for example, facial expressions, user reactions and/or noises made by the user during testing/simulations, user body language, eye movement, and any other user behavior or reaction detectable via cameras and external sensors.
- biometric data also may be collected during the testing, simulation, and/or monitoring processes performed on the user.
- biometric data may include, for instance, the user's temperature, heartrate, blood pressure, respiration, skin conductivity, and brainwave activity, and/or any known types of biometric data that may collected during testing, credentialing, and/or monitoring processes.
- the user's physical cues and/or biometric data may be collected and saved within a digital credential platform server, and associated with the user, one or more particular digital credentials, and/or with the particular testing/simulation/monitoring processes during which the data was originally detected.
- the data may be used to authenticate the testing, simulation, and/or monitoring processes, to confirm the user's identity and to prevent errors or fraudulent activities by users.
- the data may be saved with the user's digital credential and/or into a separate portfolio of evidence, which may be available to the user for review, and also may be provided upon request to potential employers for review during a review or hiring process.
- Such evidence data also may be applied to updated credentialing requirements, so that in some cases a user may simply resubmit their evidence portfolio instead of being required to recertify their digital credential when the test or credentialing standards are updated.
- the user's physical cues and/or biometric data also may be analyzed to determine the user's emotional states and reactions during the testing, simulation, and/or monitoring. Additionally or alternatively, the physical cues and biometric data may be detected for several users and analyzed collectively to provide feedback regarding the digital credential testing processes, simulations, monitoring, physical testing environments, etc.
- FIGS. 17A-17B examples are shown illustrating facial recognition and analysis functionality that may be performed in connection with a user testing/credentialing process (live or simulation), or with user on-the-job credentialing or monitoring processes.
- one or more cameras may be configured to capture the user's facial features and expressions at different points during the testing/credentialing/monitoring processes.
- a number of designated cameras may capture not only the user's face but also the user's body from several different angles.
- certain physical testing environments may be capable not only of capturing facial images of the user, but also detecting detailed facial expressions at different times during the test/simulation, and potentially eye movement patterns, gestures, body language, and any other non-verbal and non-written user expression data.
- the physical cue data and/or biometrics data collected may be limited by the cameras and sensors available.
- a laptop camera or webcam installed at the user's workstation may be use to capture facial images and/or to recognize facial expressions at different times during the testing/monitoring.
- such cameras may or may not have the resolution and image capture capabilities to perform advanced facial expression monitoring, eye movement, and/or body language detection.
- facial images might only be detectable using lower-quality security cameras or the like that are configured to monitor an entire floor or workspace. In such examples, the facial images may be still be useful for certain purposes (e.g., confirmation of user identification), but potential may be unsuitable for facial expression analysis, eye movement analysis, and the like.
- physical testing environments e.g., simulation labs
- workstation or workplace monitoring systems may include various biometric sensors configured to detect biometric data of the user at different times during the test/simulation.
- biometric data may include the user's temperature, heartrate, blood pressure, respiration, skin conductivity, and brainwave activity, and/or any known types of biometric data.
- the biometric metric may be detected and captured via a combination of external sensors, wearable sensors, and/or implanted sensors in some cases.
- mobile wearable sensors such as heartrate monitors, step trackers, and the like, may be used when more advanced wearable sensors (e.g., blood pressure, respiration, skin conductivity, brainwave activity, etc.) are not practical.
- advanced wearable sensors e.g., blood pressure, respiration, skin conductivity, brainwave activity, etc.
- a flow diagram is shown illustrating an express process of collecting physical cue data and/or biometric data for a user during a user testing, credentialing, or monitoring processes, and using the physical cue and biometrics to authenticate the user's identity and the associated data.
- the process shown in this example may be implemented within any of the testing/credentialing systems, simulators, workstation or workplace monitoring systems, and the like described herein.
- a testing, credentialing, and/or monitoring system such as those described above may execute a test, simulation, or user monitoring process for a particular user in connection with a digital credential that the user is seeking or has already obtained.
- the particular types of tests may include, for example, live simulations and/or virtual or augmented reality simulations executed within a physical testing environment 700 .
- the testing in step 1801 may correspond to an on-the-job credentialing system that monitors and evaluates a user's workplace tasks and activities, or to a post-credentialing user monitoring system configured to determine whether the user is using their previously issued digital credentials.
- one or more of the user monitoring devices described above including cameras, microphones, motion sensors, tracking devices, and/or user biometrics sensors, may capture physical cues from the user and/or biometric data of the user during the testing, simulation, or monitoring processes.
- Such physical cues may include particular facial expressions, user reactions and/or noises made by the user during testing/simulations/monitoring, as well as user body language and eye movements.
- the physical cue and user biometric data may be encapsulated and transmitted to the transmitted to the platform server 1410 .
- certain systems e.g., 1421 - 1423
- the physical cues and biometrics data of the user may be associated with particular test questions and/or particular time stamps during a testing or simulation. Additionally, when the data is transmitted, it may be compressed and edited as needed, and/or encrypted in order to assure data security and user privacy.
- the platform server 1410 may use the physical cues and/or biometrics data collected for the user as part of an authentication process in step 1804 .
- any testing/credentialing process e.g., written testing, computer-based testing, simulation lab testing, etc.
- the user's facial images, physical cues, and/or biometrics may be compared against previously stored corresponding data (e.g., user images, physical cue patterns, biometrics, etc.) in order to verify that the correct user is taking the test/simulation.
- the user's physical cues and biometrics may provide an additional level of authentication, by comparing the observed physical cues and biometrics at particular times during the test or simulation to expected physical cues and biometrics, based on what is happening during the test or simulation at that particular time. For instance, a simulation may be designed to present a challenging and stressful situation to the user at a particular timestamp or within a sequence of tasks the user is performed.
- the server may compare the user's observed physical cues and biometrics to the physical cues and biometrics that would be expected for the challenging and stressful situation, in order to confirm that the data is valid and/or that the user did not expect this situation in advance (e.g., indicating cheating).
- the platform serving 1410 having validated the user's identity and the authenticity of the user's physical cues and biometrics, may store the testing, credentialing, monitoring data in the digital credential data store as valid data.
- the image data, facial cues, and/or biometrics data also may be retained and stored by the platform server for future analysis.
- the data relating to the user's physical cues and biometrics collected during a test, simulation, or during on-the-job monitoring may be further evaluated to identify the user's emotional states at different times. For instance, certain simulations may be specifically designed to invoke certain emotional states (e.g., anger, boredom, frustration, surprise, etc.), and the user's level of performance while experiencing those emotional states may be particularly important for certain testing/credentialing processes. In these examples and other cases, either exhibiting or not exhibiting particular emotion states may be an eligibility requirement for a credential receiver to obtain certain types of digital credentials.
- certain simulations may be specifically designed to invoke certain emotional states (e.g., anger, boredom, frustration, surprise, etc.), and the user's level of performance while experiencing those emotional states may be particularly important for certain testing/credentialing processes.
- either exhibiting or not exhibiting particular emotion states may be an eligibility requirement for a credential receiver to obtain certain types of digital credentials.
- the data collected during the test, simulation, or monitoring in step 1801 may be used not only for user identification/authentication, but also may be analyzed to (1) determine the user's emotional state at different times during the test, simulation, or monitoring, (2) compare that emotional state to an expected emotional state based on what the user is experiencing, and (3) evaluate the user's reactions, levels of skills performance during different emotional states.
- the physical cues, biometrics data, and/or emotional states detected for multiple users may be aggregated for the same tests, simulations, monitoring environments, etc.
- the aggregated data for tests may be used to revise current tests and simulations, design new tests and simulations, and for training users how to respond to particular scenarios and situations (e.g., workplace accidents).
- an automatic recruiting engine may analyze available data within a digital credential platform and determine matches between individuals (e.g., digital credential receivers) and potential employers.
- an automatic recruiting engine may perform some or all of the tasks of a professional recruiter, and may take the place of a recruiter role or a job seeker role by proactively performing matching analyses between available/potential job listings and potential candidates for those jobs.
- the automatic recruiting engine and/or associated digital credential platform may provide notifications to users (e.g., credential receivers) of job listings for which the user is qualified, and/or notifications to employers that may be looking candidates having a specific skill set and/or matching a specific profile.
- automatic recruiting engine may also include functionality to provide credential receivers with suggestions of particular digital credentials that may expand their job prospects, based on analyses of the current job market/job listings and the user's digital credential portfolio and other qualifications.
- An advanced skills-based or digital credential-based matching analysis may be performed, but in certain examples the analysis may include additional factors such as age, seniority, worker career arc, personal candidate data, location, salary expectations, etc.
- an example computing environment 1900 including a digital credential platform server 1910 , in communication with employer client devices 1960 , digital credential issuer devices 1930 , and/or bade earner devices 1940 .
- the digital credential platform server 1910 may be a digital credential server similar or identical to the digital credential server 610 discussed above.
- server 1910 may be configured as a digital credential repository and credentialing system, acting as a clearinghouse for digital credential owners, issuers, earners, endorsers, etc.
- server 1910 may include an automated recruiting engine 1915 configured to perform, among other tasks, an automated matching analysis between potential employees and jobs/employers.
- the recruiting engine 1915 may retrieve and analyze worker-related and/or job related data from various data stores, which are shown as internal data stores or database in this example but may be implemented as external data sources in other cases.
- digital credential data stores may be configured to store credentialing information such as the details of the particular digital credentials earned by particular receivers (e.g., digital credential portfolios), including any combination of the digital credential data and associated data discussed above.
- a user data store may store various user data for users who have earned digital credentials stored within the digital credential data store. Such user data may include demographic data, employment and educational data, other qualifications, current employment details, current salary and salary preferences, current hours/lifestyle and hours/lifestyle preferences, current work satisfaction, current work location and work location preferences, etc.
- a job listing database may include recent/current jobs postings from various employers. Jobs postings in this database may include a variety of job description data, qualification requirements, and other factors, include data such as job title, job description, required digital credentials, required skills, required educational qualifications, required abilities/traits, job location, job hours/days commitment, job salary or range, company information, and the like. Additionally, in some embodiments data from an employment database also may be used in the matching analyses performed, including data relating to the jobs/positions currently held by workers at different companies. Such employment data may include, for a company's current workforce, current positions held, salaries, locations, job descriptions, skills, requirements, qualifications, technologies used, current job satisfaction level, etc.
- the digital credential platform server 1910 may receive such data from various external data sources, including employer devices 1960 , digital credential issuer devices 1930 , and digital credential receiver devices 1940 , as well as other external data sources including job data, employment market data, technical/skills data, etc.
- the automated recruiting engine 1915 within server 1910 may use various matching algorithms, analytics engines, and/or artificial intelligence components to analyze the data and identify potential matches between workers and jobs/companies.
- FIG. 20 a flow diagram shown illustrating an example process of analyzing worker/employee data, and job/company data, in order to determine potential matches that may be suggested to one or both parties.
- this process may be performed by digital credential platform server 1910 , using an automated recruiting engine 1915 to retrieve and analyze data in order to determine possible matches.
- the recruiting engine 1915 may retrieve digital credential data, user data, employment data, and/or job listing data relating to one or more employers and users within the digital credential network 1900 .
- the data retrieved in step 2001 may include any or all of the data from the various data sources discussed above.
- the recruiting engine 1915 may analyze the data retrieved in step 2001 to determine one or more potential matches between workers and jobs.
- the analysis in step 2002 may use data matching algorithms, analytics engines, and/or artificial intelligence components to analyze the data and identify potential matches between workers and jobs.
- the analysis in step 2002 may be limited to only job seekers, while in other cases the analysis may include currently employed workers (e.g., so the recruiting engine may potentially suggest a new job or career change). Additionally, in some cases, the analysis in step 2002 may be limited to only existing job postings from employers, while in other cases the analysis may match a worker to an employer even if that employer has is not currently hiring.
- the matching in step 2002 may be based on any combination of the user data, digital credential/skills data, job data, and employment data discussed above.
- the recruiting engine 1915 may even include data points such as the satisfaction level of a current employee which may be determined based on employee surveys or other direct feedback, or inferred based on job seeking/web-browsing behavior, a decline in performance, and/or recently acquiring new digital credentials or other qualifications which may indicate an intention to change jobs or careers.
- the platform server in step 2002 may determine a set of capabilities of the credential receiver, based on the digital credentials issued to the credential receiver, and then determine a matching set of field data objects (e.g., job listings) having capability characteristics matching the determined set of capabilities of the credential receiver.
- a matching set of field data objects e.g., job listings
- the automated recruiting engine 1915 may determine whether or not to transmit notifications based on the worker-to-job/company matches identified in step 2002 . In some cases, the determination of whether to transmit a notification may be based on the strength of the match (e.g., a high correlation between the worker characteristics and job listing characteristics greater than a similarly threshold would trigger notifications).
- the determination also may be based on whether or not the individual user and/or employer has requested or subscribed for such notifications, including any specific criteria provided by the worker (e.g., only notify me of my top 3 matches per week, only notify me of jobs in California, only notify me of jobs with a 15% salary increase, only notify me of jobs that use my most recent acquired digital credential-related skills, etc.) and/or specific criteria provided by the employer (e.g., only notify us for matches of current employees of a competitor company, only notify us for matches who have digital credential ABC, etc.).
- the notifications determined in step 2003 may be generated by the automated recruiting engine 1915 and transmitted from the digital credential platform server 2010 to the appropriate party.
- such notifications may inform the worker or employee of the potential match, but might not reveal the identity of the matching counterparty until a later time (e.g., until both parties except the match or until the party has paid their subscription fee, etc.).
- a digital credential platform server may transmit a notification to the credential receiver identifying correlation metrics for one or more field data objects, but the notification purposefully might not identify selected field data object.
- the digital credential platform server may transmit a second notification to an owner associated with a selected field data object, wherein the second notification identifies the correlation metric for the selected field data object but does not identify the credential receiver.
- Implementation of the techniques, blocks, steps and means described above may be done in various ways. For example, these techniques, blocks, steps and means may be implemented in hardware, software, or a combination thereof.
- the processing units may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described above, and/or a combination thereof.
- ASICs application specific integrated circuits
- DSPs digital signal processors
- DSPDs digital signal processing devices
- PLDs programmable logic devices
- FPGAs field programmable gate arrays
- processors controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described above, and/or a combination thereof.
- the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a swim diagram, a data flow diagram, a structure diagram, or a block diagram. Although a depiction may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged.
- a process is terminated when its operations are completed, but could have additional steps not included in the figure.
- a process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function.
- embodiments may be implemented by hardware, software, scripting languages, firmware, middleware, microcode, hardware description languages, and/or any combination thereof.
- the program code or code segments to perform the necessary tasks may be stored in a machine readable medium such as a storage medium.
- a code segment or machine-executable instruction may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a script, a class, or any combination of instructions, data structures, and/or program statements.
- a code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, and/or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.
- the methodologies may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein.
- Any machine-readable medium tangibly embodying instructions may be used in implementing the methodologies described herein.
- software codes may be stored in a memory.
- Memory may be implemented within the processor or external to the processor.
- the term “memory” refers to any type of long term, short term, volatile, nonvolatile, or other storage medium and is not to be limited to any particular type of memory or number of memories, or type of media upon which memory is stored.
- the term “storage medium” may represent one or more memories for storing data, including read only memory (ROM), random access memory (RAM), magnetic RAM, core memory, magnetic disk storage mediums, optical storage mediums, flash memory devices and/or other machine readable mediums for storing information.
- ROM read only memory
- RAM random access memory
- magnetic RAM magnetic RAM
- core memory magnetic disk storage mediums
- optical storage mediums flash memory devices and/or other machine readable mediums for storing information.
- machine-readable medium includes, but is not limited to portable or fixed storage devices, optical storage devices, and/or various other storage mediums capable of storing that contain or carry instruction(s) and/or data.
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- General Engineering & Computer Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Economics (AREA)
- Educational Administration (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Marketing (AREA)
- Development Economics (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Game Theory and Decision Science (AREA)
- Educational Technology (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Evolutionary Computation (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Multimedia (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Power Engineering (AREA)
- Biomedical Technology (AREA)
- Primary Health Care (AREA)
Abstract
Description
- This application is a non-provisional of and claims priority to U.S. Provisional Patent Application No. 62/559,433, entitled “DIGITAL CREDENTIAL PLATFORM,” filed Sep. 15, 2017, the entire contents of which are incorporated by reference herein for all purposes.
- Changes in computing technologies have provided individuals with additional options for obtaining and validating technical skills and proficiencies. Rather than attending traditional educational institutions and professional training courses, many individuals may now obtain their technical skills and proficiencies from alternative sources, such as structured or unstructured and asynchronous eLearning programs using distance learning technology, self-study research without any direct supervision, or various alternative technical learning, training, and testing entities. Although such advances in technologies and increasing globalization trends provide many more options for individuals to obtain technical skills and proficiencies, they also present challenges in publishing, verifying and tracking the sets of technical skills and proficiencies that these individuals have obtained. Many individuals and institutions no longer rely on physical certificates such as diplomas, transcripts, certification statements, and physical licenses, to verify the authenticity of an individual's proficiencies or qualifications. Instead, certain institutions may issue digital credentials (or digital badges) to qualifying individuals, and these digital credential earners may use the digital credentials to certify the skills or qualifications that the earner obtained vis-à-vis the institution.
- Various techniques are described herein for receiving data sources associated with digital credential receivers, and mapping the digital credential receivers to one or more data field data objects based on analyses of the verified data. In various embodiments, a digital credential platform server may receive requests from client devices identifying a digital credential receiver. In response to such requests, the digital credential platform server may retrieve data corresponding to a set of digital credential objects received by the digital credential receiver. The digital credential platform server also may retrieve one or more verified evaluation records, for example, from a secure third-party provider server, associated with the digital credential receiver. Verified evaluation records may include, for instance, results of personality or interest evaluations administered to the digital credential receiver, results of automated evaluations of the receiver's digital records of interactions, and the like, in order to further assess compatibility of the digital credential receiver with particular fields. Additionally, in some embodiments, the digital credential platform server may retrieve and/or determine a career phase associated with the digital credential receiver, which may be relevant to the mapping analysis.
- Additional techniques are described herein in which the digital credential platform server may analyze the various data sources associated with the digital credential receiver, in order to determine and calculate correlation scores between the digital credential receiver and various field data objects. In some embodiments, the digital credential platform server may use a combination of analyses and/or comparisons between the receiver data and related data retrieved from a plurality of different field data objects. For example, the digital credential objects (or digital badges) earned by the receiver may be compared to the corresponding capabilities, skills, certifications, etc., associated with different fields in a field data structure. Additionally, the receiver's verified evaluation records may be compared to the corresponding interest or personality data associated with the different fields. Further, in some cases, the current career phase of the digital credential receiver may be compared to corresponding career phase data associated with the different fields. Based on the various combination of analyses and comparisons, the platform server may calculate correlation scores for a plurality of different field data objects, and designate/transmit a subset of the fields that correlate most closely to the digital credential receiver.
- Various additional techniques described herein relate to both the retrieval of different types of credential receiver data and/or field data from different data sources, and/or to the analyses and correlation calculations between digital credential receivers and field data objects. For example, in some embodiments, the platform server may store and maintain expiration dates and/or half-lives for various types of data associated with digital credential receivers, such as the receiver's digital credentials, the receiver's verified evaluation data, and the receiver's career phase data. Expiration dates and/or half-lives may be used as a multiplier to weight the correlation calculations of credential receivers to field data objects. Additionally, in some embodiments, reverse analyses and calculations may be performed, in which a plurality of different digital credential receivers are selected and scored based on their correlation to a particular field data object representing an occupation or an individual job listing. Additionally, various embodiments described herein may include digital credentials issued to receivers based on the detection of specific interests and/or personality traits within the users, specific DNA traits and/or health-based traits, and/or for various other types or combinations of traits that may be detected for particular digital credential receivers, and these digital credentials may further be used in the correlation analyses. Still other aspects described herein may relate to a secure digital certification platform and/or service, to provide digital credential certification, verification, and security, in order to address problems associated with anonymous and/or unverified digital credentials and/or credential receivers.
-
FIG. 1 is a block diagram showing illustrating an example of a content distribution network. -
FIG. 2 is a block diagram illustrating a computer server and computing environment within a content distribution network. -
FIG. 3 is a block diagram illustrating an embodiment of one or more data store servers within a content distribution network. -
FIG. 4 is a block diagram illustrating an embodiment of one or more content management servers within a content distribution network. -
FIG. 5 is a block diagram illustrating the physical and logical components of a special-purpose computer device within a content distribution network. -
FIG. 6 is a block diagram illustrating an example computing environment for generating, managing, and tracking digital credential templates and digital credentials, according to one or more embodiments of the disclosure. -
FIG. 7 is an diagram illustrating an example computing environment for executing and monitoring physical simulations within a digital credential system, according to one or more embodiments of the disclosure. -
FIG. 8 is a flow diagram illustrating an example process of executing and monitoring physical simulations for generation of digital credentials, according to one or more embodiments of the disclosure. -
FIG. 9A is an diagram illustrating a computer terminal-based system for sensor-based monitoring and generation of digital credentials, according to one or more embodiments of the disclosure. -
FIG. 9B is an diagram illustrating a physical environment-based system for sensor-based monitoring, and generation of digital credentials, according to one or more embodiments of the disclosure. -
FIG. 10 is a flow diagram illustrating an example process of generating and issuing digital credentials in a sensor-monitored environment, according to one or more embodiments of the disclosure. -
FIG. 11 is an diagram illustrating an example computing environment for analyzing sensor-based activity monitoring within a digital credential system, according to one or more embodiments of the disclosure. -
FIG. 12 is a flow diagram illustrating an example process of generating digital credentials and tracking the corresponding activities in a sensor-monitored environment, according to one or more embodiments of the disclosure. -
FIG. 13 is a flow diagram illustrating an example process of analyzing activities in a sensor-monitored environment to determine digital credential expiration and/or recertification times, according to one or more embodiments of the disclosure. -
FIG. 14 is an diagram illustrating an example computing environment for generating and analyzing digital credentials using received sensor monitoring data, according to one or more embodiments of the disclosure. -
FIG. 15 is a flow diagram illustrating an example process of generating and storing digital credentials with associated data from sensor-monitored environments, according to one or more embodiments of the disclosure. -
FIGS. 16A-16B are flow diagrams illustrating example processes of retrieving sensor data associated with issued digital credentials, and generating additional and/or updated digital credentials based on the retrieved sensor data, according to one or more embodiments of the disclosure. -
FIGS. 17A-17B are diagrams illustrating facial recognition and analysis functionality performed during digital credential generation and analyses processes within sensor-monitored environments, according to one or more embodiments of the disclosure. -
FIG. 18 is a flow diagram illustrating an example process of generating and storing digital credentials with associated user feedback data from sensor-monitored environments, according to one or more embodiments of the disclosure. -
FIG. 19 is a block diagram illustrating an example system for analyzing and mapping digital credential receivers to field data objects, according to one or more embodiments of the disclosure. -
FIG. 20 is a flow diagram illustrating an example process of generating notifications based on determined correlations between digital credential receivers and field data objects, according to one or more embodiments of the disclosure. - In the appended figures, similar components and/or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label by a dash and a second label that distinguishes among the similar components. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.
- The ensuing description provides illustrative embodiment(s) only and is not intended to limit the scope, applicability or configuration of the disclosure. Rather, the ensuing description of the illustrative embodiment(s) will provide those skilled in the art with an enabling description for implementing a preferred exemplary embodiment. It is understood that various changes can be made in the function and arrangement of elements without departing from the spirit and scope as set forth in the appended claims.
- Various techniques (e.g., systems, methods, computer-program products tangibly embodied in a non-transitory machine-readable storage medium, etc.) are described herein for receiving data sources associated with digital credential receivers, and mapping the digital credential receivers to one or more data field data objects based on analyses of the verified data. In various embodiments, a digital credential platform server may receive requests from client devices identifying a digital credential receiver. In response to such requests, the digital credential platform server may retrieve data corresponding to a set of digital credential objects received by the digital credential receiver. The digital credential platform server also may retrieve one or more verified evaluation records, for example, from a secure third-party provider server, associated with the digital credential receiver. Verified evaluation records may include, for instance, results of personality or interest evaluations administered to the digital credential receiver, results of automated evaluations of the receiver's digital records of interactions, and the like, in order to further assess compatibility of the digital credential receiver with particular fields. Additionally, in some embodiments, the digital credential platform server may retrieve and/or determine a career phase associated with the digital credential receiver, which may be relevant to the mapping analysis.
- Additional techniques are described herein in which the digital credential platform server may analyze the various data sources associated with the digital credential receiver, in order to determine and calculate correlation scores between the digital credential receiver and various field data objects. In some embodiments, the digital credential platform server may use a combination of analyses and/or comparisons between the receiver data and related data retrieved from a plurality of different field data objects. For example, the digital credential objects (or digital badges) earned by the receiver may be compared to the corresponding capabilities, skills, certifications, etc., associated with different fields in a field data structure. Additionally, the receiver's verified evaluation records may be compared to the corresponding interest or personality data associated with the different fields. Further, in some cases, the current career phase of the digital credential receiver may be compared to corresponding career phase data associated with the different fields. Based on the various combination of analyses and comparisons, the platform server may calculate correlation scores for a plurality of different field data objects, and designate/transmit a subset of the fields that correlate most closely to the digital credential receiver.
- Various additional techniques described herein relate to both the retrieval of different types of credential receiver data and/or field data from different data sources, and/or to the analyses and correlation calculations between digital credential receivers and field data objects. For example, in some embodiments, the platform server may store and maintain expiration dates and/or half-lives for various types of data associated with digital credential receivers, such as the receiver's digital credentials, the receiver's verified evaluation data, and the receiver's career phase data. Expiration dates and/or half-lives may be used as a multiplier to weight the correlation calculations of credential receivers to field data objects. Additionally, in some embodiments, reverse analyses and calculations may be performed, in which a plurality of different digital credential receivers are selected and scored based on their correlation to a particular field data object representing an occupation or an individual job listing. Additionally, various embodiments described herein may include digital credentials issued to receivers based on the detection of specific interests and/or personality traits within the users, specific DNA traits and/or health-based traits, and/or for various other types or combinations of traits that may be detected for particular digital credential receivers, and these digital credentials may further be used in the correlation analyses. Still other aspects described herein may relate to a secure digital certification platform and/or service, to provide digital credential certification, verification, and security, in order to address problems associated with anonymous and/or unverified digital credentials and/or credential receivers.
- With reference now to
FIG. 1 , a block diagram is shown illustrating various components of a content distribution network (CDN) 100 which implements and supports certain embodiments and features described herein. Content distribution network 100 may include one or morecontent management servers 102. As discussed below in more detail,content management servers 102 may be any desired type of server including, for example, a rack server, a tower server, a miniature server, a blade server, a mini rack server, a mobile server, an ultra-dense server, a super server, or the like, and may include various hardware components, for example, a motherboard, a processing units, memory systems, hard drives, network interfaces, power supplies, etc.Content management server 102 may include one or more server farms, clusters, or any other appropriate arrangement and/or combination or computer servers.Content management server 102 may act according to stored instructions located in a memory subsystem of theserver 102, and may run an operating system, including any commercially available server operating system and/or any other operating systems discussed herein. - The content distribution network 100 may include one or more
data store servers 104, such as database servers and file-based storage systems.Data stores 104 may comprise stored data relevant to the functions of the content distribution network 100. Illustrative examples ofdata stores 104 that may be maintained in certain embodiments of the content distribution network 100 are described below in reference toFIG. 3 . In some embodiments, multiple data stores may reside on asingle server 104, either using the same storage components ofserver 104 or using different physical storage components to assure data security and integrity between data stores. In other embodiments, each data store may have a separate dedicateddata store server 104. - Content distribution network 100 also may include one or
more user devices 106 and/orsupervisor devices 110.User devices 106 andsupervisor devices 110 may display content received via the content distribution network 100, and may support various types of user interactions with the content.User devices 106 andsupervisor devices 110 may include mobile devices such as smartphones, tablet computers, personal digital assistants, and wearable computing devices. Such mobile devices may run a variety of mobile operating systems, and may be enabled for Internet, e-mail, short message service (SMS), Bluetooth®, mobile radio-frequency identification (M-RFID), and/or other communication protocols.Other user devices 106 andsupervisor devices 110 may be general purpose personal computers or special-purpose computing devices including, by way of example, personal computers, laptop computers, workstation computers, projection devices, and interactive room display systems. Additionally,user devices 106 andsupervisor devices 110 may be any other electronic devices, such as thin-client computers, Internet-enabled gaming systems, business or home appliances, and/or personal messaging devices, capable of communicating over network(s) 120. - In different contexts of content distribution networks 100,
user devices 106 andsupervisor devices 110 may correspond to different types of specialized devices, for example, student devices and teacher devices in an educational network, employee devices and presentation devices in a company network, different gaming devices in a gaming network, etc. In some embodiments,user devices 106 andsupervisor devices 110 may operate in the samephysical location 107, such as a classroom or conference room. In such cases, the devices may contain components that support direct communications with other nearby devices, such as a wireless transceivers and wireless communications interfaces, Ethernet sockets or other Local Area Network (LAN) interfaces, etc. In other implementations, theuser devices 106 andsupervisor devices 110 need not be used at thesame location 107, but may be used in remote geographic locations in which eachuser device 106 andsupervisor device 110 may use security features and/or specialized hardware (e.g., hardware-accelerated SSL and HTTPS, WS-Security, firewalls, etc.) to communicate with thecontent management server 102 and/or other remotely locateduser devices 106. Additionally,different user devices 106 andsupervisor devices 110 may be assigned different designated roles, such as presenter devices, teacher devices, administrator devices, or the like, and in such cases the different devices may be provided with additional hardware and/or software components to provide content and support user capabilities not available to the other devices. - The content distribution network 100 also may include a
privacy server 108 that maintains private user information at theprivacy server 108 while using applications or services hosted on other servers. For example, theprivacy server 108 may be used to maintain private data of a user within one jurisdiction even though the user is accessing an application hosted on a server (e.g., the content management server 102) located outside the jurisdiction. In such cases, theprivacy server 108 may intercept communications between auser device 106 orsupervisor device 110 and other devices that include private user information. Theprivacy server 108 may create a token or identifier that does not disclose the private information and may use the token or identifier when communicating with the other servers and systems, instead of using the user's private information. - As illustrated in
FIG. 1 , thecontent management server 102 may be in communication with one or more additional servers, such as acontent server 112, auser data server 112, and/or anadministrator server 116. Each of these servers may include some or all of the same physical and logical components as the content management server(s) 102, and in some cases, the hardware and software components of these servers 112-116 may be incorporated into the content management server(s) 102, rather than being implemented as separate computer servers. -
Content server 112 may include hardware and software components to generate, store, and maintain the content resources for distribution touser devices 106 and other devices in the network 100. For example, in content distribution networks 100 used for professional training and educational purposes,content server 112 may include data stores of training materials, presentations, interactive programs and simulations, course models, course outlines, and various training interfaces that correspond to different materials and/or different types ofuser devices 106. In content distribution networks 100 used for media distribution, interactive gaming, and the like, acontent server 112 may include media content files such as music, movies, television programming, games, and advertisements. - User data server 114 may include hardware and software components that store and process data for multiple users relating to each user's activities and usage of the content distribution network 100. For example, the
content management server 102 may record and track each user's system usage, including theiruser device 106, content resources accessed, and interactions withother user devices 106. This data may be stored and processed by the user data server 114, to support user tracking and analysis features. For instance, in the professional training and educational contexts, the user data server 114 may store and analyze each user's training materials viewed, presentations attended, courses completed, interactions, evaluation results, and the like. The user data server 114 may also include a repository for user-generated material, such as evaluations and tests completed by users, and documents and assignments prepared by users. In the context of media distribution and interactive gaming the user data server 114 may store and process resource access data for multiple users (e.g., content titles accessed, access times, data usage amounts, gaming histories, user devices and device types, etc.). -
Administrator server 116 may include hardware and software components to initiate various administrative functions at thecontent management server 102 and other components within the content distribution network 100. For example, theadministrator server 116 may monitor device status and performance for the various servers, data stores, and/oruser devices 106 in the content distribution network 100. When necessary, theadministrator server 116 may add or remove devices from the network 100, and perform device maintenance such as providing software updates to the devices in the network 100. Various administrative tools on theadministrator server 116 may allow authorized users to set user access permissions to various content resources, monitor resource usage by users anddevices 106, and perform analyses and generate reports on specific network users and/or devices (e.g., resource usage tracking reports, training evaluations, etc.). - The content distribution network 100 may include one or
more communication networks 120. Although only asingle network 120 is identified inFIG. 1 , the content distribution network 100 may include any number of different communication networks between any of the computer servers and devices shown inFIG. 1 and/or other devices described herein.Communication networks 120 may enable communication between the various computing devices, servers, and other components of the content distribution network 100. As discussed below, various implementations of content distribution networks 100 may employ different types ofnetworks 120, for example, computer networks, telecommunications networks, wireless networks, and/or any combination of these and/or other networks. - With reference to
FIG. 2 , an illustrative distributed computing environment 200 is shown including acomputer server 202, fourclient computing devices 206, and other components that may implement certain embodiments and features described herein. In some embodiments, theserver 202 may correspond to thecontent management server 102 discussed above inFIG. 1 , and theclient computing devices 206 may correspond to theuser devices 106. However, the computing environment 200 illustrated inFIG. 2 may correspond to any other combination of devices and servers configured to implement a client-server model or other distributed computing architecture. -
Client devices 206 may be configured to receive and execute client applications over one ormore networks 220. Such client applications may be web browser based applications and/or standalone software applications, such as mobile device applications.Server 202 may be communicatively coupled with theclient devices 206 via one ormore communication networks 220.Client devices 206 may receive client applications fromserver 202 or from other application providers (e.g., public or private application stores).Server 202 may be configured to run one or more server software applications or services, for example, web-based or cloud-based services, to support content distribution and interaction withclient devices 206. Users operatingclient devices 206 may in turn utilize one or more client applications (e.g., virtual client applications) to interact withserver 202 to utilize the services provided by these components. - Various different subsystems and/or components 204 may be implemented on
server 202. Users operating theclient devices 206 may initiate one or more client applications to use services provided by these subsystems and components. The subsystems and components within theserver 202 andclient devices 206 may be implemented in hardware, firmware, software, or combinations thereof. Various different system configurations are possible in different distributed computing systems 200 and content distribution networks 100. The embodiment shown inFIG. 2 is thus one example of a distributed computing system and is not intended to be limiting. - Although exemplary computing environment 200 is shown with four
client computing devices 206, any number of client computing devices may be supported. Other devices, such as specialized sensor devices, etc., may interact withclient devices 206 and/orserver 202. - As shown in
FIG. 2 , various security andintegration components 208 may be used to transmit, receive, and manage communications between theserver 202 anduser devices 206 over one ormore communication networks 220. The security andintegration components 208 may include separate servers, such as web servers and/or authentication servers, and/or specialized networking components, such as firewalls, routers, gateways, load balancers, and the like. In some cases, the security andintegration components 208 may correspond to a set of dedicated hardware and/or software operating at the same physical location and under the control of same entities asserver 202. For example,components 208 may include one or more dedicated web servers and network hardware in a datacenter or a cloud infrastructure. In other examples, the security andintegration components 208 may correspond to separate hardware and software components which may be operated at a separate physical location and/or by a separate entity. - Security and
integration components 208 may implement various security features for data transmission and storage, such as authenticating users and restricting access to unknown or unauthorized users. In various implementations, security andintegration components 208 may provide, for example, a file-based integration scheme or a service-based integration scheme for transmitting data between the various devices in the content distribution network 100. Security andintegration components 208 also may use secure data transmission protocols and/or encryption for data transfers, for example, File Transfer Protocol (FTP), Secure File Transfer Protocol (SFTP), and/or Pretty Good Privacy (PGP) encryption. - In some embodiments, one or more web services may be implemented within the security and
integration components 208 and/or elsewhere within the content distribution network 100. Such web services, including cross-domain and/or cross-platform web services, may be developed for enterprise use in accordance with various web service standards, such as RESTful web services (i.e., services based on the Representation State Transfer (REST) architectural style and constraints), and/or web services designed in accordance with the Web Service Interoperability (WS-I) guidelines. Some web services may use the Secure Sockets Layer (SSL) or Transport Layer Security (TLS) protocol to provide secure connections between theserver 202 anduser devices 206. SSL or TLS may use HTTP or HTTPS to provide authentication and confidentiality. In other examples, web services may be implemented using REST over HTTPS with the OAuth open standard for authentication, or using the WS-Security standard which provides for secure SOAP messages using XML, encryption. In other examples, the security andintegration components 208 may include specialized hardware for providing secure web services. For example, security andintegration components 208 may include secure network appliances having built-in features such as hardware-accelerated SSL and HTTPS, WS-Security, and firewalls. Such specialized hardware may be installed and configured in front of any web servers, so that any external devices may communicate directly with the specialized hardware. - Communication network(s) 220 may be any type of network familiar to those skilled in the art that can support data communications using any of a variety of commercially-available protocols, including without limitation, TCP/IP (transmission control protocol/Internet protocol), SNA (systems network architecture), IPX (Internet packet exchange), Secure Sockets Layer (SSL) or Transport Layer Security (TLS) protocols, Hyper Text Transfer Protocol (HTTP) and Secure Hyper Text Transfer Protocol (HTTPS), Bluetooth®, Near Field Communication (NFC), and the like. Merely by way of example, network(s) 220 may be local area networks (LAN), such as one based on Ethernet, Token-Ring and/or the like. Network(s) 220 also may be wide-area networks, such as the Internet.
Networks 220 may include telecommunication networks such as a public switched telephone networks (PSTNs), or virtual networks such as an intranet or an extranet. Infrared and wireless networks (e.g., using the Institute of Electrical and Electronics (IEFF) 802.11 protocol suite or other wireless protocols) also may be included innetworks 220. - Computing environment 200 also may include one or
more data stores 210 and/or back-end servers 212. In certain examples, thedata stores 210 may correspond to data store server(s) 104 discussed above inFIG. 1 , and back-end servers 212 may correspond to the various back-end servers 112-116.Data stores 210 andservers 212 may reside in the same datacenter or may operate at a remote location fromserver 202. In some cases, one ormore data stores 210 may reside on a non-transitory storage medium within theserver 202.Other data stores 210 and back-end servers 212 may be remote fromserver 202 and configured to communicate withserver 202 via one ormore networks 220. In certain embodiments,data stores 210 and back-end servers 212 may reside in a storage-area network (SAN), or may use storage-as-a-service (STaaS) architectural model. - With reference to
FIG. 3 , an illustrative set of data stores and/or data store servers is shown, corresponding to thedata store servers 104 of the content distribution network 100 discussed above inFIG. 1 . One or more individual data stores 301-309 may reside in storage on a single computer server 104 (or a single server farm or cluster) under the control of a single entity, or may reside on separate servers operated by different entities and/or at remote locations. In some embodiments, data stores 301-309 may be accessed by thecontent management server 102 and/or other devices and servers within the network 100 (e.g.,user devices 106,supervisor devices 110,administrator servers 116, etc.). Access to one or more of the data stores 301-309 may be limited or denied based on the processes, user credentials, and/or devices attempting to interact with the data store. - The paragraphs below describe examples of specific data stores that may be implemented within some embodiments of a content distribution network 100. It should be understood that the below descriptions of data stores 301-309, including their functionality and types of data stored therein, are illustrative and non-limiting. Data stores server architecture, design, and the execution of specific data stores 301-309 may depend on the context, size, and functional requirements of a content distribution network 100. For example, in content distribution systems 100 used for professional training and educational purposes, separate databases or file-based storage systems may be implemented in data store server(s) 104 to store trainee and/or student data, trainer and/or professor data, training module data and content descriptions, training results, evaluation data, and the like. In contrast, in content distribution systems 100 used for media distribution from content providers to subscribers, separate data stores may be implemented in data stores server(s) 104 to store listings of available content titles and descriptions, content title usage statistics, subscriber profiles, account data, payment data, network usage statistics, etc.
- A user profile data store 301 may include information relating to the end users within the content distribution network 100. This information may include user characteristics such as the user names, access credentials (e.g., login and passwords), user preferences, and information relating to any previous user interactions within the content distribution network 100 (e.g., requested content, posted content, content modules completed, training scores or evaluations, other associated users, etc.).
- An accounts
data store 302 may generate and store account data for different users in various roles within the content distribution network 100. For example, accounts may be created in anaccounts data store 302 for individual end users, supervisors, administrator users, and entities such as companies or educational institutions. Account data may include account types, current account status, account characteristics, and any parameters, limits, restrictions associated with the accounts. - A content
library data store 303 may include information describing the individual content items (or content resources) available via the content distribution network 100. In some embodiments, thelibrary data store 303 may include metadata, properties, and other characteristics associated with the content resources stored in thecontent server 112. Such data may identify one or more aspects or content attributes of the associated content resources, for example, subject matter, access level, or skill level of the content resources, license attributes of the content resources (e.g., any limitations and/or restrictions on the licensable use and/or distribution of the content resource), price attributes of the content resources (e.g., a price and/or price structure for determining a payment amount for use or distribution of the content resource), rating attributes for the content resources (e.g., data indicating the evaluation or effectiveness of the content resource), and the like. In some embodiments, thelibrary data store 303 may be configured to allow updating of content metadata or properties, and to allow the addition and/or removal of information relating to the content resources. For example, content relationships may be implemented as graph structures, which may be stored in thelibrary data store 303 or in an additional store for use by selection algorithms along with the other metadata. - A
pricing data store 304 may include pricing information and/or pricing structures for determining payment amounts for providing access to the content distribution network 100 and/or the individual content resources within the network 100. In some cases, pricing may be determined based on a user's access to the content distribution network 100, for example, a time-based subscription fee, or pricing based on network usage and. In other cases, pricing may be tied to specific content resources. Certain content resources may have associated pricing information, whereas other pricing determinations may be based on the resources accessed, the profiles and/or accounts of the user, and the desired level of access (e.g., duration of access, network speed, etc.). Additionally, thepricing data store 304 may include information relating to compilation pricing for groups of content resources, such as group prices and/or price structures for groupings of resources. - A
license data store 305 may include information relating to licenses and/or licensing of the content resources within the content distribution network 100. For example, thelicense data store 305 may identify licenses and licensing terms for individual content resources and/or compilations of content resources in thecontent server 112, the rights holders for the content resources, and/or common or large-scale right holder information such as contact information for rights holders of content not included in thecontent server 112. - A content access data store 306 may include access rights and security information for the content distribution network 100 and specific content resources. For example, the content access data store 306 may include login information (e.g., user identifiers, logins, passwords, etc.) that can be verified during user login attempts to the network 100. The content access data store 306 also may be used to store assigned user roles and/or user levels of access. For example, a user's access level may correspond to the sets of content resources and/or the client or server applications that the user is permitted to access. Certain users may be permitted or denied access to certain applications and resources based on their subscription level, training program, course/grade level, etc. Certain users may have supervisory access over one or more end users, allowing the supervisor to access all or portions of the end user's content, activities, evaluations, etc. Additionally, certain users may have administrative access over some users and/or some applications in the content management network 100, allowing such users to add and remove user accounts, modify user access permissions, perform maintenance updates on software and servers, etc.
- A source data store 307 may include information relating to the source of the content resources available via the content distribution network. For example, a source data store 307 may identify the authors and originating devices of content resources, previous pieces of data and/or groups of data originating from the same authors or originating devices, and the like.
- An evaluation data store 308 may include information used to direct the evaluation of users and content resources in the content management network 100. In some embodiments, the evaluation data store 308 may contain, for example, the analysis criteria and the analysis guidelines for evaluating users (e.g., trainees/students, gaming users, media content consumers, etc.) and/or for evaluating the content resources in the network 100. The evaluation data store 308 also may include information relating to evaluation processing tasks, for example, the identification of users and
user devices 106 that have received certain content resources or accessed certain applications, the status of evaluations or evaluation histories for content resources, users, or applications, and the like. Evaluation criteria may be stored in the evaluation data store 308 including data and/or instructions in the form of one or several electronic rubrics or scoring guides for use in the evaluation of the content, users, or applications. The evaluation data store 308 also may include past evaluations and/or evaluation analyses for users, content, and applications, including relative rankings, characterizations, explanations, and the like. - In addition to the illustrative data stores described above, data store server(s) 104 (e.g., database servers, file-based storage servers, etc.) may include one or more
external data aggregators 309.External data aggregators 309 may include third-party data sources accessible to the content management network 100, but not maintained by the content management network 100.External data aggregators 309 may include any electronic information source relating to the users, content resources, or applications of the content distribution network 100. For example,external data aggregators 309 may be third-party data stores containing demographic data, education related data, consumer sales data, health related data, and the like. Illustrativeexternal data aggregators 309 may include, for example, social networking web servers, public records data stores, learning management systems, educational institution servers, business servers, consumer sales data stores, medical record data stores, etc. Data retrieved from variousexternal data aggregators 309 may be used to verify and update user account information, suggest user content, and perform user and content evaluations. - With reference now to
FIG. 4 , a block diagram is shown illustrating an embodiment of one or morecontent management servers 102 within a content distribution network 100. As discussed above, content management server(s) 102 may include various server hardware and software components that manage the content resources within the content distribution network 100 and provide interactive and adaptive content to users onvarious user devices 106. For example, content management server(s) 102 may provide instructions to and receive information from the other devices within the content distribution network 100, in order to manage and transmit content resources, user data, and server or client applications executing within the network 100. - A
content management server 102 may include acontent customization system 402. Thecontent customization system 402 may be implemented using dedicated hardware within the content distribution network 100 (e.g., a content customization server 402), or using designated hardware and software resources within a sharedcontent management server 102. In some embodiments, thecontent customization system 402 may adjust the selection and adaptive capabilities of content resources to match the needs and desires of the users receiving the content. For example, thecontent customization system 402 may query various data stores andservers 104 to retrieve user information, such as user preferences and characteristics (e.g., from a user profile data store 301), user access restrictions to content recourses (e.g., from a content access data store 306), previous user results and content evaluations (e.g., from an evaluation data store 308), and the like. Based on the retrieved information fromdata stores 104 and other data sources, thecontent customization system 402 may modify content resources for individual users. - A
content management server 102 also may include auser management system 404. Theuser management system 404 may be implemented using dedicated hardware within the content distribution network 100 (e.g., a user management server 404), or using designated hardware and software resources within a sharedcontent management server 102. In some embodiments, theuser management system 404 may monitor the progress of users through various types of content resources and groups, such as media compilations, courses or curriculums in training or educational contexts, interactive gaming environments, and the like. For example, theuser management system 404 may query one or more databases and/ordata store servers 104 to retrieve user data such as associated content compilations or programs, content completion status, user goals, results, and the like. - A
content management server 102 also may include anevaluation system 406. Theevaluation system 406 may be implemented using dedicated hardware within the content distribution network 100 (e.g., an evaluation server 406), or using designated hardware and software resources within a sharedcontent management server 102. Theevaluation system 406 may be configured to receive and analyze information fromuser devices 106. For example, various ratings of content resources submitted by users may be compiled and analyzed, and then stored in a data store (e.g., a contentlibrary data store 303 and/or evaluation data store 308) associated with the content. In some embodiments, theevaluation server 406 may analyze the information to determine the effectiveness or appropriateness of content resources with, for example, a subject matter, an age group, a skill level, or the like. In some embodiments, theevaluation system 406 may provide updates to thecontent customization system 402 or theuser management system 404, with the attributes of one or more content resources or groups of resources within the network 100. Theevaluation system 406 also may receive and analyze user evaluation data fromuser devices 106,supervisor devices 110, andadministrator servers 116, etc. For instance,evaluation system 406 may receive, aggregate, and analyze user evaluation data for different types of users (e.g., end users, supervisors, administrators, etc.) in different contexts (e.g., media consumer ratings, trainee or student comprehension levels, teacher effectiveness levels, gamer skill levels, etc.). - A
content management server 102 also may include acontent delivery system 408. Thecontent delivery system 408 may be implemented using dedicated hardware within the content distribution network 100 (e.g., a content delivery server 408), or using designated hardware and software resources within a sharedcontent management server 102. Thecontent delivery system 408 may receive content resources from thecontent customization system 402 and/or from theuser management system 404, and provide the resources touser devices 106. Thecontent delivery system 408 may determine the appropriate presentation format for the content resources based on the user characteristics and preferences, and/or the device capabilities ofuser devices 106. If needed, thecontent delivery system 408 may convert the content resources to the appropriate presentation format and/or compress the content before transmission. In some embodiments, thecontent delivery system 408 may also determine the appropriate transmission media and communication protocols for transmission of the content resources. - In some embodiments, the
content delivery system 408 may include specialized security andintegration hardware 410, along with corresponding software components to implement the appropriate security features content transmission and storage, to provide the supported network and client access models, and to support the performance and scalability requirements of the network 100. The security andintegration layer 410 may include some or all of the security andintegration components 208 discussed above inFIG. 2 , and may control the transmission of content resources and other data, as well as the receipt of requests and content interactions, to and from theuser devices 106,supervisor devices 110,administrative servers 116, and other devices in the network 100. - With reference now to
FIG. 5 , a block diagram of an illustrative computer system is shown. Thesystem 500 may correspond to any of the computing devices or servers of the content distribution network 100 described above, or any other computing devices described herein. In this example,computer system 500 includes processingunits 504 that communicate with a number of peripheral subsystems via abus subsystem 502. These peripheral subsystems include, for example, astorage subsystem 510, an I/O subsystem 526, and acommunications subsystem 532. -
Bus subsystem 502 provides a mechanism for letting the various components and subsystems ofcomputer system 500 communicate with each other as intended. Althoughbus subsystem 502 is shown schematically as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses.Bus subsystem 502 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. Such architectures may include, for example, an Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus, which can be implemented as a Mezzanine bus manufactured to the IEEE P1386.1 standard. -
Processing unit 504, which may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation ofcomputer system 500. One or more processors, including single core and/or multicore processors, may be included inprocessing unit 504. As shown in the figure, processingunit 504 may be implemented as one or moreindependent processing units 506 and/or 508 with single or multicore processors and processor caches included in each processing unit. In other embodiments, processingunit 504 may also be implemented as a quad-core processing unit or larger multicore designs (e.g., hexa-core processors, octo-core processors, ten-core processors, or greater. -
Processing unit 504 may execute a variety of software processes embodied in program code, and may maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can be resident in processor(s) 504 and/or instorage subsystem 510. In some embodiments,computer system 500 may include one or more specialized processors, such as digital signal processors (DSPs), outboard processors, graphics processors, application-specific processors, and/or the like. - I/
O subsystem 526 may includedevice controllers 528 for one or more user interface input devices and/or userinterface output devices 530. User interface input andoutput devices 530 may be integral with the computer system 500 (e.g., integrated audio/video systems, and/or touchscreen displays), or may be separate peripheral devices which are attachable/detachable from thecomputer system 500. -
Input devices 530 may include a keyboard, pointing devices such as a mouse or trackball, a touchpad or touch screen incorporated into a display, a scroll wheel, a click wheel, a dial, a button, a switch, a keypad, audio input devices with voice command recognition systems, microphones, and other types of input devices.Input devices 530 may also include three dimensional (3D) mice, joysticks or pointing sticks, gamepads and graphic tablets, and audio/visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode reader 3D scanners, 3D printers, laser rangefinders, and eye gaze tracking devices.Additional input devices 530 may include, for example, motion sensing and/or gesture recognition devices that enable users to control and interact with an input device through a natural user interface using gestures and spoken commands, eye gesture recognition devices that detect eye activity from users and transform the eye gestures as input into an input device, voice recognition sensing devices that enable users to interact with voice recognition systems through voice commands, medical imaging input devices, MIDI keyboards, digital musical instruments, and the like. -
Output devices 530 may include one or more display subsystems, indicator lights, or non-visual displays such as audio output devices, etc. Display subsystems may include, for example, cathode ray tube (CRT) displays, flat-panel devices, such as those using a liquid crystal display (LCD) or plasma display, light-emitting diode (LED) displays, projection devices, touch screens, and the like. In general, use of the term “output device” is intended to include all possible types of devices and mechanisms for outputting information fromcomputer system 500 to a user or other computer. For example,output devices 530 may include, without limitation, a variety of display devices that visually convey text, graphics and audio/video information such as monitors, printers, speakers, headphones, automotive navigation systems, plotters, voice output devices, and modems. -
Computer system 500 may comprise one ormore storage subsystems 510, comprising hardware and software components used for storing data and program instructions, such assystem memory 518 and computer-readable storage media 516. Thesystem memory 518 and/or computer-readable storage media 516 may store program instructions that are loadable and executable on processingunits 504, as well as data generated during the execution of these programs. - Depending on the configuration and type of
computer system 500, system memory 318 may be stored in volatile memory (such as random access memory (RAM) 512) and/or in non-volatile storage drives 514 (such as read-only memory (ROM), flash memory, etc.) The RAM 512 may contain data and/or program modules that are immediately accessible to and/or presently being operated and executed by processingunits 504. In some implementations,system memory 518 may include multiple different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input/output system (BIOS), containing the basic routines that help to transfer information between elements withincomputer system 500, such as during start-up, may typically be stored in the non-volatile storage drives 514. By way of example, and not limitation,system memory 518 may includeapplication programs 520, such as client applications, Web browsers, mid-tier applications, server applications, etc.,program data 522, and anoperating system 524. -
Storage subsystem 510 also may provide one or more tangible computer-readable storage media 516 for storing the basic programming and data constructs that provide the functionality of some embodiments. Software (programs, code modules, instructions) that when executed by a processor provide the functionality described herein may be stored instorage subsystem 510. These software modules or instructions may be executed by processingunits 504.Storage subsystem 510 may also provide a repository for storing data used in accordance with the present invention - Storage subsystem 300 may also include a computer-readable storage media reader that can further be connected to computer-
readable storage media 516. Together and, optionally, in combination withsystem memory 518, computer-readable storage media 516 may comprehensively represent remote, local, fixed, and/or removable storage devices plus storage media for temporarily and/or more permanently containing, storing, transmitting, and retrieving computer-readable information. - Computer-
readable storage media 516 containing program code, or portions of program code, may include any appropriate media known or used in the art, including storage media and communication media, such as but not limited to, volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage and/or transmission of information. This can include tangible computer-readable storage media such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other tangible computer readable media. This can also include nontangible computer-readable media, such as data signals, data transmissions, or any other medium which can be used to transmit the desired information and which can be accessed bycomputer system 500. - By way of example, computer-
readable storage media 516 may include a hard disk drive that reads from or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive that reads from or writes to a removable, nonvolatile magnetic disk, and an optical disk drive that reads from or writes to a removable, nonvolatile optical disk such as a CD ROM, DVD, and Blu-Ray® disk, or other optical media. Computer-readable storage media 516 may include, but is not limited to, Zip® drives, flash memory cards, universal serial bus (USB) flash drives, secure digital (SD) cards, DVD disks, digital video tape, and the like. Computer-readable storage media 516 may also include, solid-state drives (SSD) based on non-volatile memory such as flash-memory based SSDs, enterprise flash drives, solid state ROM, and the like, SSDs based on volatile memory such as solid state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory based SSDs. The disk drives and their associated computer-readable media may provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data forcomputer system 500. - Communications subsystem 532 may provide a communication interface from
computer system 500 and external computing devices via one or more communication networks, including local area networks (LANs), wide area networks (WANs) (e.g., the Internet), and various wireless telecommunications networks. As illustrated inFIG. 5 , thecommunications subsystem 532 may include, for example, one or more network interface controllers (NICs) 534, such as Ethernet cards, Asynchronous Transfer Mode NICs, Token Ring NICs, and the like, as well as one or more wireless communications interfaces 536, such as wireless network interface controllers (WNICs), wireless network adapters, and the like. Additionally and/or alternatively, thecommunications subsystem 532 may include one or more modems (telephone, satellite, cable, ISDN), synchronous or asynchronous digital subscriber line (DSL) units, FireWire® interfaces, USB® interfaces, and the like. Communications subsystem 536 also may include radio frequency (RF) transceiver components for accessing wireless voice and/or data networks (e.g., using cellular telephone technology, advanced data network technology, such as 3G, 4G or EDGE (enhanced data rates for global evolution), WiFi (IEEE 802.11 family standards, or other mobile communication technologies, or any combination thereof), global positioning system (GPS) receiver components, and/or other components. - The various physical components of the
communications subsystem 532 may be detachable components coupled to thecomputer system 500 via a computer network, a FireWire® bus, or the like, and/or may be physically integrated onto a motherboard of thecomputer system 500. Communications subsystem 532 also may be implemented in whole or in part by software. - In some embodiments,
communications subsystem 532 may also receive input communication in the form of structured and/or unstructured data feeds, event streams, event updates, and the like, on behalf of one or more users who may use oraccess computer system 500. For example,communications subsystem 532 may be configured to receive data feeds in real-time from users of social networks and/or other communication services, web feeds such as Rich Site Summary (RSS) feeds, and/or real-time updates from one or more third party information sources (e.g., data aggregators 309). Additionally,communications subsystem 532 may be configured to receive data in the form of continuous data streams, which may include event streams of real-time events and/or event updates (e.g., sensor data applications, financial tickers, network performance measuring tools, clickstream analysis tools, automobile traffic monitoring, etc.). Communications subsystem 532 may output such structured and/or unstructured data feeds, event streams, event updates, and the like to one ormore data stores 104 that may be in communication with one or more streaming data source computers coupled tocomputer system 500. - Due to the ever-changing nature of computers and networks, the description of
computer system 500 depicted in the figure is intended only as a specific example. Many other configurations having more or fewer components than the system depicted in the figure are possible. For example, customized hardware might also be used and/or particular elements might be implemented in hardware, firmware, software, or a combination. Further, connection to other computing devices, such as network input/output devices, may be employed. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will appreciate other ways and/or methods to implement the various embodiments. - With reference now to
FIG. 6 , a block diagram is shown illustrating an example of a digital credential management system 600 for generating, managing, and tracking digital credential templates and digital credentials. As shown in this example, a digital credential management system 600 may include a digitalcredential platform server 610 configured to communicate with various other digital credential systems 620-680. As discussed below, the digitalcredential platform server 610 may receive and store digital credential templates from various digital credentialtemplate owner systems 620.Systems 620 may correspond to the computer servers and/or devices of educational institutions or professional training organizations, which may have the primary responsibility for defining a digital credential template and controlling the content and requirements for users to receive a digital credential from the organization. The digital credential management system 600 may include one or more digitalcredential issuer systems 630. As discussed below, eachissuer system 630 may communicate with the platform server to request and receive access to issue digital credentials based on specific digital credential templates. Theplatform server 610 may process template access requests from thecredential issuer systems 630, permitting or denying aspecific system 630 to generate (or issue) a digital credential based on a specific digital credential template. - As used herein, a digital credential template (or digital badge template) may refer to an electronic document or data structure storing a general (e.g., non-user specific) template or description of a specific type of digital credential that may be issued to an individual. Digital credential templates may include, for example, a description of the skills, proficiencies, and/or achievements that the digital credential represents. This description may take the form of diploma data, certification data, and/or license data, including the parent organization (i.e., the digital credential template owner) responsible for creating and defining the digital credential template. Examples of digital credential templates may include templates for various technology certifications, licensure exams, professional tests, training course completion certificates, and the like. In contrast to a digital credential template, a digital credential (or digital badge) may refer to an instance of an electronic document or data structure, generated for a specific individual (i.e., the credential receiver), and based on a digital credential template. Thus, a digital credential document or data structure may be based on a corresponding digital credential template, but may be customized and populated with user-specific information such as individual identification data (e.g., name, email address, and other user identifiers), credential issuance data (e.g., issue date, geographic location of issuance, authorized issuer of the credential, etc.), and links or embedded data that contain the specific user's supporting documentation or evidence relating to the credential.
- As shown in this example, the system 600 also may include a digital
credential receiver system 640 and a digitalcredential endorser system 650. The digitalcredential receiver system 640 may be a computing device associated with a credential receiver (or credential earner), for example, an individual user of an electronic learning system, professional training system, online certification course, etc. In some embodiments, credential receivers may access theplatform server 610 viasystems 640 to accept or reject newly issued digital credentials, review and update their own set of previously earned digital credentials, as well as to publish (or share) their digital credentials via communication applications or publishing platforms such as social media systems. Digitalcredential endorser system 650 may be a computing system associated with an endorsing entity, such as an educational institution, business, or technical organization that has chosen to review and endorse a specific digital credential template. Theplatform server 610 may receive and track the endorsements received fromsystems 650, and may associate the endorsements with the user-specific digital credentials issued based on the endorsed templates. - Additionally, the digital credential management system 600 in this example includes a number of
external client devices 660 and externaldigital credential publishers 670.External client devices 660 may correspond to computing systems of third-party users that may interact with theplatform server 610 to initiate various functionality or retrieve data relating to templates and/digital credentials managed by theplatform 610. For example, aclient device 660 may query theplatform server 610 for data metrics and/or analyses relating to a subset of digital credentials stored in the digitalcredential data store 615. The third-party systems 660 also may provide data to theplatform server 610 that may initiate updates to the templates and/digital credentials stored in thedata store 615. Externaldigital credential publishers 670 may correspond to third-party systems configured to receive digital credential data from theplatform 610 and publish (or present) the digital credential data to users. Examples ofpublishers 670 may include social media website and systems, digital badge wallets, and/or other specialized servers or applications configured to store and present views of digital badges to users. - In various embodiments described herein, the generation and management of digital credentials, as well as the tracking and reporting of digital credential data, may be performed within CDNs 100, such as eLearning, professional training, and certification systems 100. For example, within the context of an eLearning CDN 100, a
content management server 102 or other CDN server (e.g., 104, 112, 114, or 116) may create and store digital credential templates to describe and define various proficiencies, achievements, or certifications supported by the eLearning CDN 100. Additionally or alternatively, thecontent management server 102 or other servers of an eLearning CDN 100 may issue digital credentials to users, based on its own digital certificate templates and/or templates received from other systems or CDNs. Further, in some implementations, an eLearning CDN 100 may be configured to include a digitalcredential platform server 610 to store and manage templates and digital credentials between separate systems within the CDN 100. Thus, in various different implementations, the content management server(s) 102 of a CDN 100 may incorporate one or more digital certificate template owner system(s) 620, digital certificate issuer system(s) 630, and/or digital certificate platform server(s) 610. In such embodiments, the various components and functionalities described herein for theplatform server 610,owner system 620, and/orissuer system 630 all may be implemented within one or more content management servers 102 (and/or other servers) of an eLearning or professional training CDN 100. In other examples, a digitalcredential platform server 610 may be implemented using one or more computer servers, and other specialized hardware and software components, separately from any other CDN components such ascontent servers 112,content management servers 102,data store servers 104, and the like. In these examples, the digitalcredential platform server 610 may be configured to communicate directly with related systems 620-670, or indirectly throughcontent management servers 102 and/or other components and communications networks of the CDN 100. - In order to perform these features and other functionality described herein, each of the components and sub-components discussed in the example digital credential management system 600 may correspond to a single computer server or a complex computing system including a combination of computing devices, storage devices, network components, etc. Each of these components and their respective subcomponents may be implemented in hardware, software, or a combination thereof. Certain systems 620-670 may communicate directly with the
platform server 610, while other systems 620-670 may communicate with theplatform server 610 indirectly via one or more intermediary network components (e.g., routers, gateways, firewalls, etc.) or other devices (e.g.,content management servers 102,content servers 112, etc.). Although the different communication networks and physical network components have not been shown in this example so as not to obscure the other elements depicted in the figure, it should be understood that any of the network hardware components and network architecture designs may be implemented in various embodiments to support communication between the systems, servers, and devices in the digital credential management system 600. Additionally, different systems 620-670 may use different networks and networks types to communicate with theplatform server 610, including one or more telecommunications networks, cable networks, satellite networks, cellular networks and other wireless networks, and computer-based IP networks, and the like. Further, certain components within the digital credential management system 600 may include special purpose hardware devices and/or special purpose software, such as those included in I/O subsystem 611 andmemory 614 of theplatform server 610, as well as those within the memory of the other systems 620-670, and the digitalcredential data store 615 maintained by theplatform server 610, discussed below. - Although the various interactions between the
platform server 610 and other systems 620-670 may be described below in terms of a client-server model, it should be understood that other computing environments and various combinations of servers and devices may be used to perform the functionality described herein in other embodiments. For instance, although the requests/responses to determine the authorizedissuers 630 for specific digital credential templates, the generation of digital credentials, and the retrieval and presentation of digital credential tracking and reporting data, may be performed by a centralized web-basedplatform server 610 in collaboration with various client applications at the other systems 620-670 (e.g., web browser applications or standalone client software), in other cases these techniques may be performed entirely by a specialized digitalcredential platform server 610, or entirely by one or more digital credential tools (e.g., software services) executing on any one of the systems 620-670. In other examples, a client-server model may be used as shown in system 600, but different functional components and processing tasks may be allocated to the client-side or the sever-side in different embodiments. Additionally, the digitalcredential data store 615 may be implemented as separate servers or storage systems in some cases, and may use independent hardware and software service components. However, in other implementations, some or all of the digitalcredential data store 615 may be incorporated into the platform server 610 (as shown in this example) and/or may be incorporated into various other systems 620-670. - In some embodiments, each of the systems 620-670 that collaborate and communicate with the
platform server 610 may be implemented as client computing systems, such desktop or laptop computers, smartphones, tablet computers, and other various types of computing devices, each of which may include some or all of the hardware, software, and networking components discussed above. Specifically, any of client systems 620-670 may be implemented using any computing device with sufficient processing components, memory and software components, and I/O system components for interacting with users and supporting the desired set of communications with theplatform server 610, as described herein. Accordingly, client systems 620-670 may include the necessary hardware and software components to establish the network interfaces, security and authentication capabilities, and capabilities for transmitting/receiving digital credential templates and digital credentials, digital credential data requests/responses to theplatform server 610, etc. Each client system 620-670 may include an I/O subsystem, network interface controller, a processing unit, and memory configured to operate client software applications. The digitalcredential platform server 610 may be configured to receive and execute various programmatic and graphical interfaces for generating, managing, and tracking issued digital credentials, in collaboration with the various client systems 620-670. Accordingly, each client system 620-670 may include an I/O subsystem 611 having hardware and software components to support a specific set of output capabilities (e.g., LCD display screen characteristics, screen size, color display, video driver, speakers, audio driver, graphics processor and drivers, etc.), and a specific set of input capabilities (e.g., keyboard, mouse, touchscreen, voice control, cameras, facial recognition, gesture recognition, etc.). Different client systems 620-670 may support different input and output capabilities within their I/O subsystems, and thus different types of user interactions, andplatform server 610 functionality may be compatible or incompatible with certain client systems 620-670. For example, certain types of digital credential generation and search functionality may require specific types of processors, graphics components, network components, or I/O components in order to be optimally designed and constructed using a client system 620-670. - In some embodiments, the digital
credential platform server 610 may generate and provide software interfaces (e.g., via a web-based application, or using other programmatic or graphical interface techniques) used by the various client systems 620-670 to perform the various digital credential management functionality described herein. In response to receiving inputs from a client system 620-670 corresponding to digital credentials, templates, credential search requests and criteria, etc., theplatform server 610 may access the underlying digitalcredential data store 615 perform the various functionality described herein. In other to perform the tasks described herein,platform server 610 may include components such asnetwork interface controllers 612, processingunits 613, andmemory 614 configured to store server software, handle authentication and security, and to store, analyze, and manage the digital credentials, templates, and credential tracking data stored within the digitalcredential data store 615. As shown in this example, the digitalcredential data store 615 may be implemented as separate dedicated data stores (e.g., databases, file-based storage, etc.) used for storing digital credential template objects, issued digital credentials, credential tracking data, and authorized user/role data. Theplatform server 610 anddata store 615 may be implemented as separate software (and/or storage) components within asingle computer server 610 in some examples, while in other examples may be implemented as separate computer servers/systems having separate dedicated processing units, storage devices, and/or network components. - Certain aspects described herein related to the testing and certification processes used to verify the skills or qualifications that a user (or earner) has obtained in order to be awarded with a digital credential (or badge) or any other skill certification from an institution or credentialing body. In some embodiments, physical testing environments including “simulation laboratories” may use implemented to allow users to perform physical tasks (including mental and/or computer-based tasks) in a monitored environment. Such physical testing environments may use virtual reality and/or augmented reality in various cases. The simulation lab and/or the user may be monitored by various sensors during testing or certification processes, and the results may be analyzed to determine (at least in part) whether or not the user should be awarded a particular digital credential or certification. As discussed below in more detail, simulation labs may be implemented as testing environments for manual tasks, computer-based tasks, scenario training, etc., and various monitoring of the simulation lab environment during test may provide data metrics relating to successful completion of tasks, efficiency of task completion, user response times, user decision making behaviors, user biometrics and risk factors, etc. Further, as discussed below, certain simulation labs may provide the ability to change testing scenarios as well as environmental conditions (lighting noise, temperature, etc.) during testing.
- Referring now to
FIG. 7 , an example is shown of a physical testing environment that may be used for badge testing, skills certification, and other behavior monitoring and digital credential generation, in accordance with certain aspects described herein. In this example, abasic testing environment 700 is shown to illustrate certain features and concepts that may be included in various embodiments. Depending on the particular digital credential, activity, skill or ability to be verified, different devices and components may be included in thesimulation environments 700. For example,simulation environments 700 for standardized testing and completion of computer-based tasks may be setup to simulate an office environment, for instance, with a computer, keyboard, monitor, desk and chair, etc.Other testing environments 700 designed for other badges and/or skills certifications may be configured differently. For instance,testing environments 700 may be configured as a driving simulator (e.g., having front and side display screens, an installed automobile seat with steering wheel, pedals, vehicle controls and gauges, simulated mirror displays, etc.), or a flight simulator (e.g., having front and side display screens, up and down fields of vision, a pilot seat with a center stick and/or other airplane controls and gauges, etc.).Other testing environments 700 might not require or have any display screens, forexample testing environments 700 for CPR certification may include one or more CPR manikins and other accessories to test CPR scenarios.Additional testing environments 700 may be implemented for law enforcement use of force or defensive tactics scenarios (with or with display screens, with or without live firearms capabilities, etc.). Stillother testing environments 700 may be implemented for skills testing and verification on machine assembly tasks, and/or on machine use tasks. The machines intesting environments 700 in such scenarios may range from simple to complex, to allow users to any testable task on any machine, from bicycle assembly, to automobile maintenance, to semiconductor design, to electrical work, to laser fabrication, to welding.Other testing environments 700 may be implemented for skills testing and verification in performance of medical or dental procedures, and the like, and thus may resemble a hospital operating room or dentist office with a full complement of medical tools and devices necessary to perform the tasks to be verified. Stillother testing environments 700 may be configured to test/verify skills with respect to sports or other physical activities, and thus thetesting environments 700 may comprise a dance studio, gymnastics apparatus, golf driving range, or other sports equipment. For each of these examples, and many others, it should be understood that the different configuration oftesting environments 700 may require different sets of testing equipment, as well as different monitoring and environmental control features. Further, although many examples and implementations described herein refer to human users as the subjects of testing and simulation scenarios, in some cases the test subjects may include mechanical devices (e.g., machines configured to assemble parts), artificial intelligences and/or other software programs configured to perform certain tasks, etc. - In addition to the testing equipment and apparatuses in the
physical testing environment 700, the environment may havecameras 705 and sensors configured to monitor the performance and behavior of the user during the testing. As shown in this example, a number ofcameras 705 may be installed throughout thetesting environment 700 to capture image/video data of the user from different angles during the testing/skills verification process. In addition to cameras, in various embodiments (depending on the type of test or skill being evaluated), additional sensors may be deployed within thetesting environment 700, including microphones, light sensors, heat sensors, vibration sensors, and any other sensor type, depending on the type of testing/evaluation being performed. For instance, for testing of computer-based tasks, additional sensors such as mouse movement trackers, keystroke loggers, and user eye-tracking software may be used. For machine usage tasks, scenario training, and the like, movement sensors may be placed on the user and/or on any objects with which the user may interact during the testing scenario. Additionally, for any testing or skills evaluation scenario, certain embodiments may include biometric sensors anddevices 710 configured to detect and track the user's biometric data during the testing process. Such biometric sensors and devices may measure the user's temperature, heartrate, blood pressure, respiration, skin conductivity, body movements, brainwave activities, etc. - In some embodiments, the
physical testing environment 700 also may include various environmental controls that allow a test administrator to control the physical environmental conditions during a test or skills evaluation. Such environmental controls may includelights 715 that allow the test administrator to control the light levels, angles, and/or colors during a test. By way of example, lighting control within theenvironment 700 may allow the test administrator to evaluate the user's ability to perform a driving maneuver or roadside maintenance task at night, etc. Additional environmental controls may include may include temperature controls, weather simulation (e.g., wind, rain, snow, sunshine, fog, etc.), speakers to provide background noise or distraction, olfactory control that provides scents/odors to simulate the smells that be present during a comparable real-life scenario, vibration control to simulate the activity, and so on. - Referring now to
FIG. 8 , a flow diagram is shown illustrating an example process of executing tests or simulations, as well as monitoring and analyzing the results of the tests or simulations. As described below, the steps in this process may be performed using various components of a simulation lab and/or other physical test (or simulation)environment 700, described above. For example, each of steps 801-810 may be performed by a computer server of a test administrator associated with aphysical simulation environment 700. In other examples,physical simulation environments 700 might be configured to receive test content and configuration parameters, to execute the tests and monitor the execution, and then to transmit the test results and related observation data to a separate server (e.g., a digital credential platform server 610) for scoring and analysis. - In
step 801, a computer server controlling thephysical testing environment 700 may receive input relating to the test or skills evaluation scenario to be executed within thephysical testing environment 700. Instep 802, the server may receive data identifying the particular user designated to complete the test or skills evaluation scenario. - In
step 803, the server may retrieve the test or scenario to be loaded/executed within thephysical testing environment 700. As noted above, the test or scenario may include interactive user software (e.g., driving or flight simulator programs, law enforcement scenarios, etc.) and/or may include testing software or other software programs loaded onto a desktop, laptop, or tablet computer. For instance, the test or scenario may require the user to work with computer-aided design software, spreadsheet software, database development software, etc. In other cases, the test or scenario may include audio and/or video files to be played via speakers and/or display screens within thephysical testing environment 700, such as instructional videos or audio/visual test questions. - The test or scenario retrieved in
step 803 also may be retrieved based on the identity of the particular user who will be completing the test or skills evaluation scenario. In some embodiments, the server of thephysical testing environment 700 may be configured to select the appropriate test, scenario, and/or simulation (e.g., a particular software scenario, skill level, etc.) based on the user's current set of badges or digital credentials, the user's skill level, and/or the user's performance history on previous tests or scenarios within thetesting environment 700. Additionally, in some cases, the server may vary scenarios/test questions so that a particular user does not receive the same test questions, scenarios, or other testing content that they have already completed (or completed within a particular recent time window). - In
step 804, the server may determine and apply a set of environmental conditions within thephysical testing environment 700 for the execution of the test or scenario. As noted above, thephysical testing environment 700 in some embodiments may be capable of setting various environment conditions such as lighting (e.g., to simulate different day or night, and/or different real-world working environments), temperature and weather conditions (e.g., to simulate outdoor scenarios, different seasons and locations), noise (e.g., to provide background noise, traffic noise, distractions, etc.) and other various environment conditions. The server may select and apply environmental conditions as part of the test or scenario selected instep 803, or as a separate determination which is performed based on random chance or selected by a test administrator, etc. For instance, for certain types of badges and other certifications, separate day and night testing of certain tasks may be required. In other cases, the environmental conditions may be selected randomly and changed for each testing session. In still other cases, user may select and/or save their preferred environmental conditions for different types of testing. Further, in some embodiments, thephysical testing environment 700 may track and analyze the user's various testing or scenario performance metrics (e.g., accuracy, efficiency, safety, compliance, biometrics, etc.) under different environmental conditions, in order to determine the optimal environmental conditions for the particular user. In such cases, user's may receive different badges or certifications (or may have different badge assigned characteristics or endorsements) based on their test or scenario performance in different environmental conditions. - In
step 805, the computer server(s) associated with thephysical testing environment 700 may execute the test or simulation scenario, during which the user's performance and any/all user reactions or responses may be monitored. As noted above, even for certain tests that are entirely manual in nature, thephysical testing environment 700 may use cameras and any other sensors to monitor the user's actions. Such monitoring may include various aspects of the user's performance, such as answers to test questions selected via a testing computer terminal, or the user's interactions with physical objects (and/or other people) within thephysical testing environment 700. The user's answers and actions may be recorded by cameras and computer input devices, and additional user data may be collected using various other sensors such as microphones, biometric sensors, etc. - In
step 806, the results for the test and/or simulation scenario completed by the user may be analyzed. In some embodiments, the such analyses may be performed based not only on the user's responses to particular test questions or scenarios. Additionally or alternatively, the analysis instep 806 may include an evaluation of the user's other reactions or responses, such as speed and confidence of action (e.g., as determined by user comments, speed of response, facial expression analysis, body movement analysis, biometric data, etc.), efficiency, safety, decision making, and user biometrics. One or more of these separate analyses may be performed in steps 807-810, and each may be performed independently of the others, or may be combined into a single analysis. For instance, in some cases the goal of the simulation might be only to measure the user's biometric data, and the user's actual responses to the questions/scenarios may be irrelevant and need not be evaluated instep 807. In other tests or simulation scenarios, the opposite analysis may be applied, where only the accuracy of the user's responses or behaviors are measured and analyzed instep 807, and the user's biometric data is irrelevant and thus the analysis instep 810 is not performed. As another example, in a certain simulation of driving, machine operation, use of force training, etc., the only relevant analysis to be performed may be a safety/decision making analysis instep 809, while the efficiency analysis instep 808 need not be performed. In other similar tests/situations, the server may apply both a safety/decision making analysis instep 809 and an efficiency analysis in step 808 (e.g., to confirm that a driving maneuver or route was completed both safely and efficiently, to confirm that a suspect was subdued safely and quickly, to assure that a manufacturing assembly task was performed safely and efficiently, etc.). - For example, in certain embodiments and implementations of the concepts discussed above in reference to
FIGS. 8-9 , various techniques (e.g., systems, methods, computer-program products tangibly embodied in a non-transitory machine-readable storage medium, etc.) may include evaluating a physical simulation by a digital credential generator system (e.g., 630 and/or 610). Such techniques may include monitoring, by a digital credential generator system, a physical simulation area using a plurality of sensors, during a physical simulation. During the monitoring the digital credential generator system may detect, using the plurality of sensors, various physical actions performed by a user during the physical simulation. The digital credential generator system may analyze data corresponding to the plurality of physical actions performed by the user during the physical simulation, and determine that the user corresponds to a particular credential receiver. The digital credential generator system then may determine whether the first credential receiver is eligible to receive a digital credential, by comparing the data corresponding to the analysis of the physical actions performed by the user during the physical simulation, to one or more digital credential requirements. Finally, the digital credential generator system may generate a first digital credential associated with the first credential receiver and with the digital credential requirements, in response to determining that the first credential receiver is eligible to receive the first digital credential. - In some embodiments, outputting a physical simulation may include outputting audio and/or video simulation components within the physical simulation area, manipulating physical objects (e.g., motorized objects) during a live-action simulation within the physical simulation area, and/or outputting virtual reality simulations via a virtual reality headset. Additionally, certain embodiments may include generating physical simulation environments within a physical simulation area, including, for example, simulating ambient light conditions within the physical simulation area, outputting one or more background noise conditions within the physical simulation area, monitoring and controlling the physical temperature using a heating and cooling system installed at the physical simulation area, outputting smells to the physical simulation area using a smell output device, and/or outputting vibratory effect within the physical simulation area, using a vibration system.
- As noted above, the monitoring of a test/simulation may include monitoring physical actions/activities performed by the user using video recording devices and/or motions. Additionally or alternatively, the monitoring may be of computer-based tasks, using additional software-based sensors such as mouse movement trackers, keystroke loggers, and user eye-tracking software, etc.
- In accordance with certain aspects described herein, the processes used for testing/evaluating a user and determining that a user is eligible for a particular digital credential (or badge) need not include a specific test, designated evaluation, or scored scenario training. Rather, the testing and badging determinations may be performed automatically during the user's normal course of on-the-job performance of tasks. In such embodiments, the testing and credentialing of users may be based on observation of workers during their normal work activities. Cameras and other sensors may be installed and used to detect the completion of tasks and/or certain competencies of the users, and the data from these sensors may be evaluated to automatically determine when the user is eligible for a digital credential. Thus, on the job testing and badging may be performed entirely transparently to the worker performance of their job duties, and need not require any delay or distraction from job performance, or any designated time or location needed to perform formal testing.
- In order to perform automatic and on-the-job testing and credentialing of workers or other users (e.g., students, athletes, etc.), the “work” environment of the user may be monitored with cameras and/or sensors capable of tracking the user's activities and performance. As discussed above with respect to the implementation of physical testing environments (e.g., 700), different types of digital credentials relate to different activities that may be performed in a variety of different work environments. Referring briefly to
FIG. 9A , an example work environment 900 a is shown for a user completing computer-based tasks. In this example, the work environment 900 a may include a basic workstation, server, modem, printer, monitor, keyboard, etc., as well as desk and chair to allow the user to complete normal computer-based work activities. In this example, the user may be data entry specialist, computer programmer or design engineer, call center customer support operator, or may be performing any other computer-based job. In such examples,sensors external cameras 905 and other sensors may capture and analyze the user's interactions with the software application, and thus need not affect the operation of the software at all. - Another example work environment is shown in
FIG. 9B . In this example work environment 900 b, the entire layout of workplace floor is shown and monitored by a series ofcameras 905 and/or other sensors. The monitoring in this example may apply to works who do not perform only computer-based tasks, but whose work requires them to interact with physical objects within their workspace, and/or to move around the work environment 900 b to other workspaces. For instance, maintenance works, office mail delivery works, construction workers, electricians, plumbers, machine assembly or manufacturing works, etc., may be monitored with such systems. When monitoring a larger area for the performance of non-computer-based work tasks, in addition tocameras 905, the work environment 900 b may include motion sensors, microphones and noise sensors, as wells as movement sensors and/or tracking devices that may be placed on specific physical objects within the environment. By way of example, work environment 900 b may correspond to a shop floor, mechanic's garage, or manufacturing assembly plant, and thecameras 905 and other sensors may be used to confirm that workers are complying with safety requirements and/or health codes with respect with their work with machinery or hazardous materials, etc. As another example, work environment 900 b may be an office environment, and thecameras 905 and other sensors may be used to confirm that individual workers are working efficiently, in their assigned areas, etc., and that workers without assigned areas (e.g., cleaning, mail delivery, maintenance workers, etc.) are working efficiently and not skipping any portion of the floor 900 b. - Referring now to
FIG. 10 , a flow diagram is shown illustrating an example process of automatically monitoring work activities and issuing digital credentials via “on-the-job” testing. As described below, the steps in this process may be performed by monitoring and credentialing computing devices operating within various types of work environments 900, such as those described above. For example, each of steps 1001-1006 may be performed by a computer server operating automatically and unassisted (or at the direction of an administrator) within a work environment 900. In other examples, work environments 900 might be configured only to monitor work activities and performance, and then to transmit the results and related observation data regarding various worker to a separate server (e.g., a digital credential platform server 610) for scoring, analysis, and the issuance of digital credentials. - In
step 1001, a computer server controlling the on-the-job badging system may activate the cameras, sensors, monitoring software, etc., within the workstation and/or work environment. As discussed above, this activation may include specific monitoring software to detect computer-based tasks, and/or location monitoring devices such as cameras, sensors, biometrics, etc., depending on the type of workers and work environments 900 being monitored. In some cases, an on-the-job testing and credentialing system may be implemented as an “always on” system, in which the workstation/workplace monitoring is constantly recording and analyzing worker activities. Thus,step 1001 may be optional in such embodiments. However, in other cases, workstation/workplace monitoring might only be activated at certain times and not others, for example, only during normal work hours, only on certain specific work days designated for work evaluation, etc. In some embodiments, a system administrator and/or individual workers may activate or de-activate the workstation/workplace monitoring systems within their work environment at any time. Thus, such systems need not be an invasion of privacy for any worker that does not choose for their work to be monitored and evaluated, but workers may choose to turn the monitoring systems on in order to be eligible for evaluation and earning of additional work related digital credentials and credentials. - In
step 1002, the workstation/workplace monitoring systems may capture the user's work-related activities and behaviors, including performing various computer-based tasks and non-computer-based tasks as discussed above. Instep 1003, the user's working data as collected by the workstation/workplace monitoring systems and sensors may be analyzed by the server, in order to determine instep 1004 whether or not the user is eligible for one or more digital credentials or other credentials (e.g., professional certifications, etc.) based on their on-the job work activities. Certain digital credentials or credentials may be made available to users in response to detecting that the user has successful completed one or more specialized work tasks, thus demonstrating that the user has obtained the particular skill associated with the digital credential. In some cases, the server and/or the monitoring systems and sensors may also be configured to detect a certain level of efficiency by the user in performing the tasks, and/or may require that the user perform a certain task N number of times before the user is eligible for the digital credential or credential. - In
step 1004, if the system determines that the user is eligible for one or more particular digital credentials (1004:Yes), then instep 1005 the system may either issue the digital credential directly (e.g., if the workplace server is permitted to be digital credential issuer), and/or may initiate a communication session with abadging platform 610 and/ordigital credential issuer 630 to request that a new digital credential is issued for the worker. In such examples, the workplace server may provide the information identifying the worker (e.g., name, employee ID, digital credential system profile ID, etc.) to adigital credential platform 610 orissuer 630, along with verification that the worker has completed the requirements to earn a particular digital credential. In some embodiments, the servers operating at the workplace may be configured to capture evidence (e.g., video evidence, screen captures, facial/identity verification, etc.) and transmit the evidence to the digital credential-issuing authority, before the digital credential may be issued. - In
step 1006, the worker may be notified that they have received a digital credential based on their normal on-the-job activities. In some embodiments, the worker may indicate interest in obtaining one or more particular digital credentials, and the workstation/workplace monitoring system may be configured to evaluate the worker with respect to the particular digital credentials or credentials that the worker has expressed interest in. However, in other examples, it may be possible for a worker to receive an issued digital digital credential without expressing any interest in the digital credential (or even being aware of such a digital credential), but solely based on the determination that the worker has achieved the level of skills mastery required for the digital credential/credential, based on the automated monitoring of the worker within the workplace. In certain cases, a user may be informed that they are eligible for receiving a digital credential prior to the issuance of the digital credential instep 1005, and the user may be allowed to accept or reject the digital credential. Additionally, in some cases, the user may receive status reports (e.g., daily, weekly, etc.) identifying which digital credentials the user is being monitored for, and the user's progress with respect to earning those digital credentials. This data may include indications to the worker that he/she may earn a particular digital credential after performing a task another N times, or performing the task N amount faster, or performing the task without making any errors or backtracking, etc. - For example, in certain embodiments and implementations of the concepts discussed above in reference to
FIGS. 10-11 , various techniques (e.g., systems, methods, computer-program products tangibly embodied in a non-transitory machine-readable storage medium, etc.) may include generating digital credentials for particular credential receivers, based on monitoring of a physical environment using a plurality of sensors. A digital credential issuer (or generator) 630 and/or 610 may detect or receiver sensor data from a number of sensors corresponding to the user actions performed by a user within the physical environment. The digital credential generator may determine the operations that were/were not performed within the physical environment, based on the user actions detected. The digital credential issuer then may retrieve data from a credential receiver data store associated with the user/credential receiver, and determine one or more digital credential templates, based on the retrieved data associated with the first credential receiver. For each of the digital credential templates, the digital credential issuer/generator may retrieve criteria associated with the digital credential template, compare the operations performed by the user within the physical environment to the criteria associated with the digital credential template, and then determine whether or not a credential receiver is eligible to receive a digital credential based on the digital credential template. If the first credential receiver is eligible to receive a digital credential based on the digital credential template, the digital credential generator may generate a digital credential based on the digital credential template and user data associated with the first credential receiver. - In such cases, the generated (or issued digital credential) may be embedded with additional data such as the evaluation/simulation time, location, or the sensor system/physical environment within which the evaluation/simulation was performed. Additionally, in some cases, facial recognition data and/or biometric data may be collected from the user (credential receiver), and may be used to validate or authenticate the digital credential by verifying the user's identity. As in the above examples, the monitoring may be done using physical movement tracking sensors such as video recorders and/or motion detectors, or may use software-based sensors such as network monitoring devices, keystroke loggers, mouse movement monitors, touch screen monitors. Such software-based tools may operate as background processes on a computer terminal being monitored, and/or may be built into specific software programs with which the user is interacting.
- Additional aspects related to the automated tracking of user or worker activities, after the user/worker has been issued a badge (or digital credential), in order to determine how often the user/worker is “using” their digital credential. Depending on type of digital credential or credential, post-credentialing monitoring of the user may involve analysis of user's physical work product (e.g., documents produced, parts/items created, etc.), or may be involve observations of the user (e.g., via a workstation/workplace monitoring system). In order to evaluate how often a user is using a particular digital credential, a data store of digital credentials may be linked to particular skills, work-related, or activities. The user/worker may then be tracked to determine the number of such tasks performed, and/or the quality, efficiency, and/or competence of the user's performing those tasks, in order to determine to what extend the user/worker is “using” the digital credential.
- Referring now to
FIG. 11 , an example computing environment 1100 is shown, including a digitalcredential platform server 1110, one or more workstation/workplace monitoring systems 1120, and a credential-to-activity mapping data store 1130. In some examples, the digitalcredential platform server 1110 may be a badging server similar or identical to theserver 610 discussed above. Thus,server 1110 may be configured as a digital credential repository and credentialing system, acting as a clearinghouse for digital credential owners, issuers, earners, endorsers, etc.Server 1110 may include a digital credential (or digital credential) data store configured to store badging information such as the details of the particular digital credentials earned by particular users. As noted above, such details may include the date on which a digital credential was issued to a user, and for certain digital credentials, an expiration date associated with the digital credential. - In this example, system 1100 also includes a credential-to-activity
mapping data store 1120, which may be implemented as a separate external data store and/or may be integrated into the digital credential data store of server 1100. The credential-to-activity mapping data store 1130 may include mappings of one or more tasks or activities associated with each digital credential type that a user may potentially earn. For example, a digital credential relating to automotive maintenance for a particular make of car may have associated activities and tasks that include particular maintenance tasks (e.g., tune-ups, part replacements, etc.) for different model cars having the make. As another example, an operating system administrator-related digital credential may list, within data store 1130, various system administrator tasks and that a user may perform on the particular operating system. In some cases, the activities or tasks associated with a particular digital credential may correspond to the same set of activities or tasks that a user is required to perform to earn the particular digital credential, and as discussed below, these activities or tasks may serve as a metric to evaluate how much the user is “using” the digital credential. - One or more workstation and/or
workplace monitoring systems 1120 may provide user monitoring data to theserver 1110, to allow thesever 1110 to analyze the user's activities and determine to what extent the user is using the activities and abilities associated with their digital credentials. In some embodiments, the workstation and/orworkplace monitoring systems 1120 may be similar or identical to any of the workstation/workplace monitoring systems and sensors discussed above. For example,workplace monitoring systems 1120 may collect records detailing the user's physical work product (e.g., documents produced, modified or accessed by the user, inventory or work order records indicating tasks performed by the user, etc.). Additionally,workplace monitoring systems 1120 may include observation systems (e.g., workplace monitoring systems) including cameras and other sensors to track the user's activities and determine which specific tasks have been performed by the user. - In some embodiments, the monitoring and tracking of post-credentialing activities by the user may be used to analyze and provide digital credential or credential feedback data to various entities. For example, referring now to
FIG. 12 , a flow diagram is shown illustrating an example process that may be used to determine whether a user has or has not used the activities associated with a particular digital credential that they have obtained, and then to aggregate and report that digital credential usage data to the relevant parties. Instep 1201, a particular digital credential is issued to a user based on the user's successful completion of the badging requirements. As in the various examples discussed above, the digital credential may be associated with a computer-based activity, non-computer-based activity, or any other set of digital credential requirements determined by a digital credential owner or issuer. Additionally, the digital credential issuance instep 1201 may be the result of formal testing and/or certification processes, or may be based on on-the-job or other observational data collected for the user. - In
step 1202, thedigital credential server 1110 and/ormonitoring systems 1120 may monitor and track the activities of the credentialed user, including, for example, the workplace tasks performed by the user based on analyses of the various monitoring systems/sensor data installed at the user's workstation and/or workplace environment. As described above, determining what activities and tasks the credentialed user has performed, and when, may be performed using a variety of techniques. In some cases, determining what work-related tasks a user has performed, and what other activities they have been engaged in, may be done by analyses of written and electronic documents associated with the user or workplace. For instance, documents such as maintenance requests, work orders, customer tickets, purchase receipts, and the like may be analyzed to determine what activities or tasks the user has completed and when. For instance, a maintenance record listing the user as the assigned technician may be used in determination that the user has performed the specified task/activity at the time listed on the record. In other examples, the user's electronic mail and other electronic documents may be searched and analyzed (e.g., using a keyword analysis and/or trained artificial intelligence) to determine what tasks the user has performed and/or what activities the user has demonstrated during the relevant time periods. In some embodiments, there may be particular advantages in implementing a post-credentialing usage analysis and/or digital credential valuation process for certain digital credentials/tasks that are more discrete and detectable, for instance, a number of transmissions changed after earning a vehicle transmission certification, a number of particular medical procedures done following a digital credential credential for the procedure, a number of IT tickets resolved successfully following receiving an advanced IT computer services and computer repair digital credential, etc. In contrast, for other tasks and activities for which a user may receive a digital credential, such as leadership, communication skills, advanced C software programming, jujitsu skill levels, and the like, it may be more difficult to quantify if when, and how often a user is using the particular skill or task associated with the digital credential. - In
step 1203, a set of tasks and/or activities associated with the digital credentials obtained by the specific user may be retrieved using the credential-activity mapping data store 1130, and instep 1204 the retrieved tasks and/or activities may be compared to the tasks and activities that have been performed by the user subsequent to the digital credentials being earned (as determined in step 1202). As an example, the comparison instep 1204 may determine that in the six month since the user was issued a professional certification to perform a particular technical task, the user has performed that task on a weekly basis. Alternatively, for a different digital credential issued to the user directed to expertise in a particular software program, the comparison instep 1204 may determine that the user has used that software program only once since receiving the digital credential two years ago. In this case, the system may conclude that the professional certification issued six months ago to the user has been of greater usefulness than the software digital credential issued two years ago (allowing for the possibility of career changes, prestige-driven digital credentials rather than functional digital credentials, etc.). - In
step 1205, data from the comparison ofstep 1204, i.e., data indicating the post-credentialing usage by the user of the digital credential-associated activities or tasks, may be aggregated and analyzed, and then transmitted to one or more of the relevant system components. In various embodiments, any of several different components and roles associated with thecredentialing platform 1110 may request and receive this information for their associated digital credentials and/or associated users. For instance, digital credential owners and/or digital credential issuers may request and receive from theplatform server 1110 data regarding the post-issuance usage of the digital credentials they own or have issued. In other cases, digital credential endorsers may request and receive from theplatform server 1110 data regarding the post-issuance usage of the digital credentials they have endorsed. Digital credential earners, the users themselves also may request reports from theplatform server 1110 quantifying the post-credentialing usage (which may be expressed in terms of time, value, and/or dollar amounts) associated with their previously earned digital credentials. Employers and other organizations also may request such reports for their employees or organization members, in order to determine which digital credentials have been the most used and most useful to the organization. - Referring now to
FIG. 13 , another flow diagram is shown illustrating an related process involving determining whether a user has or has not used the activities associated with a particular digital credential that they have obtained, and then adjusting an expiration or re-certification date associated with the digital credential based on the user's usage of the digital credential skills. The steps in this example may be similar or identical to the corresponding steps inFIG. 12 , and in some embodiments, the analyses and transmission of the post-credentialing usage described instep 1205 may be performed in conjunction with the setting of an expiration or re-certification date for the digital credential as discussed below. - Steps 1301-1304 may correspond to steps 1201-1204 in some cases, and may be performed using similar or identical techniques to those discussed above. For example, in step 1301 a
platform server 1110 and/or digital credential issuer may issue a digital credential associated with one or more activities or tasks to a particular user, recording the digital credential issuance data within the digital credential data store. Instep 1302, the post-issuance activities of the particular user may be monitored, including monitoring of the user's work-related activities and tasks performed/completed, in order to determine the particular tasks and activities with which the user has been engaged following issuance of the digital credential. Instep 1303, the skills, activities, and tasks associated with the user's digital credential(s) are retrieved, and instep 1304 are compared to the post-issuance user tasks and activities determined for the user instep 1302. Finally, instep 1305, based on the comparison instep 1304, theplatform server 1110 may determine that an expiration date and/or recertification date associated with the user's digital credential should be adjusted based on the user's post-issuance activities. As an example, if the system determines instep 1305 that a user who received a digital credential corresponding to a forklift operator's license or commercial truck driving license three years ago, but has infrequently (or not at all) driven a forklift or a commercial truck since receiving their digital credential, then the system may determine that the user's license should expire at the earliest possible time (e.g., the expiration time as of when the digital credential was first issued). In contrast, if the system determines instep 1305 that the same user has frequently and consistently driven a forklift or a commercial truck ever since receiving their digital credential, and also that the user has a high-safety rating and/or high safety compliance scores, then the system may determine that the user's license may be extended. In such cases, theplatform server 1110 may determine a new extended expiration or recertification time for the digital credential, update the user's digital credential record within the digital credential data store, and transmit notifications to the affected entities (e.g., the user, employer, digital credential issuer, digital credential owner, etc.) providing the new expiration date. In other examples, rather than changing the expiration date or recertification date of a digital credential (or eliminating the expiration altogether), theplatform server 1110 may in other examples determine a new recertification course or procedure for the user, such as simple refresher course to allow the user to recertify quick than the longer complete recertification course used by other users with less post-credentialing digital credential usage. - Additional aspects described herein relate to capturing and using “evidence” data in connection with user testing and credentialing systems, on-the-job evaluation and badging systems, and/or post-credential monitoring systems. For example, within any automated badging/certification/verification system, evidence of the user's performance may be extracted and saved, for example, in a digital credential server along with an associated issued digital credential, or as part of a separate user portfolio of evidence. Evidence data may include, for example, audio and video of the user during a live simulation, or during a virtual reality or augment reality simulation, audio and keystroke data from the user during the testing processing, the user's reaction time and/or decision-making data during a split-second simulated scenario or relevant real-life event (e.g., a workplace accident, etc.), and/or any other sensor or biometric data collected during testing, credentialing, and/or monitoring. As discussed below, evidence data associated with a user may be saved with the user's digital credential and/or into a separate portfolio of evidence, which may be available to the user for review, and also may be provided upon request to potential employers for review during a review or hiring process. Such evidence data also may be applied to updated digital credential credentialing requirements, so that in some cases a user may simply resubmit their evidence portfolio instead of being required to recertify their digital credential when the test or credentialing standards are updated.
- Referring now to
FIG. 14 , an example computing environment is shown including a digitalcredential platform server 1410 in communication with a plurality of testing, credentialing, and/or monitoring systems 1421-1423, and one or moreexternal client devices 1460. In some examples, the digitalcredential platform server 1410 may be a digital credentialing server similar or identical to theserver 610 discussed above. Thus,server 1410 may be configured as a digital credential repository and credentialing system, acting as a clearinghouse for digital credential owners, issuers, earners, endorsers, etc.Server 1410 may include a digital credential (or digital credential) data store configured to store digital credential information such as the details of the particular digital credentials earned by particular users. As noted above, such details may identify the digital credential issuer and/or other testing/credential authorities responsible for administering testing or simulation scenarios as part of the digital credentialing process, and/or for pre-digital credential or post-digital credential monitoring of workstations/workplaces to detect and analyze user tasks performance and user skills/abilities - In this example, the
platform server 1410 may receive data from three testing/credentialing systems 1421-1423. Similar to the above examples, thesimulation lab system 1421 may correspond to a simulation lab or other physical testing environment, an on-the-job credentialing systems 1422 may include workstation/workplace monitoring systems and sensors to record and analyze the user's on-the-job performance, and may issue digital credentials in some cases without the need for any separate formal testing procedure; andpost-credential monitoring systems 1423 may be configured to monitor users following the issuance of a digital credential, including tracking task performance data, skills usage, and the like, and comparing the data to the skills/tasks associated with the user's digital credentials. - In some embodiments, one or more systems 1421-1423 which perform user testing, credentialing, and/or monitoring, such as those systems discussed above, may capture and transmit “evidence data” of the user during a test, simulation, or during an on-the-job monitoring process. Evidence data may include, for example, video and/or audio of the user during a test, simulation (e.g., live, VR, or AR), collected by the sensors of a
physical testing environment 700. Additional evidence data may include user reaction time data, decision-making data, facial expression and body language data, keystroke and mouse movement data, and/or user biometric data. The evidence data may correspond to a time period just before, during, and just after a test, simulation, or a task or activity performed during on-the-job monitoring. - As shown in this example, the various evidence data collected by systems 1421-1423 may be transmitted to the
platform server 1410 and stored in an evidence portfolio data store. The evidence data collected by the testing, credentialing and/or user monitoring systems may be associated with a particular user (or users) and with a particular digital credential (or digital credentials) that the user is in the process of earning or using (e.g., for post-credentialing monitoring). Thus, the evidence data may provide documented proof that the user actually completed the digital credential requirements, along with additional contextual evidence showing how the user performed during the testing, simulation, or monitoring. - Referring now to
FIG. 15 , a flow diagram is shown illustrating an example process by which a testing system, simulator, credentialing systems, workstation/workplace monitoring system, and the like, may collect and preserve evidence data related to a user and a digital credential. Instep 1501, a testing, credentialing, and/or monitoring system such as those described above may execute a test, simulation, or user monitoring process for a particular user in connection with a digital credential that the user is seeking or has already obtained. The particular types of tests may include, for example, live simulations and/or virtual or augmented reality simulations executed within aphysical testing environment 700. In other examples, the testing instep 1501 may correspond to an on-the-job credentialing system that monitors and evaluates a user's workplace tasks and activities, or to a post-credentialing user monitoring system configured to determine whether the user is using their previously issued digital credentials. Instep 1502, during any of these testing, simulation, or monitoring processes, the system 1421-1423 may capture evidence data relating to the user. As noted above, evidence data may include audio or video of the user, user reaction time data, decision-making data, facial expression data, body language data, the user's keystrokes and mouse movement data, particular software interaction data, and/or the user's biometric data. Instep 1503, the evidence data may be encapsulated and transmitted to theplatform server 1410 for storage within the user's evidence portfolio, and instep 1504 theplatform server 1410 may store the evidence data files with data records associated with the user and the particular digital credential(s) to which the evidence applies. In other embodiments, certain systems 1421-1423 may retain and store user evidence data locally, rather than the evidence data being stored in a central repository. Additionally, when the evidence data is transmitted, it may be compressed and edited as needed, and/or encrypted in order to assure data security and user privacy. - In some cases, the
platform server 1410 may determine a subset of the user activities matching digital credential requirements associated with the digital credential, wherein other user activities might bare no relevance to the requirements of the digital credential. In such cases, the platform server may store only the corresponding subset of the evidence/sensor data for the user activities matching digital credential requirements, and might not store other evidence/sensor data corresponding to irrelevant user activities upon which digital credentials do not depend. - Referring now to
FIGS. 16A and 16B , two additional flow diagrams are shown illustrating example processes by which evidence data may be retrieved and/or accessed from aplatform server 1410 or other data repository. As noted above, individual evidence data files stored by theplatform server 1410 may be associated with a particular user and/or with a particular digital credential or credential earned (or in process of earning) by the user. Thus, in some embodiments, evidence data may be stored and made available to certain authorized entities. For instance, instep 1601 ofFIG. 16A , theplatform server 1410 may receive a request for some or all of the user's evidence portfolio. Instep 1602, theplatform server 1410 may perform authorization/authentication on the request to determine (1) whether the requestor is authorized to access the user's evidence data, and/or (2) whether the requested evidence is current and valid. One or both of these determinations may require explicit authorization from the user himself or herself, in order to (1) prevent any unwanted parties from accessing the user's evidence data, and (2) to prevent any old and obsolete from being accessed, even by authorized parties. Thus,step 1602 may include verifying the requestor's identity or role and comparing to an access control list or other permissions data associated with the evidence. In some cases,step 1602 may include a real-time request sent by theplatform server 1410 to a client device associated with the user, to allow the user the option to allow or reject the request. Additionally, the request instep 1601 may specify one or more particular users and/or one or more particular digital credentials for which the associated evidence is to be retrieved, and thus authorization instep 1602 may be granted or denied for evidence relating to each possible combination of users and digital credentials. Instep 1603, assuming that the requestor has been granted access to the requested evidence data, the corresponding evidence data files may be retrieved and forwarded to the requestor. - In some examples, the request in
step 1601 may be from the user himself/herself, who wants to review and study the evidence from his/her previous tests, simulations, and monitoring data. In other examples, the request instep 1601 may be from a current or potential employer, who has been authorized by the user to retrieve and view the user's evidence data associated with all work-relevant digital credentials, as part of a hiring process or review process. The user's evidence data may verify to the employer or potential employer that the user actually completed the digital credential requirements, and also may allow the employer or potential employer to observe the user's behaviors, responses, reactions first-hand, thus allowing them to evaluate the user's reaction time, efficiency, mental state, decision-making, etc., and other difficult to quantify characteristics. In still other examples, the user may authorize a digital credential issuer or digital credential owner to view the user's evidence files related to the digital credentials issued and owned by those entities. Finally, users may make some or all of their evidence data publicly available (e.g., on a file-by-file basis) and/or may actively post their evidence data as a multimedia file or data records within a digital credential profile page of the user that is maintained and published by theplatform server 1410. - In some embodiments, in addition to (or instead of) providing evidence data in response to requests, the
platform server 1410 may provide the functionality to receive updated tests, digital credential requirements, credentialing data, etc., and to apply a user's previously stored evidence to the new testing or credentialing requirements. For instance, instep 1604 ofFIG. 16B , theplatform server 1410 may receive a request to apply previously stored evidence data within a user's portfolio to an updated testing/credentialing process. For example, testing or credentialing authorities (e.g., a digital credential owners or issuers, employers, etc.) may periodically update digital credentialing requirements in order to improve the quality of the digital credential testing, to comply with new best industry practices, to make a digital credential more restrictive by increasing the required scores or efficiency, etc. Additionally, certain testing or credentialing authorities may implement multiple different levels of the same digital credential, in which users are subjected to the same test, same simulation, same monitoring processes, etc., but different scoring ranges may equate to different levels of the digital credential that may be earned by the user. In these scenarios, whenever digital credential requirements are updated, or if a new digital credential level is made available, it may be possible to apply the user's previously collected evidence data to the new digital credential requirements or digital credential level, rather than requiring the user to retake the test, simulation, or monitoring process. As an example, a set of new requirements for particular digital credential may be similar to the previous set of requirement, with the addition of a newly imposed time limit by which the test or simulated scenario must be completed. In other example, new digital credential requirements or digital credential levels may raise the minimum performance level during a test or simulation to a higher level, and/or may require additional steps or procedures during the test or simulation that were not required in the previous version of the digital credential requirements. In these cases, rather than require the user to retest/recertify to earn the updated digital credential, theplatform server 1410 may provide the service of receiving the updated digital credential requirements or new digital credential levels, and automatically evaluating the new digital credential requirements/levels using the user's evidence data that was collected with earning the previous version of the digital credential. Thus, instep 1605, the requestor may be authenticated and the requested data may be validated, and instep 1605 the user's evidence data may be applied the updated testing/credentialing process. Referring to these same digital credential requirements changes discussed above, the evaluation instep 1606 may include automated analysis of the user's evidence data to determine whether the user complied with the newly imposed time limit, the new minimum performance level, and/or performed the additional new steps or procedures during the user's previous digital credential testing. If so, the digital credential authority may allow the user to upgrade their digital credential automatically without having to retake the test or simulation, etc. If not, the user may be informed that they are required to retake the test or simulation (or in some cases they may receive a lower digital credential level). Either way, instep 1607, the results of the evidence analysis and application to the new credentialing requirements may be output to the requestor. Another potential advantage in certain embodiments may include the protection of the user's evidence data itself. For instance, in the above example, theplatform server 1410 might perform the analysis and application of the user's previously stored evidence data to the new testing requirement, without ever allowing any other entity access to the evidence data. In other examples, theplatform server 1410 may perform the analysis and/or may provide the actual evidence data files to the requestor device, with the sufficient authorization from the user. - In various embodiments, the updated testing/credentialing process in
step 1604 may correspond to a re-issuance of a digital credential, with the same or updated requirements, or may corresponding to a different digital credential having similar and/or overlapping issuance requirements. For instance, theplatform server 1410 may receive an updated set of requirements for a digital credential previously issued to a credential receiver, may retrieve the stored set of sensor data corresponding to the relevant activities performed by the credential receiver in connection with the issuance, may compare the retrieved set of sensor data/activities to the updated set of digital credential requirements, and then may generate/issue an updated digital credential to the credential receiver, based on the comparison of the retrieved set of sensor/activity data to the updated set of digital credential requirements. Similar techniques may be performed to generate and issue digital credentials to receivers for entirely different digital credentials, rather than updated credentials, such as similar credentials and/or credentials having overlapping eligibility requirements. - Additional aspects described herein relate to capturing and using user biometric data, physical user cues, and the like, in connection with user testing and credentialing systems, on-the-job evaluation and badging systems, and/or post-credential monitoring systems. For example, within any automated badging/certification/verification system, data identifying particular physical user cues and/or user biometric data may be collected during testing/simulation/monitoring processes and saved, for example, in a digital credential platform server along with an associated issued digital credential and/or the associated user. Physical user cues may include, for example, facial expressions, user reactions and/or noises made by the user during testing/simulations, user body language, eye movement, and any other user behavior or reaction detectable via cameras and external sensors. Additionally or alternatively, various types of user biometric data also may be collected during the testing, simulation, and/or monitoring processes performed on the user. Such biometric data may include, for instance, the user's temperature, heartrate, blood pressure, respiration, skin conductivity, and brainwave activity, and/or any known types of biometric data that may collected during testing, credentialing, and/or monitoring processes.
- As discussed in more detail below, the user's physical cues and/or biometric data may be collected and saved within a digital credential platform server, and associated with the user, one or more particular digital credentials, and/or with the particular testing/simulation/monitoring processes during which the data was originally detected. Once collected, the data may be used to authenticate the testing, simulation, and/or monitoring processes, to confirm the user's identity and to prevent errors or fraudulent activities by users. The data may be saved with the user's digital credential and/or into a separate portfolio of evidence, which may be available to the user for review, and also may be provided upon request to potential employers for review during a review or hiring process. Such evidence data also may be applied to updated credentialing requirements, so that in some cases a user may simply resubmit their evidence portfolio instead of being required to recertify their digital credential when the test or credentialing standards are updated. In certain embodiments, the user's physical cues and/or biometric data also may be analyzed to determine the user's emotional states and reactions during the testing, simulation, and/or monitoring. Additionally or alternatively, the physical cues and biometric data may be detected for several users and analyzed collectively to provide feedback regarding the digital credential testing processes, simulations, monitoring, physical testing environments, etc.
- Referring now to
FIGS. 17A-17B , examples are shown illustrating facial recognition and analysis functionality that may be performed in connection with a user testing/credentialing process (live or simulation), or with user on-the-job credentialing or monitoring processes. In this example, one or more cameras may be configured to capture the user's facial features and expressions at different points during the testing/credentialing/monitoring processes. For tests performed within a simulation lab-type physical testing environment, a number of designated cameras may capture not only the user's face but also the user's body from several different angles. Thus, certain physical testing environments may be capable not only of capturing facial images of the user, but also detecting detailed facial expressions at different times during the test/simulation, and potentially eye movement patterns, gestures, body language, and any other non-verbal and non-written user expression data. - In other embodiments, such as for certain on-the-job credentialing or monitoring systems, or for formal testing/credentialing when sophisticated high-tech physical testing environments are not used, the physical cue data and/or biometrics data collected may be limited by the cameras and sensors available. In some cases, a laptop camera or webcam installed at the user's workstation may be use to capture facial images and/or to recognize facial expressions at different times during the testing/monitoring. However, such cameras may or may not have the resolution and image capture capabilities to perform advanced facial expression monitoring, eye movement, and/or body language detection. In other examples, such as on-the-job credentialing and monitoring scenarios, facial images might only be detectable using lower-quality security cameras or the like that are configured to monitor an entire floor or workspace. In such examples, the facial images may be still be useful for certain purposes (e.g., confirmation of user identification), but potential may be unsuitable for facial expression analysis, eye movement analysis, and the like.
- Additionally or alternatively, physical testing environments (e.g., simulation labs) and/or workstation or workplace monitoring systems may include various biometric sensors configured to detect biometric data of the user at different times during the test/simulation. As noted above, such biometric data may include the user's temperature, heartrate, blood pressure, respiration, skin conductivity, and brainwave activity, and/or any known types of biometric data. Thus, the biometric metric may be detected and captured via a combination of external sensors, wearable sensors, and/or implanted sensors in some cases. For on-the-job credentialing and monitoring, mobile wearable sensors such as heartrate monitors, step trackers, and the like, may be used when more advanced wearable sensors (e.g., blood pressure, respiration, skin conductivity, brainwave activity, etc.) are not practical.
- Referring now to
FIG. 18 , a flow diagram is shown illustrating an express process of collecting physical cue data and/or biometric data for a user during a user testing, credentialing, or monitoring processes, and using the physical cue and biometrics to authenticate the user's identity and the associated data. The process shown in this example may be implemented within any of the testing/credentialing systems, simulators, workstation or workplace monitoring systems, and the like described herein. Instep 1801, a testing, credentialing, and/or monitoring system such as those described above may execute a test, simulation, or user monitoring process for a particular user in connection with a digital credential that the user is seeking or has already obtained. The particular types of tests may include, for example, live simulations and/or virtual or augmented reality simulations executed within aphysical testing environment 700. In other examples, the testing instep 1801 may correspond to an on-the-job credentialing system that monitors and evaluates a user's workplace tasks and activities, or to a post-credentialing user monitoring system configured to determine whether the user is using their previously issued digital credentials. Instep 1802, during any of these testing, simulation, or monitoring processes, one or more of the user monitoring devices described above, including cameras, microphones, motion sensors, tracking devices, and/or user biometrics sensors, may capture physical cues from the user and/or biometric data of the user during the testing, simulation, or monitoring processes. Such physical cues may include particular facial expressions, user reactions and/or noises made by the user during testing/simulations/monitoring, as well as user body language and eye movements. Instep 1803, the physical cue and user biometric data may be encapsulated and transmitted to the transmitted to theplatform server 1410. In other embodiments, certain systems (e.g., 1421-1423) may retain and store user's physical cues and biometrics data locally, rather than the evidence data being stored in a central repository. Regardless of storage location, the physical cues and biometrics data of the user may be associated with particular test questions and/or particular time stamps during a testing or simulation. Additionally, when the data is transmitted, it may be compressed and edited as needed, and/or encrypted in order to assure data security and user privacy. - In some embodiments, the
platform server 1410 may use the physical cues and/or biometrics data collected for the user as part of an authentication process instep 1804. For example, during any testing/credentialing process (e.g., written testing, computer-based testing, simulation lab testing, etc.) the user's facial images, physical cues, and/or biometrics may be compared against previously stored corresponding data (e.g., user images, physical cue patterns, biometrics, etc.) in order to verify that the correct user is taking the test/simulation. Additionally, the user's physical cues and biometrics may provide an additional level of authentication, by comparing the observed physical cues and biometrics at particular times during the test or simulation to expected physical cues and biometrics, based on what is happening during the test or simulation at that particular time. For instance, a simulation may be designed to present a challenging and stressful situation to the user at a particular timestamp or within a sequence of tasks the user is performed. Instep 1804, the server may compare the user's observed physical cues and biometrics to the physical cues and biometrics that would be expected for the challenging and stressful situation, in order to confirm that the data is valid and/or that the user did not expect this situation in advance (e.g., indicating cheating). Instep 1805, the platform serving 1410 having validated the user's identity and the authenticity of the user's physical cues and biometrics, may store the testing, credentialing, monitoring data in the digital credential data store as valid data. In some embodiments, the image data, facial cues, and/or biometrics data also may be retained and stored by the platform server for future analysis. - In some embodiments, the data relating to the user's physical cues and biometrics collected during a test, simulation, or during on-the-job monitoring, may be further evaluated to identify the user's emotional states at different times. For instance, certain simulations may be specifically designed to invoke certain emotional states (e.g., anger, boredom, frustration, surprise, etc.), and the user's level of performance while experiencing those emotional states may be particularly important for certain testing/credentialing processes. In these examples and other cases, either exhibiting or not exhibiting particular emotion states may be an eligibility requirement for a credential receiver to obtain certain types of digital credentials. Thus, the data collected during the test, simulation, or monitoring in
step 1801 may be used not only for user identification/authentication, but also may be analyzed to (1) determine the user's emotional state at different times during the test, simulation, or monitoring, (2) compare that emotional state to an expected emotional state based on what the user is experiencing, and (3) evaluate the user's reactions, levels of skills performance during different emotional states. - Additionally, in some embodiments, the physical cues, biometrics data, and/or emotional states detected for multiple users may be aggregated for the same tests, simulations, monitoring environments, etc. The aggregated data for tests may be used to revise current tests and simulations, design new tests and simulations, and for training users how to respond to particular scenarios and situations (e.g., workplace accidents).
- Certain aspects described herein relate to an automatic recruiting engine and corresponding functionality that may analyze available data within a digital credential platform and determine matches between individuals (e.g., digital credential receivers) and potential employers. For example, in some embodiments, an automatic recruiting engine may perform some or all of the tasks of a professional recruiter, and may take the place of a recruiter role or a job seeker role by proactively performing matching analyses between available/potential job listings and potential candidates for those jobs. The automatic recruiting engine and/or associated digital credential platform may provide notifications to users (e.g., credential receivers) of job listings for which the user is qualified, and/or notifications to employers that may be looking candidates having a specific skill set and/or matching a specific profile. In certain embodiments, automatic recruiting engine may also include functionality to provide credential receivers with suggestions of particular digital credentials that may expand their job prospects, based on analyses of the current job market/job listings and the user's digital credential portfolio and other qualifications. An advanced skills-based or digital credential-based matching analysis may be performed, but in certain examples the analysis may include additional factors such as age, seniority, worker career arc, personal candidate data, location, salary expectations, etc.
- Referring now to
FIG. 19 , anexample computing environment 1900 is shown, including a digitalcredential platform server 1910, in communication withemployer client devices 1960, digitalcredential issuer devices 1930, and/orbade earner devices 1940. In some examples, the digitalcredential platform server 1910 may be a digital credential server similar or identical to thedigital credential server 610 discussed above. Thus,server 1910 may be configured as a digital credential repository and credentialing system, acting as a clearinghouse for digital credential owners, issuers, earners, endorsers, etc. As shown in this example,server 1910 may include anautomated recruiting engine 1915 configured to perform, among other tasks, an automated matching analysis between potential employees and jobs/employers. In some embodiments, therecruiting engine 1915 may retrieve and analyze worker-related and/or job related data from various data stores, which are shown as internal data stores or database in this example but may be implemented as external data sources in other cases. For instance, digital credential data stores may be configured to store credentialing information such as the details of the particular digital credentials earned by particular receivers (e.g., digital credential portfolios), including any combination of the digital credential data and associated data discussed above. Additionally, a user data store may store various user data for users who have earned digital credentials stored within the digital credential data store. Such user data may include demographic data, employment and educational data, other qualifications, current employment details, current salary and salary preferences, current hours/lifestyle and hours/lifestyle preferences, current work satisfaction, current work location and work location preferences, etc. Additionally, a job listing database may include recent/current jobs postings from various employers. Jobs postings in this database may include a variety of job description data, qualification requirements, and other factors, include data such as job title, job description, required digital credentials, required skills, required educational qualifications, required abilities/traits, job location, job hours/days commitment, job salary or range, company information, and the like. Additionally, in some embodiments data from an employment database also may be used in the matching analyses performed, including data relating to the jobs/positions currently held by workers at different companies. Such employment data may include, for a company's current workforce, current positions held, salaries, locations, job descriptions, skills, requirements, qualifications, technologies used, current job satisfaction level, etc. - In this example, the digital
credential platform server 1910 may receive such data from various external data sources, includingemployer devices 1960, digitalcredential issuer devices 1930, and digitalcredential receiver devices 1940, as well as other external data sources including job data, employment market data, technical/skills data, etc. As discussed below in more detail, theautomated recruiting engine 1915 withinserver 1910 may use various matching algorithms, analytics engines, and/or artificial intelligence components to analyze the data and identify potential matches between workers and jobs/companies. - Referring now to
FIG. 20 , a flow diagram shown illustrating an example process of analyzing worker/employee data, and job/company data, in order to determine potential matches that may be suggested to one or both parties. Thus, in some embodiments, this process may be performed by digitalcredential platform server 1910, using anautomated recruiting engine 1915 to retrieve and analyze data in order to determine possible matches. Instep 2001, therecruiting engine 1915 may retrieve digital credential data, user data, employment data, and/or job listing data relating to one or more employers and users within thedigital credential network 1900. The data retrieved instep 2001 may include any or all of the data from the various data sources discussed above. Instep 2002, therecruiting engine 1915 may analyze the data retrieved instep 2001 to determine one or more potential matches between workers and jobs. As noted above, the analysis instep 2002 may use data matching algorithms, analytics engines, and/or artificial intelligence components to analyze the data and identify potential matches between workers and jobs. In some cases, the analysis instep 2002 may be limited to only job seekers, while in other cases the analysis may include currently employed workers (e.g., so the recruiting engine may potentially suggest a new job or career change). Additionally, in some cases, the analysis instep 2002 may be limited to only existing job postings from employers, while in other cases the analysis may match a worker to an employer even if that employer has is not currently hiring. The matching instep 2002 may be based on any combination of the user data, digital credential/skills data, job data, and employment data discussed above. For instance, therecruiting engine 1915 may even include data points such as the satisfaction level of a current employee which may be determined based on employee surveys or other direct feedback, or inferred based on job seeking/web-browsing behavior, a decline in performance, and/or recently acquiring new digital credentials or other qualifications which may indicate an intention to change jobs or careers. For example, the platform server instep 2002 may determine a set of capabilities of the credential receiver, based on the digital credentials issued to the credential receiver, and then determine a matching set of field data objects (e.g., job listings) having capability characteristics matching the determined set of capabilities of the credential receiver. - Additional factors such as salary matching, location matching, career arc projections, lifestyle matching (e.g., hours, stress, dress code, corporate culture), also may be used along with digital credentials and skills matching, to determine candidates for jobs/companies and vice versa. In
step 2003, theautomated recruiting engine 1915 may determine whether or not to transmit notifications based on the worker-to-job/company matches identified instep 2002. In some cases, the determination of whether to transmit a notification may be based on the strength of the match (e.g., a high correlation between the worker characteristics and job listing characteristics greater than a similarly threshold would trigger notifications). The determination also may be based on whether or not the individual user and/or employer has requested or subscribed for such notifications, including any specific criteria provided by the worker (e.g., only notify me of my top 3 matches per week, only notify me of jobs in California, only notify me of jobs with a 15% salary increase, only notify me of jobs that use my most recent acquired digital credential-related skills, etc.) and/or specific criteria provided by the employer (e.g., only notify us for matches of current employees of a competitor company, only notify us for matches who have digital credential ABC, etc.). Instep 2004, the notifications determined in step 2003 (if any) may be generated by theautomated recruiting engine 1915 and transmitted from the digital credential platform server 2010 to the appropriate party. In some cases, such notifications may inform the worker or employee of the potential match, but might not reveal the identity of the matching counterparty until a later time (e.g., until both parties except the match or until the party has paid their subscription fee, etc.). For instance, a digital credential platform server may transmit a notification to the credential receiver identifying correlation metrics for one or more field data objects, but the notification purposefully might not identify selected field data object. Similarly, the digital credential platform server may transmit a second notification to an owner associated with a selected field data object, wherein the second notification identifies the correlation metric for the selected field data object but does not identify the credential receiver. - A number of variations and modifications of the disclosed embodiments can also be used. Specific details are given in the above description to provide a thorough understanding of the embodiments. However, it is understood that the embodiments may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.
- Implementation of the techniques, blocks, steps and means described above may be done in various ways. For example, these techniques, blocks, steps and means may be implemented in hardware, software, or a combination thereof. For a hardware implementation, the processing units may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described above, and/or a combination thereof.
- Also, it is noted that the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a swim diagram, a data flow diagram, a structure diagram, or a block diagram. Although a depiction may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process is terminated when its operations are completed, but could have additional steps not included in the figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function.
- Furthermore, embodiments may be implemented by hardware, software, scripting languages, firmware, middleware, microcode, hardware description languages, and/or any combination thereof. When implemented in software, firmware, middleware, scripting language, and/or microcode, the program code or code segments to perform the necessary tasks may be stored in a machine readable medium such as a storage medium. A code segment or machine-executable instruction may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a script, a class, or any combination of instructions, data structures, and/or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, and/or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.
- For a firmware and/or software implementation, the methodologies may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. Any machine-readable medium tangibly embodying instructions may be used in implementing the methodologies described herein. For example, software codes may be stored in a memory. Memory may be implemented within the processor or external to the processor. As used herein the term “memory” refers to any type of long term, short term, volatile, nonvolatile, or other storage medium and is not to be limited to any particular type of memory or number of memories, or type of media upon which memory is stored.
- Moreover, as disclosed herein, the term “storage medium” may represent one or more memories for storing data, including read only memory (ROM), random access memory (RAM), magnetic RAM, core memory, magnetic disk storage mediums, optical storage mediums, flash memory devices and/or other machine readable mediums for storing information. The term “machine-readable medium” includes, but is not limited to portable or fixed storage devices, optical storage devices, and/or various other storage mediums capable of storing that contain or carry instruction(s) and/or data.
- While the principles of the disclosure have been described above in connection with specific apparatuses and methods, it is to be clearly understood that this description is made only by way of example and not as limitation on the scope of the disclosure.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/993,555 US20190087832A1 (en) | 2017-09-15 | 2018-05-30 | Digital credential field data mapping |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201762559433P | 2017-09-15 | 2017-09-15 | |
US15/993,555 US20190087832A1 (en) | 2017-09-15 | 2018-05-30 | Digital credential field data mapping |
Publications (1)
Publication Number | Publication Date |
---|---|
US20190087832A1 true US20190087832A1 (en) | 2019-03-21 |
Family
ID=65719286
Family Applications (12)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/993,557 Abandoned US20190087833A1 (en) | 2017-09-15 | 2018-05-30 | Digital credential receiver performance model |
US15/993,394 Active 2040-08-13 US11341508B2 (en) | 2017-09-15 | 2018-05-30 | Automatically certifying worker skill credentials based on monitoring worker actions in a virtual reality simulation environment |
US15/993,546 Pending US20190087830A1 (en) | 2017-09-15 | 2018-05-30 | Generating digital credentials with associated sensor data in a sensor-monitored environment |
US15/993,410 Abandoned US20190089691A1 (en) | 2017-09-15 | 2018-05-30 | Generating digital credentials based on actions in a sensor-monitored environment |
US15/993,558 Pending US20190087834A1 (en) | 2017-09-15 | 2018-05-30 | Digital credential analysis in a digital credential platform |
US15/993,555 Abandoned US20190087832A1 (en) | 2017-09-15 | 2018-05-30 | Digital credential field data mapping |
US15/993,553 Pending US20190089692A1 (en) | 2017-09-15 | 2018-05-30 | Time-based degradation of digital credentials in a digital credential platform |
US15/993,549 Abandoned US20190087831A1 (en) | 2017-09-15 | 2018-05-30 | Generating digital credentials based on sensor feedback data |
US16/123,831 Active 2039-02-07 US10885530B2 (en) | 2017-09-15 | 2018-09-06 | Digital credentials based on personality and health-based evaluation |
US16/123,870 Abandoned US20190087558A1 (en) | 2017-09-15 | 2018-09-06 | Digital credential system for evaluating skills-based profiles |
US16/123,910 Active 2038-12-10 US11042885B2 (en) | 2017-09-15 | 2018-09-06 | Digital credential system for employer-based skills analysis |
US17/187,425 Pending US20210192539A1 (en) | 2017-09-15 | 2021-02-26 | Time-based degradation of digital credentials in a digital credential platform |
Family Applications Before (5)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/993,557 Abandoned US20190087833A1 (en) | 2017-09-15 | 2018-05-30 | Digital credential receiver performance model |
US15/993,394 Active 2040-08-13 US11341508B2 (en) | 2017-09-15 | 2018-05-30 | Automatically certifying worker skill credentials based on monitoring worker actions in a virtual reality simulation environment |
US15/993,546 Pending US20190087830A1 (en) | 2017-09-15 | 2018-05-30 | Generating digital credentials with associated sensor data in a sensor-monitored environment |
US15/993,410 Abandoned US20190089691A1 (en) | 2017-09-15 | 2018-05-30 | Generating digital credentials based on actions in a sensor-monitored environment |
US15/993,558 Pending US20190087834A1 (en) | 2017-09-15 | 2018-05-30 | Digital credential analysis in a digital credential platform |
Family Applications After (6)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/993,553 Pending US20190089692A1 (en) | 2017-09-15 | 2018-05-30 | Time-based degradation of digital credentials in a digital credential platform |
US15/993,549 Abandoned US20190087831A1 (en) | 2017-09-15 | 2018-05-30 | Generating digital credentials based on sensor feedback data |
US16/123,831 Active 2039-02-07 US10885530B2 (en) | 2017-09-15 | 2018-09-06 | Digital credentials based on personality and health-based evaluation |
US16/123,870 Abandoned US20190087558A1 (en) | 2017-09-15 | 2018-09-06 | Digital credential system for evaluating skills-based profiles |
US16/123,910 Active 2038-12-10 US11042885B2 (en) | 2017-09-15 | 2018-09-06 | Digital credential system for employer-based skills analysis |
US17/187,425 Pending US20210192539A1 (en) | 2017-09-15 | 2021-02-26 | Time-based degradation of digital credentials in a digital credential platform |
Country Status (2)
Country | Link |
---|---|
US (12) | US20190087833A1 (en) |
WO (1) | WO2019055286A1 (en) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110377702A (en) * | 2019-07-18 | 2019-10-25 | 中科鼎富(北京)科技发展有限公司 | A kind of biographic information matching process and device |
US10812502B2 (en) * | 2018-02-06 | 2020-10-20 | Bank Of America Corporation | Network device owner identification and communication triggering system |
US10819731B2 (en) | 2018-02-06 | 2020-10-27 | Bank Of America Corporation | Exception remediation logic rolling platform |
US10862915B2 (en) * | 2018-02-06 | 2020-12-08 | Bank Of America Corporation | Exception remediation logic routing and suppression platform |
WO2020249568A1 (en) * | 2019-06-14 | 2020-12-17 | Interdigital Ce Patent Holdings | Method and apparatus for associating a first device with a second device |
US11089042B2 (en) | 2018-02-06 | 2021-08-10 | Bank Of America Corporation | Vulnerability consequence triggering system for application freeze and removal |
US11265340B2 (en) | 2018-02-06 | 2022-03-01 | Bank Of America Corporation | Exception remediation acceptable use logic platform |
US20220116467A1 (en) * | 2020-10-09 | 2022-04-14 | Olive AI, Inc. | Methods and apparatus for controlling the presentation of data through an operator interface by monitoring user interactions across various compute devices |
Families Citing this family (118)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140353369A1 (en) * | 2013-06-03 | 2014-12-04 | Ginger G. Malin | Method and System for Issuing, Managing, Verifying and Displaying |
CN111081345B (en) | 2014-09-02 | 2023-10-10 | 苹果公司 | Physical activity and fitness monitor |
US11903680B2 (en) | 2015-06-14 | 2024-02-20 | Facense Ltd. | Wearable-based health state verification for physical access authorization |
EP4321088A3 (en) | 2015-08-20 | 2024-04-24 | Apple Inc. | Exercise-based watch face |
AU2017100667A4 (en) | 2016-06-11 | 2017-07-06 | Apple Inc. | Activity and workout updates |
US10736543B2 (en) | 2016-09-22 | 2020-08-11 | Apple Inc. | Workout monitor interface |
WO2018156809A1 (en) * | 2017-02-24 | 2018-08-30 | Masimo Corporation | Augmented reality system for displaying patient data |
WO2018156804A1 (en) | 2017-02-24 | 2018-08-30 | Masimo Corporation | System for displaying medical monitoring data |
US10845955B2 (en) * | 2017-05-15 | 2020-11-24 | Apple Inc. | Displaying a scrollable list of affordances associated with physical activities |
US20190087833A1 (en) | 2017-09-15 | 2019-03-21 | Pearson Education, Inc. | Digital credential receiver performance model |
US11645593B2 (en) * | 2017-09-22 | 2023-05-09 | Johnson Controls Tyco IP Holdings LLP | Use of identity and access management for service provisioning |
US10829088B2 (en) | 2017-09-22 | 2020-11-10 | Sensormatic Electronics, LLC | Identity management for implementing vehicle access and operation management |
US10453319B2 (en) | 2017-09-22 | 2019-10-22 | Sensormatic Electronics, LLC | Methods and apparatus for management of intrusion detection systems using verified identity |
US11055802B2 (en) | 2017-09-22 | 2021-07-06 | Sensormatic Electronics, LLC | Methods and apparatus for implementing identity and asset sharing management |
US20190197461A1 (en) * | 2017-12-27 | 2019-06-27 | Pearson Education, Inc. | On-demand utilization of proctoring resources |
DK201870599A1 (en) | 2018-03-12 | 2019-10-16 | Apple Inc. | User interfaces for health monitoring |
CN110990407B (en) * | 2018-04-27 | 2020-11-10 | 腾讯科技(深圳)有限公司 | Block chain based data storage method and device, server and storage medium |
US11317833B2 (en) | 2018-05-07 | 2022-05-03 | Apple Inc. | Displaying user interfaces associated with physical activities |
US10565229B2 (en) | 2018-05-24 | 2020-02-18 | People.ai, Inc. | Systems and methods for matching electronic activities directly to record objects of systems of record |
US11924297B2 (en) | 2018-05-24 | 2024-03-05 | People.ai, Inc. | Systems and methods for generating a filtered data set |
US11463441B2 (en) | 2018-05-24 | 2022-10-04 | People.ai, Inc. | Systems and methods for managing the generation or deletion of record objects based on electronic activities and communication policies |
US11276317B2 (en) * | 2018-07-16 | 2022-03-15 | David ZEILER | System for career technical education |
US11200625B2 (en) * | 2018-08-10 | 2021-12-14 | International Business Machines Corporation | Dynamic modification of user skill profile using determined crowdsourced social presence |
US10873577B2 (en) * | 2018-08-17 | 2020-12-22 | Evgeny Chereshnev | Identifying and authorizing user data over a network based on biometric and statistical data |
KR102511260B1 (en) * | 2018-08-21 | 2023-03-21 | 엘지전자 주식회사 | Vehicle user interface device and service information provision system |
US11032063B2 (en) | 2018-09-19 | 2021-06-08 | International Business Machines Corporation | Distributed platform for computation and trusted validation |
US11940978B2 (en) * | 2018-09-19 | 2024-03-26 | International Business Machines Corporation | Distributed platform for computation and trusted validation |
US11212076B2 (en) | 2018-09-19 | 2021-12-28 | International Business Machines Corporation | Distributed platform for computation and trusted validation |
US11170335B2 (en) * | 2018-09-28 | 2021-11-09 | Accenture Global Solutions Limited | Adaptive artificial intelligence for user training and task management |
AU2019354735A1 (en) * | 2018-10-02 | 2021-06-03 | Mutualink, Inc. | Consensus-based voting for network member identification employing blockchain-based identity signature mechanisms |
US20200111043A1 (en) * | 2018-10-04 | 2020-04-09 | Milan Cheeks | System and method for providing personalized planning |
US20200117788A1 (en) * | 2018-10-11 | 2020-04-16 | Ncr Corporation | Gesture Based Authentication for Payment in Virtual Reality |
US11093479B2 (en) | 2018-11-06 | 2021-08-17 | Workday, Inc. | Ledger data generation and storage for trusted recall of professional profiles |
US11327950B2 (en) * | 2018-11-06 | 2022-05-10 | Workday, Inc. | Ledger data verification and sharing system |
US11449791B2 (en) * | 2018-11-16 | 2022-09-20 | Cognizant Technology Solutions India Pvt. Ltd. | System and method for monitoring lab processes and predicting their outcomes |
US10282248B1 (en) | 2018-11-27 | 2019-05-07 | Capital One Services, Llc | Technology system auto-recovery and optimality engine and techniques |
US10275331B1 (en) | 2018-11-27 | 2019-04-30 | Capital One Services, Llc | Techniques and system for optimization driven by dynamic resilience |
US11087577B2 (en) * | 2018-12-14 | 2021-08-10 | Johnson Controls Tyco IP Holdings LLP | Systems and methods of secure pin code entry |
US11616816B2 (en) * | 2018-12-28 | 2023-03-28 | Speedchain, Inc. | Distributed ledger based document image extracting and processing within an enterprise system |
US20210329036A1 (en) * | 2018-12-28 | 2021-10-21 | Speedchain, Inc. | Reconciliation Digital Facilitators in a Distributed Network |
US10958637B2 (en) | 2018-12-28 | 2021-03-23 | Mox-SpeedChain, LLC | Preselected issuance and data operations loops in a hybrid distributed network ecosystem |
US20200279205A1 (en) * | 2019-02-28 | 2020-09-03 | Autodesk, Inc. | Techniques for tailoring fabrication environments based on user, task, and expertise |
US20200280550A1 (en) * | 2019-02-28 | 2020-09-03 | Nok Nok Labs, Inc. | System and method for endorsing a new authenticator |
US10728443B1 (en) | 2019-03-27 | 2020-07-28 | On Time Staffing Inc. | Automatic camera angle switching to create combined audiovisual file |
US10963841B2 (en) | 2019-03-27 | 2021-03-30 | On Time Staffing Inc. | Employment candidate empathy scoring system |
US20200327556A1 (en) * | 2019-04-12 | 2020-10-15 | Salesforce.Com, Inc. | Method to accept certifications with blockchain transactions |
DK201970532A1 (en) | 2019-05-06 | 2021-05-03 | Apple Inc | Activity trends and workouts |
US20220207535A1 (en) * | 2019-05-30 | 2022-06-30 | Jeffrey B. Miller | System and method of incorporating confidential and unique information into protection measures, with real time evaluations of environmental conditions and exposure unique to each user |
JP7297940B2 (en) * | 2019-06-01 | 2023-06-26 | アップル インコーポレイテッド | Multimodal activity tracking user interface |
US11809618B2 (en) * | 2019-06-01 | 2023-11-07 | Apple Inc. | Controlling a user selection queue |
US20200394933A1 (en) * | 2019-06-13 | 2020-12-17 | International Business Machines Corporation | Massive open online course assessment management |
US20240013669A1 (en) * | 2019-06-14 | 2024-01-11 | Quantum Interface Llc | Predictive virtual training systems, apparatuses, interfaces, and methods for implementing same |
CN111164594B (en) | 2019-07-02 | 2023-08-25 | 创新先进技术有限公司 | System and method for mapping a de-centralized identity to a real entity |
WO2019179534A2 (en) | 2019-07-02 | 2019-09-26 | Alibaba Group Holding Limited | System and method for creating decentralized identifiers |
CN111213147B (en) * | 2019-07-02 | 2023-10-13 | 创新先进技术有限公司 | Systems and methods for blockchain-based cross-entity authentication |
WO2019179533A2 (en) | 2019-07-02 | 2019-09-26 | Alibaba Group Holding Limited | System and method for issuing verifiable claims |
CN111316303B (en) | 2019-07-02 | 2023-11-10 | 创新先进技术有限公司 | Systems and methods for blockchain-based cross-entity authentication |
CN111095327B (en) | 2019-07-02 | 2023-11-17 | 创新先进技术有限公司 | System and method for verifying verifiable claims |
CN111091731B (en) * | 2019-07-11 | 2021-11-26 | 广东小天才科技有限公司 | Dictation prompting method based on electronic equipment and electronic equipment |
IT201900016859A1 (en) * | 2019-09-20 | 2021-03-20 | Leonardo Spa | SYSTEM AND METHOD OF EVALUATING THE TRAINING OF AN OPERATOR IN A REAL-TIME TRAINING MISSION |
US11762809B2 (en) * | 2019-10-09 | 2023-09-19 | Capital One Services, Llc | Scalable subscriptions for virtual collaborative workspaces |
US20210136064A1 (en) * | 2019-10-30 | 2021-05-06 | Governor's Office of Information Technology | Secure use of authoritative data within biometry based digital identity authentication and verification |
TWI742463B (en) * | 2019-11-13 | 2021-10-11 | 宏正自動科技股份有限公司 | Surveillance system |
US11127232B2 (en) | 2019-11-26 | 2021-09-21 | On Time Staffing Inc. | Multi-camera, multi-sensor panel data extraction system and method |
CN113160003A (en) * | 2020-01-07 | 2021-07-23 | 钟苑菁 | Paired learning system |
US20210216942A1 (en) * | 2020-01-10 | 2021-07-15 | International Business Machines Corporation | Assessing an individual's competencies through social network analysis |
WO2021146150A1 (en) * | 2020-01-13 | 2021-07-22 | Paxmentys, LLC | Cognitive readiness determination and control system and method |
US11373145B2 (en) * | 2020-01-29 | 2022-06-28 | International Business Machines Corporation | Technology for candidate insight evaluation |
DK202070616A1 (en) | 2020-02-14 | 2022-01-14 | Apple Inc | User interfaces for workout content |
US11023735B1 (en) | 2020-04-02 | 2021-06-01 | On Time Staffing, Inc. | Automatic versioning of video presentations |
US11822881B1 (en) | 2020-04-29 | 2023-11-21 | Trueblue, Inc. | Recommendation platform for skill development |
US20230170079A1 (en) * | 2020-05-01 | 2023-06-01 | Healthpointe Solutions, Inc. | Method to build a trust chain of testing or dispensation of medical consultation in a medical network |
US20210383392A1 (en) * | 2020-06-05 | 2021-12-09 | Capital One Services, Llc | Systems and methods for fraud detection and prevention |
US11875242B2 (en) * | 2020-07-28 | 2024-01-16 | Ncs Pearson, Inc. | Systems and methods for risk analysis and mitigation with nested machine learning models for exam registration and delivery processes |
US11854103B2 (en) | 2020-07-28 | 2023-12-26 | Ncs Pearson, Inc. | Systems and methods for state-based risk analysis and mitigation for exam registration and delivery processes |
CN113064955B (en) * | 2020-08-26 | 2022-02-25 | 视伴科技(北京)有限公司 | Method and device for displaying geographic marking information |
TWI735322B (en) * | 2020-08-28 | 2021-08-01 | 第一商業銀行股份有限公司 | Operation authority management method |
US11494718B2 (en) * | 2020-09-01 | 2022-11-08 | International Business Machines Corporation | Runbook deployment based on confidence evaluation |
US11144882B1 (en) | 2020-09-18 | 2021-10-12 | On Time Staffing Inc. | Systems and methods for evaluating actions over a computer network and establishing live network connections |
US20220101239A1 (en) * | 2020-09-25 | 2022-03-31 | Agency 360, Llc | Technologies for coordinating public safety employee management programs |
CN111985582B (en) * | 2020-09-27 | 2021-06-01 | 上海松鼠课堂人工智能科技有限公司 | Knowledge point mastering degree evaluation method based on learning behaviors |
US11575680B1 (en) * | 2020-09-28 | 2023-02-07 | Amazon Technologies, Inc. | Data modeling to improve security |
US11550832B2 (en) * | 2020-10-02 | 2023-01-10 | Birchhoover Llc | Systems and methods for micro-credential accreditation |
US11487891B2 (en) * | 2020-10-14 | 2022-11-01 | Philip Chidi Njemanze | Method and system for mental performance computing using artificial intelligence and blockchain |
WO2022104379A1 (en) * | 2020-11-15 | 2022-05-19 | Pricewaterhousecoopers Llp | Systems, methods, and user interfaces for a web-based personalized upskilling platform including visualizing role data and workforce data |
GB2601165A (en) * | 2020-11-20 | 2022-05-25 | Wallife S R L | Transaction verification |
US11113641B1 (en) * | 2020-12-14 | 2021-09-07 | Fmr Llc | Systems and methods for access control governance recommendation |
WO2022154847A1 (en) | 2021-01-12 | 2022-07-21 | Emed Labs, Llc | Health testing and diagnostics platform |
US11361062B1 (en) * | 2021-03-02 | 2022-06-14 | Bank Of America Corporation | System and method for leveraging microexpressions of users in multi-factor authentication |
US20220286519A1 (en) * | 2021-03-05 | 2022-09-08 | Valence Enterprises, Inc. | Method and system for dynamic valuation of member data in a network |
GB2604916A (en) * | 2021-03-19 | 2022-09-21 | Ibuildpro Holdings Ltd | A method for monitoring progress of mixed reality training |
US11373756B1 (en) | 2021-05-24 | 2022-06-28 | Emed Labs, Llc | Systems, devices, and methods for diagnostic aid kit apparatus |
US11929168B2 (en) | 2021-05-24 | 2024-03-12 | Emed Labs, Llc | Systems, devices, and methods for diagnostic aid kit apparatus |
US11615888B2 (en) | 2021-03-23 | 2023-03-28 | Emed Labs, Llc | Remote diagnostic testing and treatment |
EP4323992A1 (en) | 2021-05-15 | 2024-02-21 | Apple Inc. | User interfaces for group workouts |
EP4105803B1 (en) * | 2021-06-14 | 2023-10-25 | Tata Consultancy Services Limited | Method and system for personalized programming guidance using dynamic skill assessment |
WO2022271668A1 (en) | 2021-06-22 | 2022-12-29 | Emed Labs, Llc | Systems, methods, and devices for non-human readable diagnostic tests |
US11394758B1 (en) * | 2021-07-02 | 2022-07-19 | Rovi Guides, Inc. | Systems and methods for selecting a local device in a collaborative environment |
US20230008868A1 (en) * | 2021-07-08 | 2023-01-12 | Nippon Telegraph And Telephone Corporation | User authentication device, user authentication method, and user authentication computer program |
US20230015446A1 (en) * | 2021-07-17 | 2023-01-19 | Lauretta Zerelda WHITE | Systems and methods for storing, managing, verifying, and accrediting digital data |
US11785018B2 (en) * | 2021-07-29 | 2023-10-10 | Bank Of America Corporation | Mobile device management system for securely managing device communication |
US11620797B2 (en) | 2021-08-05 | 2023-04-04 | Bank Of America Corporation | Electronic user interface with augmented detail display for resource location |
US11727040B2 (en) | 2021-08-06 | 2023-08-15 | On Time Staffing, Inc. | Monitoring third-party forum contributions to improve searching through time-to-live data assignments |
US11423071B1 (en) | 2021-08-31 | 2022-08-23 | On Time Staffing, Inc. | Candidate data ranking method using previously selected candidate data |
US20230063441A1 (en) * | 2021-09-01 | 2023-03-02 | Emed Labs, Llc | Image processing and presentation techniques for enhanced proctoring sessions |
US11366963B1 (en) * | 2021-10-06 | 2022-06-21 | Capital One Services, Llc | Systems and methods for using machine learning models to organize and select modular components for user interface templates |
CN113965261B (en) * | 2021-12-21 | 2022-04-29 | 南京英田光学工程股份有限公司 | Measuring method by using space laser communication terminal tracking precision measuring device |
US11809594B2 (en) | 2022-01-24 | 2023-11-07 | My Job Matcher, Inc. | Apparatus and method for securely classifying applications to posts using immutable sequential listings |
US11646036B1 (en) | 2022-01-31 | 2023-05-09 | Humancore Llc | Team member identification based on psychographic categories |
US20230252416A1 (en) * | 2022-02-08 | 2023-08-10 | My Job Matcher, Inc. D/B/A Job.Com | Apparatuses and methods for linking action data to an immutable sequential listing identifier of a user |
WO2023171380A1 (en) * | 2022-03-07 | 2023-09-14 | ソニーグループ株式会社 | Information processing system, information processing device, and information processing method |
CN114610982B (en) * | 2022-04-06 | 2023-01-06 | 中咨数据有限公司 | Computer network data acquisition, analysis and management method, equipment and storage medium |
US11847660B2 (en) | 2022-05-13 | 2023-12-19 | Stynt Inc. | Apparatus for automatic credential classification |
US11907652B2 (en) | 2022-06-02 | 2024-02-20 | On Time Staffing, Inc. | User interface and systems for document creation |
US11977729B2 (en) | 2022-06-05 | 2024-05-07 | Apple Inc. | Physical activity information user interfaces |
US11896871B2 (en) | 2022-06-05 | 2024-02-13 | Apple Inc. | User interfaces for physical activity information |
US11893150B2 (en) | 2022-06-28 | 2024-02-06 | Bank Of America Corporation | Systems and methods for multi-point validation in communication network with associated virtual reality application layer |
US11893151B1 (en) * | 2022-07-25 | 2024-02-06 | Gravystack, Inc. | Apparatus for external activity verification and a method for its use |
Family Cites Families (418)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4413277A (en) * | 1981-01-23 | 1983-11-01 | Instant Replay Systems | Instant replay productivity motivation system |
US4425097A (en) * | 1981-09-08 | 1984-01-10 | Owens Lawrence L | Apparatus for training equipment operators |
US4671772A (en) * | 1985-10-22 | 1987-06-09 | Keilty, Goldsmith & Boone | Performance appraisal and training system and method of utilizing same |
US5212635A (en) * | 1989-10-23 | 1993-05-18 | International Business Machines Corporation | Method and apparatus for measurement of manufacturing technician efficiency |
US5103408A (en) * | 1990-01-16 | 1992-04-07 | Atlantic Richfield Company | Apparatus and method for determining the ability of an individual to perform a task |
US5239460A (en) * | 1991-01-03 | 1993-08-24 | At&T Bell Laboratories | Arrangement for motivating telemarketing agents |
US6850252B1 (en) | 1999-10-05 | 2005-02-01 | Steven M. Hoffberg | Intelligent electronic appliance system and method |
US5444226A (en) * | 1993-05-13 | 1995-08-22 | At&T Global Information Solutions Company | Real-time barcode scanning performance feedback system |
ES2138649T3 (en) * | 1993-09-22 | 2000-01-16 | Teknekron Infoswitch Corp | MONITORING OF TELECOMMUNICATION SYSTEMS. |
JPH08263438A (en) | 1994-11-23 | 1996-10-11 | Xerox Corp | Distribution and use control system of digital work and access control method to digital work |
US7613659B1 (en) * | 1994-11-28 | 2009-11-03 | Yt Acquisition Corporation | System and method for processing tokenless biometric electronic transmissions using an electronic rule module clearinghouse |
US5748738A (en) * | 1995-01-17 | 1998-05-05 | Document Authentication Systems, Inc. | System and method for electronic transmission, storage and retrieval of authenticated documents |
US6722888B1 (en) * | 1995-01-20 | 2004-04-20 | Vincent J. Macri | Method and apparatus for tutorial, self and assisted instruction directed to simulated preparation, training and competitive play and entertainment |
US5890906A (en) * | 1995-01-20 | 1999-04-06 | Vincent J. Macri | Method and apparatus for tutorial, self and assisted instruction directed to simulated preparation, training and competitive play and entertainment |
US5892900A (en) | 1996-08-30 | 1999-04-06 | Intertrust Technologies Corp. | Systems and methods for secure transaction management and electronic rights protection |
US7133845B1 (en) * | 1995-02-13 | 2006-11-07 | Intertrust Technologies Corp. | System and methods for secure transaction management and electronic rights protection |
US6948070B1 (en) | 1995-02-13 | 2005-09-20 | Intertrust Technologies Corporation | Systems and methods for secure transaction management and electronic rights protection |
EP1555591B1 (en) * | 1995-02-13 | 2013-08-14 | Intertrust Technologies Corp. | Secure transaction management |
US7143290B1 (en) * | 1995-02-13 | 2006-11-28 | Intertrust Technologies Corporation | Trusted and secure techniques, systems and methods for item delivery and execution |
US5963649A (en) | 1995-12-19 | 1999-10-05 | Nec Corporation | Message authorization system for authorizing message for electronic document |
US6688888B1 (en) * | 1996-03-19 | 2004-02-10 | Chi Fai Ho | Computer-aided learning system and method |
US5909669A (en) * | 1996-04-01 | 1999-06-01 | Electronic Data Systems Corporation | System and method for generating a knowledge worker productivity assessment |
US7483670B2 (en) * | 1996-05-09 | 2009-01-27 | Walker Digital, Llc | Method and apparatus for educational testing |
US5781732A (en) * | 1996-06-20 | 1998-07-14 | Object Technology Licensing Corp. | Framework for constructing shared documents that can be collaboratively accessed by multiple users |
US6157808A (en) | 1996-07-17 | 2000-12-05 | Gpu, Inc. | Computerized employee certification and training system |
US6513014B1 (en) | 1996-07-24 | 2003-01-28 | Walker Digital, Llc | Method and apparatus for administering a survey via a television transmission network |
ATE229214T1 (en) * | 1996-09-25 | 2002-12-15 | Sylvan Learning Systems Inc | AUTOMATIC TESTING AND ELECTRONIC SYSTEM FOR TEACHING THE TEACHING AND MANAGING STUDENTS |
US6246975B1 (en) | 1996-10-30 | 2001-06-12 | American Board Of Family Practice, Inc. | Computer architecture and process of patient generation, evolution, and simulation for computer based testing system |
US6056556A (en) * | 1997-03-05 | 2000-05-02 | Educational Testing Service | Computer-based simulation examination of architectural practice |
WO1998040126A1 (en) * | 1997-03-12 | 1998-09-17 | Neurocom International, Inc. | System and method for monitoring training programs |
US6033226A (en) * | 1997-05-15 | 2000-03-07 | Northrop Grumman Corporation | Machining tool operator training system |
US6735574B2 (en) * | 1997-11-05 | 2004-05-11 | Micron Technology, Inc. | Method and system for tracking employee productivity in a client/server environment |
US20150052075A1 (en) | 1997-11-17 | 2015-02-19 | Fidelis Education, Inc. | Learning relationship management system |
US6119097A (en) * | 1997-11-26 | 2000-09-12 | Executing The Numbers, Inc. | System and method for quantification of human performance factors |
US9792659B2 (en) | 1999-04-13 | 2017-10-17 | Iplearn, Llc | Computer-aided methods and apparatus to access materials in a network environment |
US6151675A (en) | 1998-07-23 | 2000-11-21 | Tumbleweed Software Corporation | Method and apparatus for effecting secure document format conversion |
US7188138B1 (en) | 1999-03-22 | 2007-03-06 | Eric Schneider | Method, product, and apparatus for resource identifier registration and aftermarket services |
US7760905B2 (en) | 1999-06-29 | 2010-07-20 | Digimarc Corporation | Wireless mobile phone with content processing |
US6988199B2 (en) * | 2000-07-07 | 2006-01-17 | Message Secure | Secure and reliable document delivery |
US6431875B1 (en) * | 1999-08-12 | 2002-08-13 | Test And Evaluation Software Technologies | Method for developing and administering tests over a network |
CN1276612C (en) | 1999-09-07 | 2006-09-20 | 索尼公司 | Contents management system, device, method, and program storage medium |
US20020095389A1 (en) * | 1999-10-05 | 2002-07-18 | Gaines Robert Vallee | Method, apparatus and system for identity authentication |
US7630986B1 (en) | 1999-10-27 | 2009-12-08 | Pinpoint, Incorporated | Secure data interchange |
US6876991B1 (en) * | 1999-11-08 | 2005-04-05 | Collaborative Decision Platforms, Llc. | System, method and computer program product for a collaborative decision platform |
US7277925B2 (en) * | 1999-12-08 | 2007-10-02 | Warnock Kevin L | Internet document services |
US7505919B2 (en) | 1999-12-13 | 2009-03-17 | Richardson Mary L | Method and system for employment placement |
WO2001052054A2 (en) | 2000-01-14 | 2001-07-19 | Saba Software, Inc. | Method and apparatus for a business applications server |
AU2001238519A1 (en) | 2000-02-18 | 2001-08-27 | Vasco Data Security, Inc. | Field programmable smart card terminal and token device |
US20010047326A1 (en) | 2000-03-14 | 2001-11-29 | Broadbent David F. | Interface system for a mortgage loan originator compliance engine |
US7441263B1 (en) | 2000-03-23 | 2008-10-21 | Citibank, N.A. | System, method and computer program product for providing unified authentication services for online applications |
US20010032094A1 (en) | 2000-04-21 | 2001-10-18 | Swarupanda Ghosh | System and method for managing licensing information |
US7237114B1 (en) | 2000-04-26 | 2007-06-26 | Pronvest, Inc. | Method and system for signing and authenticating electronic documents |
US20020052896A1 (en) | 2000-05-23 | 2002-05-02 | Streit Jason Mitchell | Secure signature and date placement system |
US20020023140A1 (en) * | 2000-06-08 | 2002-02-21 | Hile John K. | Electronic document delivery system |
US7251728B2 (en) | 2000-07-07 | 2007-07-31 | Message Secure Corporation | Secure and reliable document delivery using routing lists |
US6792394B1 (en) * | 2000-07-31 | 2004-09-14 | Ncr Corporation | Method and apparatus for determining the retail performance metric of entry identification time |
US20040220815A1 (en) * | 2000-08-18 | 2004-11-04 | Johanne Belanger | Apparatus and method for the compilation, assembly, and distribution of product documentation and associated information |
US20030196092A1 (en) * | 2000-08-28 | 2003-10-16 | Contentguard Holdings, Inc. | Method and apparatus for sharing secure communications |
US7743259B2 (en) * | 2000-08-28 | 2010-06-22 | Contentguard Holdings, Inc. | System and method for digital rights management using a standard rendering engine |
JP4581200B2 (en) | 2000-08-31 | 2010-11-17 | ソニー株式会社 | Personal authentication system, personal authentication method, information processing apparatus, and program providing medium |
JP4556308B2 (en) | 2000-08-31 | 2010-10-06 | ソニー株式会社 | Content distribution system, content distribution method, information processing apparatus, and program providing medium |
US6871195B2 (en) * | 2000-09-13 | 2005-03-22 | E-Promentor | Method and system for remote electronic monitoring and mentoring of computer assisted performance support |
AR033954A1 (en) | 2000-09-26 | 2004-01-21 | Duwimax Internat S A | SELF-ADHESIVE SECURITY SEAL LABEL FOR DACTILAR PRINTS AND A.D.N. |
US7660902B2 (en) | 2000-11-20 | 2010-02-09 | Rsa Security, Inc. | Dynamic file access control and management |
WO2002052764A2 (en) * | 2000-12-27 | 2002-07-04 | Nettrust Israel Ltd. | Methods and systems for authenticating communications |
US7099849B1 (en) * | 2000-12-28 | 2006-08-29 | Rightsline, Inc. | Integrated media management and rights distribution apparatus |
US20040039704A1 (en) * | 2001-01-17 | 2004-02-26 | Contentguard Holdings, Inc. | System and method for supplying and managing usage rights of users and suppliers of items |
US7206765B2 (en) * | 2001-01-17 | 2007-04-17 | Contentguard Holdings, Inc. | System and method for supplying and managing usage rights based on rules |
CA2369274A1 (en) * | 2001-01-24 | 2002-07-24 | Roger L. Duerksen | Telephonic certification of electronic death registration |
US6589055B2 (en) * | 2001-02-07 | 2003-07-08 | American Association Of Airport Executives | Interactive employee training system and method |
US7302634B2 (en) * | 2001-03-14 | 2007-11-27 | Microsoft Corporation | Schema-based services for identity-based data access |
US20020143818A1 (en) * | 2001-03-30 | 2002-10-03 | Roberts Elizabeth A. | System for generating a structured document |
US7340058B2 (en) * | 2001-04-09 | 2008-03-04 | Lucent Technologies Inc. | Low-overhead secure information processing for mobile gaming and other lightweight device applications |
US20030187798A1 (en) | 2001-04-16 | 2003-10-02 | Mckinley Tyler J. | Digital watermarking methods, programs and apparatus |
US7251831B2 (en) | 2001-04-19 | 2007-07-31 | International Business Machines Corporation | Method and system for architecting a secure solution |
WO2002089093A1 (en) | 2001-05-01 | 2002-11-07 | National Student Clearinghouse | Method for communicating confidential educational information |
US7222086B2 (en) * | 2001-05-02 | 2007-05-22 | Ncr Corp. | Systems and methods for providing performance feedback to a cashier at a point-of-sale terminal |
US20020192624A1 (en) * | 2001-05-11 | 2002-12-19 | Darby David G. | System and method of testing cognitive function |
US6895503B2 (en) * | 2001-05-31 | 2005-05-17 | Contentguard Holdings, Inc. | Method and apparatus for hierarchical assignment of rights to documents and documents having such rights |
EP1402224A2 (en) * | 2001-06-08 | 2004-03-31 | Beamhit, LLC | Firearm laser training system and method facilitating firearm training for extended range targets with feedback of firearm control |
US7110525B1 (en) | 2001-06-25 | 2006-09-19 | Toby Heller | Agent training sensitive call routing system |
WO2003003329A1 (en) * | 2001-06-27 | 2003-01-09 | Fujitsu Limited | Data originality validating method and system |
US7505557B2 (en) * | 2006-01-30 | 2009-03-17 | Rapiscan Security Products, Inc. | Method and system for certifying operators of x-ray inspection systems |
US20030028494A1 (en) | 2001-08-06 | 2003-02-06 | King Shawn L. | Electronic document management system and method |
GB0119629D0 (en) * | 2001-08-10 | 2001-10-03 | Cryptomathic As | Data certification method and apparatus |
US6973196B2 (en) | 2001-08-15 | 2005-12-06 | Eastman Kodak Company | Authentic document and method of making |
US20030048294A1 (en) * | 2001-09-07 | 2003-03-13 | Switchboard Incorporated | System and method for the creation of interactive display ads |
US7305381B1 (en) * | 2001-09-14 | 2007-12-04 | Ricoh Co., Ltd | Asynchronous unconscious retrieval in a network of information appliances |
US20030070072A1 (en) | 2001-10-09 | 2003-04-10 | Nick Nassiri | System and method of identity and signature and document authentication using a video conference |
US6904131B2 (en) | 2001-11-30 | 2005-06-07 | David Weksel | System and method for delivering a message to a plurality of receivers in respective reception formats |
WO2003050715A1 (en) | 2001-12-10 | 2003-06-19 | Redbank Manor Pty Ltd | A system for secure distribution of electronic content and collection of fees |
US20030120589A1 (en) | 2001-12-20 | 2003-06-26 | The Police Foundation | Method and apparatus for risk analysis management and quality of service analysis |
JP2005516278A (en) * | 2002-01-23 | 2005-06-02 | クルーズ、スカイ | Method and system for transmitting and distributing information in a secret manner and for physically exemplifying information transmitted in an intermediate information storage medium |
US7424438B2 (en) | 2002-03-19 | 2008-09-09 | Marc Vianello | Apparatus and methods for providing career and employment services |
CA2378289A1 (en) * | 2002-03-22 | 2003-09-22 | Billwhiz Inc. | Method and system for document presentment between generic publishers and generic subscribers |
WO2003083739A1 (en) | 2002-03-28 | 2003-10-09 | Fujitsu Limited | System, method, and program for issuing certificate |
US20030191653A1 (en) * | 2002-04-05 | 2003-10-09 | Dani Birnbaum | Method for evaluating a test advertisement with redemptions of electronically distributed coupons |
US6772081B1 (en) | 2002-05-21 | 2004-08-03 | Data Recognition Corporation | Priority system and method for processing standardized tests |
US7860736B2 (en) | 2002-06-28 | 2010-12-28 | Accenture Global Services Gmbh | Course content development method and computer readable medium for business driven learning solutions |
US7702531B2 (en) * | 2002-06-28 | 2010-04-20 | Accenture Global Services Gmbh | Business driven learning solution particularly suitable for sales-oriented organizations |
US20040054893A1 (en) * | 2002-09-18 | 2004-03-18 | Anthony Ellis | Method and system for a file encryption and monitoring system |
JP4366916B2 (en) * | 2002-10-29 | 2009-11-18 | 富士ゼロックス株式会社 | Document confirmation system, document confirmation method, and document confirmation program |
WO2004040464A2 (en) | 2002-10-30 | 2004-05-13 | Vidius Inc. | A method and system for managing confidential information |
US7278168B1 (en) * | 2002-11-27 | 2007-10-02 | Adobe Systems Incorporated | Dynamic enabling of functionality in electronic document readers |
CA2412109A1 (en) * | 2002-12-19 | 2004-06-19 | Claude Choquet | Virtual simulator method and system for neuromuscular training and certification via a communication network |
WO2004058172A2 (en) | 2002-12-23 | 2004-07-15 | Gametech International, Inc. | Enhanced gaming system |
US20040131999A1 (en) | 2003-01-06 | 2004-07-08 | Dresnick Stephen J. | Computer-based system for instruction using multi-modal teaching methods |
US20040158476A1 (en) * | 2003-02-06 | 2004-08-12 | I-Sim, Llc | Systems and methods for motor vehicle learning management |
US7562215B2 (en) * | 2003-05-21 | 2009-07-14 | Hewlett-Packard Development Company, L.P. | System and method for electronic document security |
US20040248071A1 (en) * | 2003-06-06 | 2004-12-09 | Serguei Bedziouk | System and method for autonomous training |
AU2003902911A0 (en) * | 2003-06-11 | 2003-06-26 | The Commonwealth Of Australia | Credential communication device |
US7197161B2 (en) * | 2003-06-20 | 2007-03-27 | Xerox Corporation | Embedding information in images using two-layer conjugate screening |
US8112391B2 (en) | 2003-07-15 | 2012-02-07 | Accenture Global Services Gmbh | Synchronization of agent skill data |
US20050027568A1 (en) * | 2003-07-29 | 2005-02-03 | Dorris David W. | System, method and computer program product for managing patient information |
US8190468B1 (en) * | 2003-10-31 | 2012-05-29 | Verizon Laboratories Inc. | Personnel productivity indices |
US7752269B2 (en) | 2004-01-19 | 2010-07-06 | Avaya Inc. | Adhoc secure document exchange |
US20050177412A1 (en) | 2004-02-09 | 2005-08-11 | Hire The Winners, Llc | Automobile sales simulator |
US7676568B2 (en) * | 2004-03-08 | 2010-03-09 | Cisco Technology, Inc. | Centrally-controlled distributed marking of content |
US7097617B1 (en) | 2004-03-31 | 2006-08-29 | Wallace Lynn Smith | Method for diagnosis of pain relief probability through medical treatment |
US20050222899A1 (en) * | 2004-03-31 | 2005-10-06 | Satyam Computer Services Inc. | System and method for skill managememt of knowledge workers in a software industry |
US7237144B2 (en) | 2004-04-06 | 2007-06-26 | Hewlett-Packard Development Company, L.P. | Off-chip lockstep checking |
US9003548B2 (en) | 2004-04-13 | 2015-04-07 | Nl Systems, Llc | Method and system for digital rights management of documents |
CN1973469A (en) | 2004-05-03 | 2007-05-30 | 法格电子公司 | Managed credential issuance |
US20050288952A1 (en) | 2004-05-18 | 2005-12-29 | Davis Bruce L | Official documents and methods of issuance |
US7685516B2 (en) * | 2004-05-18 | 2010-03-23 | Ubs Ag | Creation of electronically processable signature files |
WO2006002288A2 (en) | 2004-06-23 | 2006-01-05 | Swallows, Llc | Method and system for recruiting for employee positions and employees bidding for the same |
US7860923B2 (en) * | 2004-08-18 | 2010-12-28 | Time Warner Inc. | Method and device for the wireless exchange of media content between mobile devices based on user information |
CA2482240A1 (en) * | 2004-09-27 | 2006-03-27 | Claude Choquet | Body motion training and qualification system and method |
JP2006155585A (en) | 2004-10-27 | 2006-06-15 | Ricoh Co Ltd | Document management service providing device, authentication service providing device, document management service providing program, authentication service providing program, recording medium, document management service providing method, and authentication service providing method |
US20060095317A1 (en) * | 2004-11-03 | 2006-05-04 | Target Brands, Inc. | System and method for monitoring retail store performance |
US7665141B2 (en) * | 2004-11-24 | 2010-02-16 | Ghada Young | Origin and custody of copies from a stored electronic record verified page by page |
US8297983B2 (en) * | 2004-11-30 | 2012-10-30 | The Regents Of The University Of California | Multimodal ultrasound training system |
US7660981B1 (en) * | 2004-11-30 | 2010-02-09 | Adobe Systems Incorporated | Verifiable chain of transfer for digital documents |
US20170176127A1 (en) | 2004-12-03 | 2017-06-22 | Bob Ferris | Simulated firearms entertainment system |
US20160059136A1 (en) | 2004-12-03 | 2016-03-03 | Bob Ferris | Simulated firearms entertainment system |
US7941376B2 (en) | 2005-03-16 | 2011-05-10 | Dt Labs, Llc | System and method for customer authentication of an item |
US7805382B2 (en) | 2005-04-11 | 2010-09-28 | Mkt10, Inc. | Match-based employment system and method |
US7870079B2 (en) * | 2005-04-11 | 2011-01-11 | Jobfox, Inc. | Method and apparatus for employment system distributed hiring and co-operative pooling |
US20060274945A1 (en) * | 2005-06-03 | 2006-12-07 | Chu Soy L | System and method for automatically extracting a picture of a person from a government issued identification piece for use on a badge |
US20060282661A1 (en) | 2005-06-13 | 2006-12-14 | True Sandra L | System and method of providing certified document retrieval |
US7661146B2 (en) | 2005-07-01 | 2010-02-09 | Privamed, Inc. | Method and system for providing a secure multi-user portable database |
EP1905188B1 (en) | 2005-07-07 | 2018-05-30 | Intrinsic ID B.V. | Method, apparatus and system for verifying authenticity of an object |
JP4722599B2 (en) | 2005-07-13 | 2011-07-13 | 富士通株式会社 | Electronic image data verification program, electronic image data verification system, and electronic image data verification method |
EP1911263A4 (en) * | 2005-07-22 | 2011-03-30 | Kangaroo Media Inc | System and methods for enhancing the experience of spectators attending a live sporting event |
US20070192140A1 (en) * | 2005-08-17 | 2007-08-16 | Medcommons, Inc. | Systems and methods for extending an information standard through compatible online access |
US7793106B2 (en) | 2005-08-17 | 2010-09-07 | The Boeing Company | Method and system for certifying the authority of a signer of an electronic document |
US8239682B2 (en) | 2005-09-28 | 2012-08-07 | Nl Systems, Llc | Method and system for digital rights management of documents |
WO2007058907A2 (en) | 2005-11-10 | 2007-05-24 | Pkware, Inc. | Systems and methods for trusted information exchange |
US20070124584A1 (en) | 2005-11-30 | 2007-05-31 | Microsoft Corporation | Proving ownership of shared information to a third party |
US7769712B2 (en) * | 2005-12-21 | 2010-08-03 | Decernis, Llc | Document validation system and method |
US8195657B1 (en) | 2006-01-09 | 2012-06-05 | Monster Worldwide, Inc. | Apparatuses, systems and methods for data entry correlation |
US9129233B2 (en) * | 2006-02-15 | 2015-09-08 | Catepillar Inc. | System and method for training a machine operator |
WO2007106851A2 (en) * | 2006-03-14 | 2007-09-20 | Document Atm Incorporated | Distributed access to valuable and sensitive documents and data |
CN101043319B (en) | 2006-03-22 | 2011-02-02 | 鸿富锦精密工业(深圳)有限公司 | Digital content protective system and method |
US20070269788A1 (en) * | 2006-05-04 | 2007-11-22 | James Flowers | E learning platform for preparation for standardized achievement tests |
US20080005024A1 (en) | 2006-05-17 | 2008-01-03 | Carter Kirkwood | Document authentication system |
US8014992B2 (en) * | 2006-05-31 | 2011-09-06 | William H Smith | Electric power system training simulation software and process for electric power system training |
DE102006027030A1 (en) * | 2006-06-08 | 2007-12-13 | Wittkötter, Erland, Dr. | Device and method for protected distribution of electronic documents |
US20070298392A1 (en) * | 2006-06-13 | 2007-12-27 | International Business Machines Corporation | Candidate transition analysis method and system |
US20070294092A1 (en) | 2006-06-15 | 2007-12-20 | Mycredententials, Inc. | System and method for creating and organizing job applicant credential information |
US8182271B2 (en) | 2006-07-25 | 2012-05-22 | Siemens Aktiengesellschaft | Training method and system |
US20080066181A1 (en) | 2006-09-07 | 2008-03-13 | Microsoft Corporation | DRM aspects of peer-to-peer digital content distribution |
US20080071746A1 (en) | 2006-09-14 | 2008-03-20 | David Joseph Concordia | Method For Interactive Employment Searching, Rating, And Selecting of Employment Listing |
US8474027B2 (en) | 2006-09-29 | 2013-06-25 | Microsoft Corporation | Remote management of resource license |
US20080091954A1 (en) | 2006-10-17 | 2008-04-17 | Morris Daniel R | Method and system for facilitating printed page authentication, unique code generation and content integrity verification of documents |
JP2008117337A (en) | 2006-11-08 | 2008-05-22 | Fuji Xerox Co Ltd | Document management system and document management program |
WO2008082554A1 (en) | 2006-12-19 | 2008-07-10 | I6 Llc | Market-specific credential and system and method for making same |
US20080208646A1 (en) * | 2007-02-28 | 2008-08-28 | Thompson Ralph E | Method for increasing productivity and safety in the mining and heavy construction industries |
US20080235236A1 (en) | 2007-03-20 | 2008-09-25 | Docommand Solution, Inc. | Secure Document Management System |
US20080235175A1 (en) | 2007-03-20 | 2008-09-25 | Docommand Solution, Inc. | Secure Document Management System |
WO2008128125A1 (en) | 2007-04-12 | 2008-10-23 | Avow Systems, Inc. | Electronic document management and delivery |
WO2008131201A1 (en) | 2007-04-19 | 2008-10-30 | Global Rainmakers, Inc. | Method and system for biometric recognition |
US9198621B2 (en) | 2007-06-18 | 2015-12-01 | University of Pittsburgh—of the Commonwealth System of Higher Education | Method, apparatus and system for food intake and physical activity assessment |
US20090049070A1 (en) | 2007-08-15 | 2009-02-19 | Arieh Steinberg | Web-based social network badges |
US9036871B2 (en) * | 2007-09-01 | 2015-05-19 | Eyelock, Inc. | Mobility identity platform |
US8212870B2 (en) | 2007-09-01 | 2012-07-03 | Hanna Keith J | Mirror system and method for acquiring biometric data |
WO2009049282A2 (en) * | 2007-10-11 | 2009-04-16 | University Of Florida Research Foundation, Inc. | Mixed simulator and uses thereof |
JP5040580B2 (en) | 2007-10-18 | 2012-10-03 | 富士ゼロックス株式会社 | Document management system and program |
US8694793B2 (en) * | 2007-12-11 | 2014-04-08 | Visa U.S.A. Inc. | Biometric access control transactions |
US20090172777A1 (en) | 2007-12-31 | 2009-07-02 | Pitney Bowes Inc. | System and method for tracking documents |
US8693737B1 (en) | 2008-02-05 | 2014-04-08 | Bank Of America Corporation | Authentication systems, operations, processing, and interactions |
US7817162B2 (en) | 2008-02-11 | 2010-10-19 | University Of Northern Iowa Research Foundation | Virtual blasting system for removal of coating and/or rust from a virtual surface |
US8714981B2 (en) | 2008-04-02 | 2014-05-06 | Sinapse Print Simulators | Automatic trace analysis and comparison system for interactive learning and training systems |
US20090299827A1 (en) * | 2008-05-30 | 2009-12-03 | Oracle International Corporation | Verifying Operator Competence |
US20090299993A1 (en) | 2008-05-30 | 2009-12-03 | Novack Michael D | Candidate Recruiting |
US7920935B2 (en) * | 2008-08-19 | 2011-04-05 | International Business Machines Corporation | Activity based real-time production instruction adaptation |
US9280913B2 (en) * | 2009-07-10 | 2016-03-08 | Lincoln Global, Inc. | Systems and methods providing enhanced education and training in a virtual reality environment |
US8725058B2 (en) | 2008-08-26 | 2014-05-13 | Ideal Response, Inc. | Sales response training system and methods |
US20100057487A1 (en) | 2008-08-27 | 2010-03-04 | Yung-Chung Heh | Configuration for language interpreter certification |
US10086262B1 (en) | 2008-11-12 | 2018-10-02 | David G. Capper | Video motion capture for wireless gaming |
US8655803B2 (en) | 2008-12-17 | 2014-02-18 | Xerox Corporation | Method of feature extraction from noisy documents |
US20100167248A1 (en) * | 2008-12-31 | 2010-07-01 | Haptica Ltd. | Tracking and training system for medical procedures |
US9449300B2 (en) | 2009-01-23 | 2016-09-20 | Cary Kalscheuer | Prospective city government jobs posting system for multiple city government employers with integrated service features |
US20100205649A1 (en) | 2009-02-06 | 2010-08-12 | Microsoft Corporation | Credential gathering with deferred instantiation |
US7865937B1 (en) | 2009-08-05 | 2011-01-04 | Daon Holdings Limited | Methods and systems for authenticating users |
US8443202B2 (en) | 2009-08-05 | 2013-05-14 | Daon Holdings Limited | Methods and systems for authenticating users |
US8893086B2 (en) | 2009-09-11 | 2014-11-18 | International Business Machines Corporation | System and method for resource modeling and simulation in test planning |
US8260269B2 (en) * | 2009-11-25 | 2012-09-04 | Visa International Service Association | Input device with an accelerometer |
US20110165542A1 (en) | 2010-01-07 | 2011-07-07 | Fairfield University | Multi-parameter, customizable simulation building system for clinical scenarios for educating and training nurses and other health care professionals |
US20180225982A1 (en) | 2010-01-15 | 2018-08-09 | ProctorU, INC. | System for online automated exam proctoring |
US10672286B2 (en) | 2010-03-14 | 2020-06-02 | Kryterion, Inc. | Cloud based test environment |
US8826030B2 (en) | 2010-03-22 | 2014-09-02 | Daon Holdings Limited | Methods and systems for authenticating users |
US8628331B1 (en) * | 2010-04-06 | 2014-01-14 | Beth Ann Wright | Learning model for competency based performance |
CA2795826C (en) * | 2010-04-08 | 2015-08-11 | Vrsim, Inc. | Simulator for skill-oriented training |
US11004019B2 (en) | 2010-05-10 | 2021-05-11 | The Institute for Motivational Living | Method of matching employers with job seekers |
US8727782B2 (en) | 2010-05-11 | 2014-05-20 | Across The Street Productions Inc. | Hazard-zone incident command training and certification systems |
US8926335B2 (en) * | 2010-05-12 | 2015-01-06 | Verificient Technologies, Inc. | System and method for remote test administration and monitoring |
US8457353B2 (en) * | 2010-05-18 | 2013-06-04 | Microsoft Corporation | Gestures and gesture modifiers for manipulating a user-interface |
WO2011149558A2 (en) | 2010-05-28 | 2011-12-01 | Abelow Daniel H | Reality alternate |
US8764454B1 (en) | 2010-06-14 | 2014-07-01 | Scott Christopher Turner | Individualized academic focus credential structure, system and methods thereof |
US8554756B2 (en) * | 2010-06-25 | 2013-10-08 | Microsoft Corporation | Integrating social network data with search results |
KR20120005364A (en) | 2010-07-08 | 2012-01-16 | 정보통신산업진흥원 | Electronic address, and eletronic document distribution system |
US8713130B2 (en) | 2010-08-04 | 2014-04-29 | Kryterion, Inc. | Peered proctoring |
US8944928B2 (en) | 2010-08-26 | 2015-02-03 | Blast Motion Inc. | Virtual reality system for viewing current and previously stored or calculated motion data |
US9076041B2 (en) | 2010-08-26 | 2015-07-07 | Blast Motion Inc. | Motion event recognition and video synchronization system and method |
US9607652B2 (en) | 2010-08-26 | 2017-03-28 | Blast Motion Inc. | Multi-sensor event detection and tagging system |
US9247212B2 (en) | 2010-08-26 | 2016-01-26 | Blast Motion Inc. | Intelligent motion capture element |
US8380837B2 (en) | 2010-09-07 | 2013-02-19 | International Business Machines Corporation | Software license management within a cloud computing environment |
RU2595885C2 (en) * | 2010-09-24 | 2016-08-27 | Виза Интернэшнл Сервис Ассосиэйшн | Method and system using universal identifier and biometric data |
KR101390383B1 (en) * | 2010-11-16 | 2014-04-29 | 한국전자통신연구원 | Apparatus for managing a reconfigurable platform for virtual reality based training simulator |
US9838373B2 (en) | 2010-11-29 | 2017-12-05 | Biocatch Ltd. | System, device, and method of detecting a remote access user |
US20120156668A1 (en) * | 2010-12-20 | 2012-06-21 | Mr. Michael Gregory Zelin | Educational gaming system |
US10043229B2 (en) | 2011-01-26 | 2018-08-07 | Eyelock Llc | Method for confirming the identity of an individual while shielding that individual's personal data |
US9990856B2 (en) * | 2011-02-08 | 2018-06-05 | The Trustees Of The University Of Pennsylvania | Systems and methods for providing vibration feedback in robotic systems |
RU2589859C2 (en) * | 2011-02-17 | 2016-07-10 | АЙЛОК ЭлЭлСи | Efficient method and system for obtaining image data of scene and iris image using one sensor |
BR112013027076A2 (en) | 2011-04-19 | 2019-09-24 | Eyelock Inc | biometric chain of origin |
US9430791B1 (en) | 2011-05-02 | 2016-08-30 | Drake Sutton-Shearer | Virtual goods having nested content and system and method for distributing the same |
EP2724309A4 (en) | 2011-06-24 | 2015-02-25 | Monster Worldwide Inc | Social match platform apparatuses, methods and systems |
US9492120B2 (en) | 2011-07-05 | 2016-11-15 | Saudi Arabian Oil Company | Workstation for monitoring and improving health and productivity of employees |
US9710788B2 (en) | 2011-07-05 | 2017-07-18 | Saudi Arabian Oil Company | Computer mouse system and associated, computer medium and computer-implemented methods for monitoring and improving health and productivity of employees |
US9962083B2 (en) * | 2011-07-05 | 2018-05-08 | Saudi Arabian Oil Company | Systems, computer medium and computer-implemented methods for monitoring and improving biomechanical health of employees |
US9526455B2 (en) | 2011-07-05 | 2016-12-27 | Saudi Arabian Oil Company | Systems, computer medium and computer-implemented methods for monitoring and improving health and productivity of employees |
US10307104B2 (en) | 2011-07-05 | 2019-06-04 | Saudi Arabian Oil Company | Chair pad system and associated, computer medium and computer-implemented methods for monitoring and improving health and productivity of employees |
US20130012802A1 (en) | 2011-07-05 | 2013-01-10 | Saudi Arabian Oil Company | Systems, Computer Medium and Computer-Implemented Methods For Monitoring and Improving Cognitive and Emotive Health of Employees |
CA2840804C (en) | 2011-07-05 | 2018-05-15 | Saudi Arabian Oil Company | Floor mat system and associated, computer medium and computer-implemented methods for monitoring and improving health and productivity of employees |
US9844344B2 (en) | 2011-07-05 | 2017-12-19 | Saudi Arabian Oil Company | Systems and method to monitor health of employee when positioned in association with a workstation |
US9786193B2 (en) * | 2011-09-01 | 2017-10-10 | L-3 Communications Corporation | Adaptive training system, method and apparatus |
US8819793B2 (en) * | 2011-09-20 | 2014-08-26 | Csidentity Corporation | Systems and methods for secure and efficient enrollment into a federation which utilizes a biometric repository |
US9208470B2 (en) | 2011-10-04 | 2015-12-08 | Yahoo! Inc. | System for custom user-generated achievement badges based on activity feeds |
US20130097093A1 (en) | 2011-10-12 | 2013-04-18 | George Kolber | Systems and Methods for Quantifying Job Candidates |
US8834163B2 (en) * | 2011-11-29 | 2014-09-16 | L-3 Communications Corporation | Physics-based simulation of warhead and directed energy weapons |
US20150302769A1 (en) * | 2011-12-05 | 2015-10-22 | Raymond C. Johnson | Virtual Hand-Washing Coach |
US20130203509A1 (en) | 2012-02-03 | 2013-08-08 | Microsoft Corporation | Virtual location check-in |
US8538402B2 (en) | 2012-02-12 | 2013-09-17 | Joel Vidal | Phone that prevents texting while driving |
US8914014B2 (en) | 2012-02-12 | 2014-12-16 | Joel Vidal | Phone that prevents concurrent texting and driving |
US9384518B2 (en) | 2012-03-26 | 2016-07-05 | Amerasia International Technology, Inc. | Biometric registration and verification system and method |
US20130317791A1 (en) | 2012-04-26 | 2013-11-28 | Conocophillips Company | Hydrodynamic slug flow model |
US20130311244A1 (en) * | 2012-05-16 | 2013-11-21 | David D. Abotchie | Computer Automated System for Selecting, Tracking, and Rating Workers |
US10114609B2 (en) | 2012-05-31 | 2018-10-30 | Opportunity Partners Inc. | Computing interface for users with disabilities |
ES2438440B1 (en) * | 2012-06-13 | 2014-07-30 | Seabery Soluciones, S.L. | ADVANCED DEVICE FOR SIMULATION-BASED WELDING TRAINING WITH INCREASED REALITY AND REMOTE UPDATE |
WO2013188301A2 (en) | 2012-06-13 | 2013-12-19 | International Association Of Flight Training Professionals, Inc. | Secure system and method for collecting, authenticating, and using personal data |
US9106611B2 (en) | 2012-06-27 | 2015-08-11 | GM Global Technology Operations LLC | Method for efficient message verification on resource-constrained platforms for real-time tracking applications |
US10824972B2 (en) | 2012-08-02 | 2020-11-03 | iQ4 LLC | Skilled based, staffing system coordinated with communication based, project management application |
US8414387B1 (en) | 2012-08-08 | 2013-04-09 | Lookout Gaming, LLC | Peer-to-peer wagering platform |
US8715077B2 (en) | 2012-08-08 | 2014-05-06 | Skillz Inc. | Dynamic gameplay advertisements |
US20140195312A1 (en) | 2012-09-06 | 2014-07-10 | Locu, Inc. | System and method for management of processing workers |
US8535059B1 (en) | 2012-09-21 | 2013-09-17 | Noble Systems Corporation | Learning management system for call center agents |
WO2014049749A1 (en) | 2012-09-26 | 2014-04-03 | 株式会社 東芝 | Biometric reference information registration system, device, and program |
US9112730B2 (en) | 2012-10-05 | 2015-08-18 | Cvtrust Sa | Centralized method for certifying credentials |
US20140122355A1 (en) | 2012-10-26 | 2014-05-01 | Bright Media Corporation | Identifying candidates for job openings using a scoring function based on features in resumes and job descriptions |
US20140205990A1 (en) | 2013-01-24 | 2014-07-24 | Cloudvu, Inc. | Machine Learning for Student Engagement |
US10388176B2 (en) | 2012-11-28 | 2019-08-20 | Vrsim, Inc. | Simulator for skill-oriented training |
US9167038B2 (en) * | 2012-12-18 | 2015-10-20 | Arash ESMAILZDEH | Social networking with depth and security factors |
US9654358B2 (en) | 2013-01-15 | 2017-05-16 | International Business Machines Corporation | Managing user privileges for computer resources in a networked computing environment |
US9526443B1 (en) * | 2013-01-19 | 2016-12-27 | Bertec Corporation | Force and/or motion measurement system and a method of testing a subject |
US10231662B1 (en) * | 2013-01-19 | 2019-03-19 | Bertec Corporation | Force measurement system |
US10856796B1 (en) * | 2013-01-19 | 2020-12-08 | Bertec Corporation | Force measurement system |
US20140207534A1 (en) * | 2013-01-24 | 2014-07-24 | International Business Machines Corporation | Skills Amelioration System |
US9373082B2 (en) | 2013-02-01 | 2016-06-21 | Worcester Polytechnic Institute | Inquiry skills tutoring system |
US9154748B2 (en) | 2013-02-25 | 2015-10-06 | Board Of Trustees Of Michigan State University | Online examination proctoring system |
US20140279629A1 (en) | 2013-03-12 | 2014-09-18 | Salesforce.Com, Inc. | System and method for generating an organization profile based on skill information |
US20140279587A1 (en) | 2013-03-13 | 2014-09-18 | GapCorp FNI FZ LLC | System for tracking vehicle speed violations |
US10223926B2 (en) * | 2013-03-14 | 2019-03-05 | Nike, Inc. | Skateboard system |
US20140282868A1 (en) * | 2013-03-15 | 2014-09-18 | Micah Sheller | Method And Apparatus To Effect Re-Authentication |
US9300646B1 (en) | 2013-03-15 | 2016-03-29 | Microstrategy Incorporated | Logging location and time data associated with a credential |
US9495526B2 (en) * | 2013-03-15 | 2016-11-15 | Eyelock Llc | Efficient prevention of fraud |
US9858828B1 (en) | 2013-03-15 | 2018-01-02 | Querium Corporation | Expert systems and methods for dynamic assessment and assessment authoring |
US20170206064A1 (en) | 2013-03-15 | 2017-07-20 | JIBO, Inc. | Persistent companion device configuration and deployment platform |
US20140304787A1 (en) | 2013-04-05 | 2014-10-09 | Microsoft Corporation | Badge notification subscriptions |
WO2014172323A1 (en) | 2013-04-15 | 2014-10-23 | Flextronics Ap, Llc | Driver facts behavior information storage system |
US20140315164A1 (en) * | 2013-04-17 | 2014-10-23 | Caterpillar Inc. | System and method for improving operator performance |
US20140330412A1 (en) * | 2013-05-03 | 2014-11-06 | Maelingar Og Thjalfun Ehf | Computerized interactive sports system and a method of assessing sports skills of users |
US9213893B2 (en) | 2013-05-23 | 2015-12-15 | Intuit Inc. | Extracting data from semi-structured electronic documents |
US20140349255A1 (en) * | 2013-05-24 | 2014-11-27 | Honeywell International Inc. | Operator competency management |
US20140353369A1 (en) | 2013-06-03 | 2014-12-04 | Ginger G. Malin | Method and System for Issuing, Managing, Verifying and Displaying |
US9251413B2 (en) | 2013-06-14 | 2016-02-02 | Lexmark International Technology, SA | Methods for automatic structured extraction of data in OCR documents having tabular data |
US20140379602A1 (en) * | 2013-06-25 | 2014-12-25 | Apollo Education Group, Inc. | Skill-driven education marketplace |
US20150037781A1 (en) | 2013-08-02 | 2015-02-05 | David S. Breed | Monitoring device and system for remote test taking |
US9536065B2 (en) | 2013-08-23 | 2017-01-03 | Morphotrust Usa, Llc | System and method for identity management |
US9472119B2 (en) * | 2013-08-26 | 2016-10-18 | Yokogawa Electric Corporation | Computer-implemented operator training system and method of controlling the system |
WO2015034937A1 (en) | 2013-09-03 | 2015-03-12 | Laureate Education, Inc. | System and method for interfacing with student portfolios |
US20150066792A1 (en) | 2013-09-03 | 2015-03-05 | Samuel S. SPRAGUE | Matching professional service providers with employers having work assignments |
WO2015063449A1 (en) | 2013-09-04 | 2015-05-07 | Zero360, Inc. | Device for providing alerts via electric stimulation |
EP3042485B1 (en) | 2013-09-05 | 2020-03-11 | NIKE Innovate C.V. | Conducting sessions with captured image data of physical activity and uploading using token-verifiable proxy uploader |
US20150072323A1 (en) * | 2013-09-11 | 2015-03-12 | Lincoln Global, Inc. | Learning management system for a real-time simulated virtual reality welding training environment |
US20150079545A1 (en) | 2013-09-16 | 2015-03-19 | Thomas R. Kurtz | Systems and methods for automated flight instruction and certification |
WO2015048338A1 (en) | 2013-09-26 | 2015-04-02 | Publicover Mark W | Providing targeted content based on a user's moral values |
US10127378B2 (en) | 2014-10-01 | 2018-11-13 | Kalman Csaba Toth | Systems and methods for registering and acquiring E-credentials using proof-of-existence and digital seals |
US9646150B2 (en) | 2013-10-01 | 2017-05-09 | Kalman Csaba Toth | Electronic identity and credentialing system |
US20150104757A1 (en) * | 2013-10-15 | 2015-04-16 | Mbfarr, Llc | Driving assessment and training method and apparatus |
US20150187224A1 (en) * | 2013-10-15 | 2015-07-02 | Mbfarr, Llc | Driving assessment and training method and apparatus |
US20150249661A1 (en) | 2013-10-18 | 2015-09-03 | LeiLani Cauthen | System and method for providing an information platform with credentials validation |
JP6071847B2 (en) | 2013-11-06 | 2017-02-01 | 株式会社東芝 | Authentication system, method and program |
AU2014347365A1 (en) * | 2013-11-08 | 2016-06-23 | Performance Lab Technologies Limited | Classification of activity derived from multiple locations |
US9589183B2 (en) * | 2013-11-22 | 2017-03-07 | Parchment, Inc. | System and method for identification and extraction of data |
CA2990779C (en) | 2013-12-16 | 2018-11-06 | Blue Ocean Laboratories, Inc. | Sleep system alarm |
EP4184483A1 (en) * | 2013-12-20 | 2023-05-24 | Intuitive Surgical Operations, Inc. | Simulator system for medical procedure training |
EP3087533A4 (en) * | 2013-12-23 | 2017-09-20 | Eyelock Llc | Methods and apparatus for power-efficient iris recognition |
WO2015108700A1 (en) * | 2014-01-14 | 2015-07-23 | Zsolutionz, LLC | Sensor-based evaluation and feedback of exercise performance |
WO2015108701A1 (en) * | 2014-01-14 | 2015-07-23 | Zsolutionz, LLC | Fuzzy logic-based evaluation and feedback of exercise performance |
US10225522B1 (en) * | 2014-01-17 | 2019-03-05 | Cerner Innovation, Inc. | Method and system for determining whether an individual takes appropriate measures to prevent the spread of healthcare-associated infections |
US9331994B2 (en) | 2014-02-07 | 2016-05-03 | Bank Of America Corporation | User authentication based on historical transaction data |
US9836987B2 (en) * | 2014-02-14 | 2017-12-05 | Lincoln Global, Inc. | Virtual reality pipe welding simulator and setup |
US20150242979A1 (en) | 2014-02-25 | 2015-08-27 | University Of Maryland, College Park | Knowledge Management and Classification in a Quality Management System |
US20150242797A1 (en) * | 2014-02-27 | 2015-08-27 | University of Alaska Anchorage | Methods and systems for evaluating performance |
US9240887B2 (en) | 2014-05-02 | 2016-01-19 | Dell Products L.P. | Off-host authentication system |
US20150363727A1 (en) | 2014-06-13 | 2015-12-17 | Newvistas, Llc | Apparatus and method for automatically allocating the use of assets |
US9381434B2 (en) | 2014-06-30 | 2016-07-05 | Linkedin Corporation | Social network interaction via games |
US9703952B2 (en) | 2014-07-07 | 2017-07-11 | University Of Ontario Institute Of Technology | Device and method for providing intent-based access control |
US10540907B2 (en) | 2014-07-31 | 2020-01-21 | Intelligent Technologies International, Inc. | Biometric identification headpiece system for test taking |
US9875665B2 (en) * | 2014-08-18 | 2018-01-23 | Illinois Tool Works Inc. | Weld training system and method |
US20160180248A1 (en) * | 2014-08-21 | 2016-06-23 | Peder Regan | Context based learning |
US20200118456A1 (en) | 2014-08-22 | 2020-04-16 | Intelligent Technologies International, Inc. | Secure Testing Device With Combiner |
EP3185748A4 (en) | 2014-08-27 | 2018-03-21 | Eyessessment Technologies Ltd. | Evaluating test taking |
US9405967B2 (en) | 2014-09-03 | 2016-08-02 | Samet Privacy Llc | Image processing apparatus for facial recognition |
US9729556B2 (en) | 2014-09-12 | 2017-08-08 | Salesforce.Com, Inc. | Digital badging for facilitating virtual recognition of an achievement |
US9729411B2 (en) | 2014-12-01 | 2017-08-08 | Payoda Inc. | Centralized device management system for monitoring and controlling various application specific network components across data centers |
US10614726B2 (en) | 2014-12-08 | 2020-04-07 | Life Long Driver, Llc | Behaviorally-based crash avoidance system |
WO2016094454A1 (en) | 2014-12-08 | 2016-06-16 | Brad Keays | Remote sobriety monitoring systems, devices and methods |
US20160188765A1 (en) | 2014-12-31 | 2016-06-30 | Ge Aviation Systems Llc | Aircraft simulation system |
US9942056B2 (en) | 2015-02-19 | 2018-04-10 | Vivint, Inc. | Methods and systems for automatically monitoring user activity |
US9977884B2 (en) | 2015-02-27 | 2018-05-22 | Plantronics, Inc. | Authentication server for a probability-based user authentication system and method |
CA3017401C (en) * | 2015-03-12 | 2019-12-31 | Eyelock Llc | Methods and systems for managing network activity using biometrics |
WO2016164214A1 (en) | 2015-04-08 | 2016-10-13 | Novatime Technology Inc. | Electronic barcode badge for employee access |
US10373523B1 (en) * | 2015-04-29 | 2019-08-06 | State Farm Mutual Automobile Insurance Company | Driver organization and management for driver's education |
US20160321591A1 (en) | 2015-04-30 | 2016-11-03 | International Business Machines Corporation | Outcomes-based application monitoring |
JP6462122B2 (en) | 2015-06-03 | 2019-01-30 | 株式会社日立製作所 | License management system and license management method |
US20160356751A1 (en) | 2015-06-08 | 2016-12-08 | Lunatech, Llc | Respiration Simulating Analysis And Distribution Device |
US20160360791A1 (en) | 2015-06-10 | 2016-12-15 | Lunatech, Llc | Air Analyzer And Treatment System |
US10060639B2 (en) | 2015-06-11 | 2018-08-28 | Lunatech, Llc | Air analyzer and treatment apparatus |
US20160363917A1 (en) | 2015-06-11 | 2016-12-15 | Lunatech, Llc | User Interface For An Analysis And Vapor Dispensing Apparatus |
US10088463B2 (en) | 2015-06-11 | 2018-10-02 | Lunatech, Llc | Calibrating electronic vapor device |
US20160363582A1 (en) | 2015-06-12 | 2016-12-15 | Lunatech, Llc | Breath analyzer |
US20160363339A1 (en) | 2015-06-12 | 2016-12-15 | Lunatech, Llc | Remotely Controllable System For Localized Air Sensing |
US10215430B2 (en) | 2015-06-15 | 2019-02-26 | Lunatech, Llc | Electronic vapor and analysis with HVAC integration |
US9933790B2 (en) | 2015-06-15 | 2018-04-03 | Lunatech, Llc | Peer-to-peer air analysis and treatment |
US9981532B2 (en) | 2015-06-15 | 2018-05-29 | Lunatech, Llc | Transportation vehicle air detection and augmentation system |
US10215429B2 (en) | 2015-06-15 | 2019-02-26 | Lunatech, Llc | Localized air sensing and treatment |
US20160363572A1 (en) | 2015-06-15 | 2016-12-15 | Lunatech, Llc | Vapor Processing And Analyzing Device And System |
US20160363567A1 (en) | 2015-06-15 | 2016-12-15 | Lunatech, Llc | Vapor Device For Stressing And Analyzing Compounds |
US10042369B2 (en) | 2015-06-16 | 2018-08-07 | Lunatech, Llc | Vapor device for filtering and testing material |
US20160370335A1 (en) | 2015-06-16 | 2016-12-22 | Lunatech, Llc | Vapor Device For Security Screening |
US10088464B2 (en) * | 2015-06-16 | 2018-10-02 | Lunatech, Llc | Systems and methods for analyzing pharmaceuticals |
US20160370337A1 (en) | 2015-06-16 | 2016-12-22 | Lunatech, Llc | Analysis System For Biological Compounds, And Method Of Operation |
US20160367925A1 (en) | 2015-06-16 | 2016-12-22 | Lunatech, Llc | Air Analyzer, Treatment And Peer Networking Apparatus |
US10065138B2 (en) | 2015-06-17 | 2018-09-04 | Lunatech, Llc | Remote controllable air treatment apparatus |
US10581828B2 (en) | 2015-06-26 | 2020-03-03 | Accenture Global Services Limited | Certification management system |
US20170005868A1 (en) | 2015-07-02 | 2017-01-05 | Pearson Education, Inc. | Automated network generation |
US20170020195A1 (en) | 2015-07-20 | 2017-01-26 | Lunatech, Llc | Electronic Vaporizer Testing |
US20170032248A1 (en) | 2015-07-28 | 2017-02-02 | Microsoft Technology Licensing, Llc | Activity Detection Based On Activity Models |
US10051001B1 (en) | 2015-07-31 | 2018-08-14 | Palo Alto Networks, Inc. | Efficient and secure user credential store for credentials enforcement using a firewall |
CA3002034A1 (en) | 2015-10-14 | 2017-04-20 | Cambridge Blockchain, LLC | Systems and methods for managing digital identities |
US10409964B2 (en) | 2015-11-04 | 2019-09-10 | Screening Room Media, Inc. | Pairing devices to prevent digital content misuse |
US10365962B2 (en) | 2015-11-16 | 2019-07-30 | Pearson Education, Inc. | Automated testing error assessment system |
US9984586B2 (en) * | 2015-11-22 | 2018-05-29 | Victor Popa-Simil | Method and device to improve the flying abilities of the airborne devices operator |
US10037418B2 (en) | 2015-11-25 | 2018-07-31 | Dell Products L.P. | Pre-boot authentication credential sharing system |
US20170154307A1 (en) | 2015-11-30 | 2017-06-01 | Linkedln Corporation | Personalized data-driven skill recommendations and skill gap prediction |
US20170154310A1 (en) | 2015-11-30 | 2017-06-01 | Linkedln Corporation | Identifying actual coworkers for a particular job listing |
US20170154542A1 (en) | 2015-12-01 | 2017-06-01 | Gary King | Automated grading for interactive learning applications |
US10475351B2 (en) * | 2015-12-04 | 2019-11-12 | Saudi Arabian Oil Company | Systems, computer medium and methods for management training systems |
US10311299B2 (en) | 2015-12-21 | 2019-06-04 | Eyelock Llc | Reflected optic camera module for iris recognition in a computing device |
US20170193845A1 (en) | 2015-12-30 | 2017-07-06 | International Business Machines Corporation | Detection of anomalous behavior in digital education settings based on portable device movement |
US10142347B2 (en) | 2016-02-10 | 2018-11-27 | Bank Of America Corporation | System for centralized control of secure access to process data network |
JP2017156972A (en) * | 2016-03-01 | 2017-09-07 | 株式会社小松製作所 | Evaluation device, management device, evaluation system, and evaluation method |
US9812027B2 (en) | 2016-03-08 | 2017-11-07 | Gholam Hossein Zereshkian | Anti-cheating device for online examination |
US9894062B2 (en) | 2016-03-16 | 2018-02-13 | Dell Products, L.P. | Object management for external off-host authentication processing systems |
US10033536B2 (en) * | 2016-03-25 | 2018-07-24 | Credly, Inc. | Generation, management, and tracking of digital credentials |
US10068074B2 (en) * | 2016-03-25 | 2018-09-04 | Credly, Inc. | Generation, management, and tracking of digital credentials |
US10187394B2 (en) | 2016-03-31 | 2019-01-22 | Microsoft Technology Licensing, Llc | Personalized inferred authentication for virtual assistance |
US10242501B1 (en) * | 2016-05-03 | 2019-03-26 | WorldViz, Inc. | Multi-user virtual and augmented reality tracking systems |
WO2017192726A1 (en) * | 2016-05-03 | 2017-11-09 | Azuga, Inc. | Method and apparatus for evaluating driver performance and determining driver rewards |
US10311300B2 (en) | 2016-05-18 | 2019-06-04 | Eyelock Llc | Iris recognition systems and methods of using a statistical model of an iris for authentication |
US20170344927A1 (en) | 2016-05-24 | 2017-11-30 | Accenture Global Solutions Limited | Skill proficiency system |
US20170357928A1 (en) * | 2016-06-08 | 2017-12-14 | Honeywell International Inc. | System and method for industrial process control and automation system operator evaluation and training |
US10092827B2 (en) | 2016-06-16 | 2018-10-09 | Disney Enterprises, Inc. | Active trigger poses |
US11188864B2 (en) | 2016-06-27 | 2021-11-30 | International Business Machines Corporation | Calculating an expertise score from aggregated employee data |
US10489526B1 (en) | 2016-07-29 | 2019-11-26 | Rockwell Collins, Inc. | Adaptive control for simulator input generation |
US10559217B2 (en) | 2016-08-05 | 2020-02-11 | Intel Corporation | Methods and apparatus to develop in-vehicle experiences in simulated environments |
US11429859B2 (en) | 2016-08-15 | 2022-08-30 | Cangrade, Inc. | Systems and processes for bias removal in a predictive performance model |
US10218719B2 (en) | 2016-09-21 | 2019-02-26 | Apple Inc. | Credential modification notifications |
US20180096306A1 (en) | 2016-09-30 | 2018-04-05 | Linkedin Corporation | Identifying a skill gap based on member profiles and job postings |
US10613709B2 (en) | 2016-09-30 | 2020-04-07 | Salesforce.Com, Inc. | Leveraging and managing assessment environments in an assessment hub |
US10642345B2 (en) * | 2016-10-18 | 2020-05-05 | Raytheon Company | Avionics maintenance training |
US10133856B2 (en) | 2016-11-07 | 2018-11-20 | Honeywell International Inc. | Method and system for managing software license for vehicle |
US11186290B2 (en) * | 2016-11-16 | 2021-11-30 | Honda Motor Co., Ltd. | Emotion inference device and emotion inference system |
US11003305B2 (en) | 2016-11-18 | 2021-05-11 | Zspace, Inc. | 3D user interface |
US10127715B2 (en) | 2016-11-18 | 2018-11-13 | Zspace, Inc. | 3D user interface—non-native stereoscopic image conversion |
US10783801B1 (en) * | 2016-12-21 | 2020-09-22 | Aptima, Inc. | Simulation based training system for measurement of team cognitive load to automatically customize simulation content |
US11216766B2 (en) * | 2017-01-09 | 2022-01-04 | Kabushiki Kaisha Toshiba | System and method for generalized skill assessment using activity data |
US20180203238A1 (en) * | 2017-01-18 | 2018-07-19 | Marshall Leroy Smith, JR. | Method of education and simulation learning |
US11423755B2 (en) | 2017-05-17 | 2022-08-23 | Blue Storm Media, Inc. | System and method for a digital proof of vaccine |
US10052026B1 (en) * | 2017-03-06 | 2018-08-21 | Bao Tran | Smart mirror |
US9931539B1 (en) * | 2017-03-14 | 2018-04-03 | Brooklyn Fitboxing International, S.L. | Integrated system for boxing and martial arts-based group competitive training and method of use the same |
US20180268341A1 (en) * | 2017-03-16 | 2018-09-20 | Selleration, Inc. | Methods, systems and networks for automated assessment, development, and management of the selling intelligence and sales performance of individuals competing in a field |
US10402771B1 (en) * | 2017-03-27 | 2019-09-03 | Uber Technologies, Inc. | System and method for evaluating drivers using sensor data from mobile computing devices |
US20180284453A1 (en) | 2017-04-03 | 2018-10-04 | Walmart Apollo, Llc | Customer interaction system |
US10101804B1 (en) | 2017-06-21 | 2018-10-16 | Z5X Global FZ-LLC | Content interaction system and method |
CN107292271B (en) * | 2017-06-23 | 2020-02-14 | 北京易真学思教育科技有限公司 | Learning monitoring method and device and electronic equipment |
US20190026473A1 (en) | 2017-07-21 | 2019-01-24 | Pearson Education, Inc. | System and method for automated feature-based alert triggering |
US20190051199A1 (en) * | 2017-08-11 | 2019-02-14 | Turner Corbett | Certifying User Knowledge and Skills |
US10489976B2 (en) | 2017-08-11 | 2019-11-26 | Jing Jin | Incident site investigation and management support system based on unmanned aerial vehicles |
US10152141B1 (en) * | 2017-08-18 | 2018-12-11 | Osterhout Group, Inc. | Controller movement tracking with light emitters |
US20190087833A1 (en) | 2017-09-15 | 2019-03-21 | Pearson Education, Inc. | Digital credential receiver performance model |
US20180101806A1 (en) * | 2017-09-27 | 2018-04-12 | Alpha Consults LLC | Quantitative Competency Management System for Learning Management System |
CA3022094A1 (en) | 2017-10-25 | 2019-04-25 | Marc Chelnik | Affinity and proximity information exchange systems and methods |
US10951613B2 (en) | 2017-12-28 | 2021-03-16 | iProov Ltd. | Biometric methods for online user authentication |
JP7298932B2 (en) * | 2018-02-02 | 2023-06-27 | アクセス ヴァーチャル, エルエルシー | Virtual reality based pilot training system |
US20190295101A1 (en) | 2018-03-22 | 2019-09-26 | Pearson Education, Inc. | Digital credential receiver field mappings |
-
2018
- 2018-05-30 US US15/993,557 patent/US20190087833A1/en not_active Abandoned
- 2018-05-30 US US15/993,394 patent/US11341508B2/en active Active
- 2018-05-30 US US15/993,546 patent/US20190087830A1/en active Pending
- 2018-05-30 US US15/993,410 patent/US20190089691A1/en not_active Abandoned
- 2018-05-30 US US15/993,558 patent/US20190087834A1/en active Pending
- 2018-05-30 US US15/993,555 patent/US20190087832A1/en not_active Abandoned
- 2018-05-30 US US15/993,553 patent/US20190089692A1/en active Pending
- 2018-05-30 US US15/993,549 patent/US20190087831A1/en not_active Abandoned
- 2018-09-06 US US16/123,831 patent/US10885530B2/en active Active
- 2018-09-06 US US16/123,870 patent/US20190087558A1/en not_active Abandoned
- 2018-09-06 WO PCT/US2018/049767 patent/WO2019055286A1/en active Application Filing
- 2018-09-06 US US16/123,910 patent/US11042885B2/en active Active
-
2021
- 2021-02-26 US US17/187,425 patent/US20210192539A1/en active Pending
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10812502B2 (en) * | 2018-02-06 | 2020-10-20 | Bank Of America Corporation | Network device owner identification and communication triggering system |
US10819731B2 (en) | 2018-02-06 | 2020-10-27 | Bank Of America Corporation | Exception remediation logic rolling platform |
US10862915B2 (en) * | 2018-02-06 | 2020-12-08 | Bank Of America Corporation | Exception remediation logic routing and suppression platform |
US11089042B2 (en) | 2018-02-06 | 2021-08-10 | Bank Of America Corporation | Vulnerability consequence triggering system for application freeze and removal |
US11265340B2 (en) | 2018-02-06 | 2022-03-01 | Bank Of America Corporation | Exception remediation acceptable use logic platform |
WO2020249568A1 (en) * | 2019-06-14 | 2020-12-17 | Interdigital Ce Patent Holdings | Method and apparatus for associating a first device with a second device |
CN110377702A (en) * | 2019-07-18 | 2019-10-25 | 中科鼎富(北京)科技发展有限公司 | A kind of biographic information matching process and device |
US20220116467A1 (en) * | 2020-10-09 | 2022-04-14 | Olive AI, Inc. | Methods and apparatus for controlling the presentation of data through an operator interface by monitoring user interactions across various compute devices |
US11765247B2 (en) * | 2020-10-09 | 2023-09-19 | Olive AI, Inc. | Methods and apparatus for controlling the presentation of data through an operator interface by monitoring user interactions across various compute devices |
Also Published As
Publication number | Publication date |
---|---|
US20190087829A1 (en) | 2019-03-21 |
US20190087834A1 (en) | 2019-03-21 |
US20190089692A1 (en) | 2019-03-21 |
US11042885B2 (en) | 2021-06-22 |
US20190087830A1 (en) | 2019-03-21 |
US10885530B2 (en) | 2021-01-05 |
US11341508B2 (en) | 2022-05-24 |
US20190087833A1 (en) | 2019-03-21 |
US20190087519A1 (en) | 2019-03-21 |
US20210192539A1 (en) | 2021-06-24 |
US20190087831A1 (en) | 2019-03-21 |
US20190089691A1 (en) | 2019-03-21 |
WO2019055286A1 (en) | 2019-03-21 |
US20190089701A1 (en) | 2019-03-21 |
US20190087558A1 (en) | 2019-03-21 |
US20190087781A1 (en) | 2019-03-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11341508B2 (en) | Automatically certifying worker skill credentials based on monitoring worker actions in a virtual reality simulation environment | |
US9667321B2 (en) | Predictive recommendation engine | |
US20230237504A1 (en) | Systems and methods for verifying issuance of new digital credentials | |
US10027740B2 (en) | System and method for increasing data transmission rates through a content distribution network with customized aggregations | |
US10803104B2 (en) | Digital credential field mapping | |
US20130330704A1 (en) | Student information system | |
US11210965B2 (en) | Diagnostic analyzer for visual-spatial content | |
US11006294B2 (en) | Diagnostic analyzer for content receiver using wireless execution device | |
US10516691B2 (en) | Network based intervention | |
US20220375015A1 (en) | Systems and methods for experiential skill development | |
US20220406207A1 (en) | Systems and methods for objective-based skill training | |
US20200257741A1 (en) | Intervention analyzer for content distribution networks | |
US11868374B2 (en) | User degree matching algorithm | |
US20160364821A1 (en) | Forward-pruning depth-first recursion | |
US11983723B2 (en) | Tracking digital credential usage in a sensor-monitored environment | |
US10127302B2 (en) | Data redundancy maximization tool | |
WO2018200054A1 (en) | Method and system for bayesian network-based standard or skill mastery determination | |
US20220138881A1 (en) | Systems and methods for skill development monitoring and feedback | |
US20160364820A1 (en) | Load impact prediction processor |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: PEARSON EDUCATION, INC., NEW YORK Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MERCURY, MARK;SCHMIDT, JARIN;PORTER, E. CLARKE;AND OTHERS;SIGNING DATES FROM 20170914 TO 20170918;REEL/FRAME:046916/0900 |
|
AS | Assignment |
Owner name: PEARSON EDUCATION, INC., NEW YORK Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE CONVEYING PARTY'S EXECUTION DATES PREVIOUSLY RECORDED ON REEL 046916 FRAME 0900. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT OF ASSIGNOR'S INTEREST;ASSIGNORS:MERCURY, MARK;SCHMIDT, JARIN;PORTER, E. CLARKE;AND OTHERS;SIGNING DATES FROM 20170914 TO 20170918;REEL/FRAME:047158/0832 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |