WO2013165923A1 - Recruitment enhancement system - Google Patents

Recruitment enhancement system Download PDF

Info

Publication number
WO2013165923A1
WO2013165923A1 PCT/US2013/038702 US2013038702W WO2013165923A1 WO 2013165923 A1 WO2013165923 A1 WO 2013165923A1 US 2013038702 W US2013038702 W US 2013038702W WO 2013165923 A1 WO2013165923 A1 WO 2013165923A1
Authority
WO
WIPO (PCT)
Prior art keywords
individual
information
job
recruitment
individuals
Prior art date
Application number
PCT/US2013/038702
Other languages
French (fr)
Inventor
Luca Bonmassar
Sheeroy DESAI
John Dane SMILANICK
Original Assignee
Gild, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US13/493,791 external-priority patent/US8719179B2/en
Priority claimed from US13/557,812 external-priority patent/US20140032435A1/en
Priority claimed from US13/568,493 external-priority patent/US20130290206A1/en
Priority claimed from US13/652,749 external-priority patent/US20130290207A1/en
Priority claimed from US13/739,381 external-priority patent/US20130290208A1/en
Application filed by Gild, Inc. filed Critical Gild, Inc.
Publication of WO2013165923A1 publication Critical patent/WO2013165923A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/06Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063Operations research, analysis or management
    • G06Q10/0639Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G06Q10/06398Performance of employee with respect to a job function
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/01Social networking

Abstract

A recruiting service is disclosed that generates profiles of software developers having specific skills. Public code repositories are examined to identify projects of software developers. The projects are analyzed to estimate the number of years of experience a software developer has with an individual language and determine a score with respect to other developers. Social media information and a messaging link may also be provided with each profile. A graphical user interface for displaying the information is disclosed.

Description

RECRUITMENT ENHANCEMENT SYSTEM
FIELD OF THE INVENTION
[0001] The present invention is generally related to employment recruitment tools. More particularly, the present invention is directed to a user interface, search technology, and scoring technique to automatically provide information to aid in recruiting software developers.
BACKGROUND OF THE INVENTION
[0002] Recruiting skilled software developers is a difficult task. How does one find qualified candidates? Many of the conventional recruiting approaches based on reviewing resumes do not work well for recruiting software developers.
[0003] One of the problems in the prior art is identifying individuals who have expertise in specific software languages as well as the passion and ingenuity to solve specific problems. How does a recruiter evaluate the actual skills and talents of prospective candidates? Education alone is not adequate to determine actual talent. Nor is the number of years working in industry a good measure of talent.
[0004] Another problem in the prior art is identifying whether a candidate for a software development position will be a good social fit for a company. Conventional resumes do not provide a good indicator of the social fit of a candidate.
[0005] One aspect of these problems in the prior art is that it is difficult to perform a pre-screening to identify talented candidates to fill a software development position. As a result, many companies waste enormous amounts of time trying to find qualified candidates to fill software development positions. Additionally, the difficulties in assessing the actual talent of a candidate means that companies sometimes end up with employees that cannot perform as expected. SUMMARY OF THE INVENTION
[0006] A recruiting service generates a graphical user interface in response to a query. The graphical user interface provides profile information for software developers. The profile information includes a ranking based on analysis of public code repositories and may also include other information regarding the knowledge, experience, and influence of a developer The profile information may also be augmented with additional social media information, such as social media links for a developer. A messaging link may also optionally be provided to contact a developer. The recruiting service thus permits a user to input a query to find software developers with specific skills and receive a graphical user interface providing objective evaluation information based on the code written by the developer.
[0007] First examples of a recruitment enhancement system, method, and computer program product are disclosed that can be used by job recruiters to assess a job applicant's suitability for particular jobs. The recruitment enhancement system can maintain and regularly update a proprietary database of individuals where the individuals can be scored or ranked within a profession. In one embodiment, online content authored by an individual is analyzed and classified for emotional, psychological, or personality information. This information can be used to provide information on an optimum time to recruit an individual, the potential fit of an individual to an opening, and to aid in reaching out to the candidate.
[0008] In one embodiment, a computer system includes a processor and a memory, and acquires social media content authored by an individual. The content is classified using Machine Learning (ML) according to at least one factor indicative of a psychological, emotional, or personality characteristics of the individual relevant to recruiting the individual for a job. The classification may be displayed using a user interface. Trend data may be displayed for information relevant to determining an optimum time to recruit the individual.
[0009] Second examples of a recruitment enhancement system are disclosed that can be used by job recruiters to assess job applicant's suitability for particular jobs. The recruitment enhancement system includes a module for generating and encouraging job applicants to participate in recruitment enhancement activities, such as knowledge-based contests. The participant's performance in the recruitment enhancement activities can be evaluated and characterized. Job recruiters can use the characterization of the participant's performance in the recruitment enhancement activities to identify candidates for a job position or filter applicants that have applied for a job position.
[0010] A first aspect of the described embodiments relates to a method of electronic job recruiting in a server including a processor and memory. The method can be generally characterized as comprising: a) determining in the processor a recruitment enhancement activity useful for filtering job candidates in a recruiting process is to be generated; b) generating in the processor an interface state that allows a participant to grant access to social media information from a social media site; c) retrieving in the processor the social media information from the social media site; d) generating in the processor a plurality of interface states that allow the participant to participate in the recruitment enhancement activity; e) scoring in the processor the participant's participation in the recruitment enhancement activity; f) based upon the participant's score, ranking in the processor the participant's performance; and g) generating in the processor an interface state that allows a job recruiter to view the participant's score, ranking and the social media information retrieved about the participant from the social media site.
[0011] Third examples of a recruitment enhancement system are disclosed that can be used by job recruiters to assess job applicant's suitability for particular jobs. The recruitment enhancement system can maintain and regularly update a proprietary database of individuals where the individuals can be scored or ranked within a profession. In one embodiment, a score or rank can be developed for a software developer in the software development profession. However, scores and ranks can also developed for individuals in other professions as well and the example of the software development professions is provided for the purposes of illustration only.
[0012] In various embodiments, the recruitment enhancement system can be configured to receive a recruitment service request from an outside entity. On type of service request can be a request for a rank or score of a number of individuals named in the request. The system can be configured to try to match the names received in the request to individuals that have been previously scored or ranked where scored individual's information is maintained in database. A report can be generated that includes information about the matched individuals and returned in a response to the request.
[0013] A second aspect of the present invention can be generally characterized as a method of electronic job recruiting in a server including a processor and memory. The method can include 1) receiving in the processor information associated with professional skills of a first plurality of individuals; 2) for each the first plurality of individuals, generating in the processor from the received information a score where the score provides a qualitative indicator of an aptitude at a profession associated with the professional skills; 3) storing by the processor at least the score and identification information for each of the first plurality of individuals; 4) receiving in the processor a recruitment service request including information associated with a second plurality of individuals; 5) based upon the information received in the service request, determining in the processor whether each of the second plurality of individuals matches one of the first plurality of individuals; and 6) generating in the processor a response to the recruitment service requests that includes each of the second of plurality of individuals that are matched to one of the first plurality of individuals and the score for the matched individuals. In one embodiment, the profession is a software developer and the professional skills include software development skills.
[0014] Fourth examples of a recruitment enhancement system, method, and computer program product are disclosed that can be used by job recruiters to assess a job applicant's suitability for particular jobs. In one embodiment, a computer system includes a processor and a memory, and acquires social media content generated by an individual. The social media content can be obtained via crawling the web for publically available information, such as information available on social media sites. The information can be aggregated and then filtered. One objective of the filtering can be to present information that is safe to use in a hiring decision. For example, the aggregated data can be filtered to remove protected class information associated with federal regulations. In general, the filtering can be performed to prevent
discriminatory hiring practices that conflict with federal regulations, such as hiring decisions based upon race, age or gender discrimination, while still allowing recruiters to learn additional information about individuals that can be safely used in a hiring process. [0015] A third aspect of the embodiments described herein is related to a method of electronic job recruiting in a system including a processor and memory. The method can be generally characterized as comprising training by the processor one or more filters to identify safe data, unsafe data or combinations thereof where the safe data is selected to be safe to use in a hiring decision, receiving by the processor hiring data from a plurality of data sources for a plurality of individuals where the hiring data includes resume data and social media data; aggregating by the processor the received hiring data for each individual; storing by the processor the aggregated hiring data for each individual to a database , wherein the individual's aggregated hiring data includes a plurality of different portions; receiving by the processor search parameters associated with a hiring decision; based upon the search parameters, locating by the processor at least one individual in the database that satisfy the search parameters; applying by the processor the one or more filters to each portion of the at least one individual's aggregated hiring data to identify safe or unsafe portions of the data; and outputting by the processor only the safe portions.
BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 is a high level diagram illustrating a recruiting service for software developers in accordance with an embodiment of the present invention.
[0017] Figure 2 is a screenshot illustrating a graphical user interface displaying an initial listing of profiles matching a query in accordance with an embodiment of the present invention.
[0018] Figure 3 is a screenshot illustrating the graphical user displaying a first portion of an individual profile in accordance with an embodiment of the present invention.
[0019] Figure 4 is a screen shot illustrating the graphical user interface displaying a second portion of an individual profile in accordance with an embodiment of the present invention.
[0020] Figure 5 is a block diagram of a recruiting service in accordance with an embodiment of the present invention.
[0021] Figure 6 is a block diagram of a recruitment ecosystem in accordance with an embodiment of the present invention.
[0022] Figure 7 is a block diagram of a recruitment enhancement system in accordance with an embodiment of the present invention.
[0023] Figure 8 is a block diagram of a recruitment enhancement system including a developer database interacting with two applicant tracking systems in accordance with an embodiment of the present invention.
[0024] Figures 9A and 9B are examples of information communicated between a recruitment enhancement system and an applicant tracking system in accordance with an embodiment of the present invention.
[0025] Figure 10 is a flow chart of a method of a recruitment enhancement system in accordance with an embodiment of the present invention.
[0026] Figure 1 1 is a user interface for displaying information indicative of recruiting candidate's psychological state or personality type in accordance with an embodiment of the present invention.
[0027] Figure 12 illustrates a method of generating information on the psychological, emotional, and personality characteristics of a recruiting candidate in accordance with an embodiment of the present invention. [0028] Figure 13 is a user interface for displaying information indicative of a networking prospect's psychological state or personality type in accordance with an embodiment of the present invention.
[0029] Figure 14 illustrates a method of generating information on the psychological, emotional, and personality characteristics of a networking prospect in accordance with an embodiment of the present invention.
[0030] Figure 15 is a block diagram of a recruitment ecosystem in accordance with an embodiment of the present invention.
[0031] Figure 16 is a flow chart of a method of job recruiting in accordance with an embodiment of the present invention.
[0032] Figures 17-19 are pages of an administrative interface for configuring a recruitment activity in accordance with an embodiment of the present invention.
[0033] Figures 20-27 depicts a number of different interface states involving a user participating in a recruitment activity implemented on a portable electronic device in accordance with embodiments of the present invention.
[0034] Figure 28 is a block diagram of a recruitment ecosystem in accordance with an embodiment of the present invention.
[0035] Figure 29 is a block diagram of a recruitment enhancement system in accordance with an embodiment of the present invention.
[0036] Figure 30 is a block diagram of a recruitment enhancement system including a developer database interacting with two applicant tracking systems in accordance with an embodiment of the present invention.
[0037] Figures 31 A and 3 IB are examples of information communicated between a recruitment enhancement system and an applicant tracking system in accordance with an embodiment of the present invention.
[0038] Figure 32 is a flow chart of a method of a recruitment enhancement system in accordance with an embodiment of the present invention.
[0039] Figure 33 is a block diagram of a recruitment ecosystem in accordance with an embodiment of the present invention.
[0040] Figure 34 is a block diagram of a recruitment enhancement system in accordance with an embodiment of the present invention. [0041] Figure 35 is a block diagram of a recruitment enhancement system including a developer database interacting with two applicant tracking systems in accordance with an embodiment of the present invention.
[0042] Figures 36A and 36B are examples of information communicated between a recruitment enhancement system and an applicant tracking system in accordance with an embodiment of the present invention.
[0043] Figure 37 illustrates a method of gathering and aggregating information for an individual in accordance with an embodiment of the present invention.
[0044] Figure 38 is a flow chart of a method of filtering user data for protected class information in accordance with an embodiment of the present invention.
[0045] Figure 39 is a flow chart of a method of filtering user data for protected class information including training a filter in accordance with an embodiment of the present invention.
[0046] Figures 40A and 40B shows an example of an interface configured to display information that can be gathered and aggregated by the system for an individual and then filtered in accordance with an embodiment of the present invention.
DETAILED DESCRIPTION
[0047] In the following paper, numerous specific details are set forth to provide a thorough understanding of the concepts underlying the described embodiments. It will be apparent, however, to one skilled in the art, that the described embodiments may be practiced without some or all of these specific details. In other instances, well known process steps have not been described in detail in order to avoid unnecessarily obscuring the underlying concepts. I. RECRUITMENT ENHANCEMENT SYSTEM INCLUDING
CANDIDATE SCORING AND ASSOCIATED USER INTERFACE
[0048] Figure 1 is a high level system diagram of a recruiting service 100 in accordance with an embodiment of the present invention. The recruiting service 100 is a computer- implemented service that may include one or more servers and associated hardware, such as computer processors, a database, and a memory for storing computer program instructions. The recruiting service 100 accesses information sources on the Internet to obtain information on software developers to develop profile information that includes information about the skills and experience of software developers.
[0049] In some situations a large organization could maintain the recruiting service as an in-house tool available to users within the organization via a local area network or Intranet. However, more generally the recruiting service may be implemented as a web-hosted service available over the Internet to individuals, companies, or organizations seeking to obtain information on potential candidates for software development positions.
[0050] An exemplary set of Internet information sources is illustrated in Figure 1. One aspect of the present invention is that code repositories 105, such as public code repositories, are searched. Public code repositories are repositories in which programmers can store a software project that they have worked on in an individual repository for others to view and comment on. Examples of repositories include
Github, Inc. of San Francisco, California, which permits a programmer to push source code to a repository so that it is accessible and transparent to others. In Github, each project in a repository includes a file history listing each commit that changed the file along with the author (or authors) for each commit. Other examples include sites operated by companies and organizations such as Bit Bucket, Google Code of Google, Inc. of Mountain View California, Source Forge, Launch Pad, and Type, the Apache foundation and the Mozilla foundation.
[0051] At least one other source of information is preferably accessed to obtain additional information for each profile. Another potential source of information the recruiting service can access are forum and discussion groups 1 10 used by programmers, such as Stack Overflow (operated by Stack Exchange, Inc. of New York, New York), and news groups like Hacker News or Android developer mailing lists. For example, forum and discussion groups may be used to provide a source of information on the reputation and influence of individual developers. Additionally another option is for the recruiting service to access general or social media sites 1 15, such as those provided by companies such as Facebook, Inc. and Linkedln. Another source of information are contact services and social intelligence services 120, which provide resources to identify individuals from partial contact information and otherwise expand an initial set of contact information into a wider set of contact information and social information from which links to social media can be determined. More generally, other public information sources 125 may also be searched as well that are relevant to determining the influence, skills, or biographical information about software developers, such as professional network sites.
[0052] A user utilizes a computer 102 in communication with the recruiting service 100 via the Internet. The user's computer 102 displays a graphical user interface generated by the recruiting service 100. A user searching for candidates to fill a software development position accesses the recruiting service 100 to input a query 130 defining an initial candidate specification, such as proficiency in one or more programming languages. Other examples of a candidate specification include a geographical area specification. In response, the graphical user interface generated by the recruiting service provides a listing of profiles of potential candidates as illustrated by arrow 140, which may also be presented in a ranked order. The user can then request more detailed profile information for individual candidates. An exemplary set of profile information includes the number of years and relative ranking of the candidate in different programming languages, an influence score, overall experience level, a summary of programming projects and links to the projects, a summary of employment history, and social media information. A messaging link is preferably provided to permit the candidate to be contacted either directly via email (e.g., via either anonymous or non-anonymous email) or by other contact modalities (e.g., messaging, phone, etc.).
[0053] Figure 2 is a screenshot of an exemplary graphical user interface. A search field 205, permits a user to enter queries based on skills. For example, in one embodiment a user may input language skills and any Boolean logic operators (AND or OR) to define a skill portion of the query. A location search field 210 permits the query to be limited by geographical area and a name field 215 permits the query to be limited by name of the developer. Additionally, it is contemplated that other search fields could be included, if desired, to focus a search.
[0054] In this example, a skill query based on "Java" skills is input into search field 205. A search button 207 permits the search to be triggered. This results in an initial listing 220 of profiles 225. In one implementation the profiles are sorted and ranked by overall knowledge. Other profile information may be displayed in the initial listing such as the developer's name 227, photo 229 (if available), brief summary of employment history 231 (if available), and ranked scoring 233 in different programming languages including those in the query and other selected languages for the profile. Thus in this example, the ranked scoring includes the Java language ranking first (because the query was for Java) along with other top scores. The user interface may also provide an indication of the ranking in terms of the top rankings (e.g., through a set of top rankings, such as top 10%, 20%, or 30%) via a tab other visual indicator. Thus, the user can quickly search for profiles in the initial listing corresponding to developers that are knowledgeable and skilled in a language of interest.
[0055] The graphical user interface permits a user to select an individual profile and then displays detailed profile information for the individual profile. Figure 3 illustrates a first portion of an individual profile for a developer. The profile may include the person's name 227, photo 229 (if available), a brief summary of code analysis 305, full results of code analysis 310, knowledge ranking 315 (e.g., a number from 0 to 100), overall experience level 320 (estimated number of years of experience), influence score 325 (e.g., a number from zero to five), and a messaging link 330. [0056] The code analysis 310 is based on analyzing code from code repositories to provide objective information regarding a minimum number of years of experience in a particular programming language as well as an objective analysis of the code itself to providing a ranking of the developer's skills. The full analysis includes the estimated number of years of experience with each language. Tabs are provided indicating top rankings (e.g., through a set useful to the end-user, such as top 10%, top 20%, top 30%, etc.). The number of views by others and the adoption of code by others may be used to generate the influence score 325 as a measure of how influential the programmer is.
[0057] Figure 3B illustrates a second portion of the profile for the developer. The employment history 410 of the developer is summarized when it is is available. For example, such employment information is sometimes (but not always) posted on public websites such as Linkedln. Social profile information 420 is provided, which may include links 425 to social media websites that the developer uses.
[0058] A bio summary 430 may be extracted from social media. Alternatively, in one embodiment, a software developer is permitted to check their profile and take ownership of their profile in the sense of providing some limited voluntary inputs, such as bio summary information, and also provide feedback on any errors.
[0059] A summary of projects 440 accessible in code repositories is also provided. The summary preferably also includes links to the code in the repository for each project 445 for users interested in performing a more detailed analysis of the code itself. Additionally, information about the project may be included such as file size in terms of number of lines of code, number of views by other developers, and number of collaborators.
[0060] Figure 5 illustrates in more detail a functional block diagram of a recruiting service 500 in accordance with an embodiment of the present invention. The recruiting service 500 may reside on one or more servers with associated processors and memory, wherein the computer code is stored on a computer readable memory. A database memory may be provided to store information for the recruiting service, including candidate profile information.
[0061] A crawler 505 is provided to crawl code repositories. For example, the crawler may use an API for code hosting sites such as GitHub and Stack Overflow. A new candidate profile generation module 510 determines whether the crawler has identified a new developer. If so, a profile ID is generated to build a new profile. A code file type analysis module 515 determined the file type of files being crawled. After the file type has been determined, the language-specific code analysis module is selected by module 520. Scoring and cheating detection is then performed by module 525. Profiles are stored in a profile information database 530. A social media access module 535 provides access to social media information sites and a social media aggregation module 540 correlates aggregated social media information for individual profiles. A messaging interface 545 is included in one embodiment as a means for recruiters to contact individual developers. However it will be understood the messaging interface 545 may be omitted in some implementations. The messaging may, for example, be brokered in the sense of cloaking the user information and email address of the recruiter during initial attempts to contact a developer. A recruiting search engine and graphical user interface module 550 is responsible for generating the graphical user interface that is provided for display on a user's computer.
[0062] The new candidate profile generation module 510 utilizes author information from crawled sites to detect that there is a new developer to be added to the system. Code repository sites include author information for each project. This author information is searched by the crawler. Each individual person with a profile has a unique ID. The unique ID is created the first time an individual programmer's name is discovered in crawling author information in code hosting sites. For example, when the crawler finds the names of people that have contributed code to a code hosting site, the system compares the unique ID from the network that the person is found on to the unique IDs in the database of the recruiting service for that network. If an ID doesn't exist, a new user ID is created.
[0063] The crawling process generated project information for each developer. One way to obtain project information for a particular person is specifically ask a code hosting site (or content site like Stack Overflow) for a list of projects for each developer. For example, this may be done through an API for sites such as GitHub and Stack Overflow.
[0064] The crawling is updated regularly and the profiles are refreshed according to a cycle. An exemplary refresh cycle is a two-week profile refresh cycle. That is to say, the update from the crawlers may be constant, but the profiles may be updated according to a schedule, such as every two-weeks. [0065] One aspect of the crawling process is that the source code for a particular project is downloaded for analysis. As illustrative examples, the source may be downloaded using technologies such as Git, SVN, Mercury, and CVS, which are technologies that allow for synchronization with the local computer of a code repository.
[0066] It is preferable to download all of the available information in a repository for analysis. However, note that the source code for a project may be in any one of a variety of different file types.
[0067] Downloaded files are then processed, starting first with the code file type analysis module 515. An individual file is analyzed to determine what's in it by looking at file extensions and the binary data or text that the file contains. Specific patterns in the source code are analyzed. For example, specific languages— like Ruby— always start with a certain few lines of code - e.g. the hashbang - so that by looking for the patterns of a specific set of keywords in the file permits the language to be identified. Additionally, the analysis of the code can include looking for the "magic number "— a set of bytes at the beginning of the file that indicates file type. For example, images always start with a specific byte configuration. The pattern associated with each different file type is checked until a match is found. The pattern matching may be performed, for example, using a sequence of if-then clauses to identify the file type of a particular file.
[0068] After the file type is determined, the language specific code analysis selection module 520 makes a selection of an evaluation tool or tools appropriate for the language of the file type. Most software languages have evaluation tools to evaluate the quality and complexity of the coding. The evaluation tools are specific to a particular language and may, for example, look at the length of the code and patterns in the code. For many cases, the evaluation tools for a specific language are open source and/or available from commercial vendors. For example, there is a unique set of tools to evaluate Ruby— tools that differ from those used to evaluate C++. Thus if the recruiting service is designed to analyze code in languages such as Java, Scala, Shell, ActionScript, XML, CSS, HTML, Groovy, PHP, Perl, Python, Lisp, etc. then the system includes the corresponding evaluation tool for each supported language. Thus, the recruiting service includes a wide range of evaluation tools to support different languages and makes the selection of the proper evaluation tool based on the file type. The file is then analyzed using the appropriate selected tool(s) for the language associated with the file type.
[0069] The scoring and cheating detection module 525 utilizes the evaluation of the code and also information from the commit log for the file. To identify the author of the code of a particular file, the commit log is evaluated for the repository. The commit log is a list of who did what for the repository. This permits an evaluation of the developer's specific contribution(s) to that project. For example, by analyzing the commit log an evaluation can be made of the time(s) when the developer made a contribution.
[0070] The process is continued for all of the developer's repositories to permit a determination to be made of what languages that a person has experience in and how much experience the person has in each language. Recruiting service downloads all the developer's repositories and evaluates their contributions to determine the languages they've written code in. To evaluate experience the commit log is examined to look at the date and time of when the developer contributed to the project. Different factors can be used to determine actual experience. One factor is that contributions can be evaluated by their frequency and regularity to weight the actual number of years of experience in a particular language.
[0071] As an illustrative example, consider a developer named Joe. If Joe started contributing to a project 3 years ago, and the commit log shows that he's been contributing regularly to it, that's an indication that he has 3 years of actual experience. Thus, a weighting function can take into account the frequency and regularity of Joe's contributions. For example, if Joe has been making four or more contributions per year that's an indication that Joe has been regularly working on Java.
[0072] However, if Joe made a single contribution 3 years ago and just contributed again for the first time 2 months ago, then the commit log indicates "episodic" contributions with a wide spacing between contributions. For this second case, the weighting factor can be used to reduce Joe's number of years of experience such that he does not get 3 years of experience credit. The exact weighting function chosen can be empirically determined based on common behavior patterns of software developers. For example, if the commit log shows Joe made a single contribution to a Java project 3 years ago and made a second smaller contribution a month ago there could be a possibility that Joe is either 1) trying to "inflate" his resume about the number of years of experience he has in Java; or 2) may have become aware of the recruiting service and is intentionally trying to trick the recruiting service. In this example, the weighting function may also include one or more rules to discount recent contributions, particularly those of a minor character, such as a minor code tweak or a contribution made with many other contributors.
[0073] Thus, while the raw data provides an indication of a maximum potential number of years of experience, a weighting function may include different factors related to frequency of contribution, size of contribution, and number of co- contributors to perform a weighting function to arrive at a more accurate
interpretation of the number of years of experience for a developer. The weighting function may be determined empirically, based on observations about the way software developers normally work, to optimize different weighting factors and periodically adjusted to discourage gaming of the system. Other types of gaming (such as posting the same code at different times on different sites or plagiarizing code from others) could also, in theory, be checked as part of a larger fraud detection function.
[0074] As an illustrative example, patterns in a commit log may be examined for suspicious factors of how the developer is developing his/her source code. The simplest example of a developer cheating is that the developer downloads someone else's source code, opens their own repository, and submits that same exact code to the new repository. In that case, there would be a huge update all at once, and then nothing else in terms of activity. This is inconsistent with normal commit log behavior in which a user normally makes a series of regular contributions over time. An honest developer would normally (except for perhaps extremely small projects) be consistently committing code they are developing for their project. As a result when a huge aberrant spike occurs in a commit log a presumption can be made that there is a high likelihood that cheating has occurred. In this case, the weighting function can severely or totally discount the project, i.e., give it extremely little or no credit.
[0075] As previously described, in one embodiment there are three kinds of scores that are calculated for each developer. These include knowledge, experience and influence. This level of scoring provides a variety of useful information to evaluate candidates. However it will be understand that the recruiting service could also be implemented with a subset of this set of scores.
[0076] The scores are preferably calculated on a language-specific basis and an overall basis. Language specific scores are useful to evaluate skills in a particular language. However, generating an overall score provides an additional indicator of a developer's talent.
[0077] An exemplary language-specific scoring process will now be described. In one implementation, to determine a knowledge score, an examination is made of lines of code and the number of repositories that the developer has contributed to. The score is then calculated by a function that weights the total number of lines of code in all of the different repositories. That is, a developer who has written more lines of code has more experience and credit is given for contributing to different repositories.
However, the number of lines of code can reach very large numbers. Thus, one way to score knowledge is apply a logarithm function based on the number of lines of code. As one example, a knowledge score for a developer can be generated using a natural log curve: ln(lines) x number of repositories, where this equation is a simplified equation to illustrate a general approach that one of ordinary skill in the art would further optimize for a particular implementation to optimize empirical results. Other variations based on a logarithm function are also possible and other factors could be included in determining a knowledge score.
[0078] An exemplary language-specific scoring process for experience looks at different factors indicative of experience and then weights the factors. For example, to calculate a total experience score the individual experience scores may be combined with work experience, i.e., Total Experience = Individual experience skill experience scores x work experience. Of course, many variations are possible in terms of weighting individual skills experience with work experience. In one implementation the process looks at the lines of code that have been written with particular attention to the lines of code written per day, in addition to commits per day, and the number of days of activity. A weighted function SUM(r) can be used to perform an initial analysis of experience in different skills. The scoring can be further weighted by work experience, resulting in simplified equation to determine a language-specific experience score: SUM(r) x Work Experience, where this equation is a simplified equation to illustrate a general approach that one of ordinary skill in the art would further optimize for a particular implementation to optimize empirical results.
[0079] An overall score can be calculating by weighting individual scores. An overall experience score can be determined as follows:
Experience = Individual experience skill experience scores x work experience.
[0080] An overall knowledge score can be determined using different weighting approaches. In one approach a logarithm function is used to weight the sum of different knowledge skill scores so that a high score requires the developer to have a wide variety of skills:
Knowledge = ln(SUM(Skill Knowledge)), where this equation is a simplified equation to illustrate a general approach that one of ordinary skill in the art would further optimize for a particular implementation to optimize empirical results.
[0081] The influence score is a measure of the developer's influence in the larger developer community. In one embodiment the influence score includes how much the developer's code influences other developers. Additionally the influence score may include the developer's influence in social media. For example, the influence score may include a component based on how a developer's projects have influenced others, based for example on the number of followers, forks, and contributors, which may be determined from data within code hosting repositories. However, an individual developer may have different influence in different languages, which has to be taken into account in determining an overall influence score. Additionally, the developers influence in social media may also be considered, such as weighting the influence in social media by a weighting function. For example, one measure of influence in social media is a Klout score. Thus an exemplary overall influence score may be determined as follows:
Influence = f(Klout) + ln(SUM(Skill Influence))
[0082] where a developer is given credit for influence in different skills may also be given some credit for social media influence and where this equation is a simplified equation to illustrate a general approach that one of ordinary skill in the art would further optimize for a particular implementation to optimize empirical results.
[0083] The scoring and weighting functions that are applied are determined empirically to give a desired distribution based, for example by examining what weighting functions give the best real-world results at a particular point in time for recruiters. Thus, for example, the actual constants used as weighting factors and aspects of the weighting functions may be varied based on feedback on the usefulness of the scoring for simulated or actual recruiting efforts.
[0084] For example, when calculating knowledge, one approach is to look for extensive experience in several languages. This is because in the real world highly knowledgeable developers have a broad range of experiences to draw upon and are skilled in different languages. Thus, even if a developer has top 10% scores in one or two languages, they cannot get a top 10% overall knowledge score, because that only happens when the developer has top-tier scores in several languages. That is to say, breadth counts.
[0085] The public code repositories are crawled on a regular basis. In one
embodiment all the raw data obtained from a crawl of the public code repositories is saved, except for the source code. That is, it is preferable to save the information obtained by analyzing the code, in addition to the source log itself. The next time the crawler encounters that repository, the source code is downloaded again, and a
"refresh" is made based on new contributions. Additionally, the logs are checked to determine the individual(s) that made the new contribution. Thus, if Michael has a repository with a project the system will also confirm from the log entries who made any new contributions. Thus if Luca makes a follow-on contribution to Michael's project, the follow-on contribution will be credited to Luca. This cross-checking of which individual made which new contribution to a project is useful to improve accuracy and reliability of the scoring.
[0086] The social media access module 535 and the social media aggregation module 540 provide a comprehensive set of social media links for each profile. The author information obtained from public repository sites such as GitHub and Stack Overflow may be incomplete or contain inaccuracies. However typically the author information will include at least an email address and perhaps also a name. This information can then be used to obtain additional social media information using commercial services such as Full Contact, Inc. of Denver, Colorado, Fliptop, Inc. of San Francisco, California and Rap Leaf of San Francisco, California. Many commercial services check by unique information, like email address, or a hash of the email address (a hash is a unique number generated by an email address. That way, companies can match users by email addresses, but protect their privacy by looking at hash numbers). In one embodiment a search of social media sites is performed of all of the sites listed under Full Contact Social Network Types (http://www.fullcontact.com/docs/). From this information profile information identifying the names of developers may be generated along with associated information. For example, work history may also be scraped from social networking sites.
[0087] Direct scanning of social media sites is also an option, such as the option of scanning sites such as Linkedln and Google Plus. However, there's usually not a one- to-one results process. For example, if a developer has a common name, such as "John Smith," a scan based on their name may turn up more than one hit. To find additional social media links for a particular profile it is thus desirable to look for multiple matching factors (location, title, company, name, etc.), and then calculate the probability that it's a match. If the probability is higher than a certain number, the system automatically merges the profiles. If the probability is less than that threshold, the system sends a notification that there needs to be a manual review process.
[0088] Once links to social media are identified for a developer they can be refreshed at a rate slower than other information in the public code repositories. Individuals typically add new social networks infrequently and the URLs of social media sites are generally static.
[0089] The graphical user interface discussed in this application includes a set of features that are useful in making recruiting decisions. However, it will be understood that subsets of these features may be used. That is, one of ordinary skill in the art would understand that variations in the graphical user interface to include variations of what has been described are possible.
II. RECRUITMENT ENHACMENT SYSTEM USING
PSYCHOLOGICAL, EMOTIONAL AND PERSONALITY INFORMATION FOR JOB RECRUITING
[0090] As will be described in more detail as follows, a recruitment enhancement system for electronic job recruiting system is described. Additionally, a system and method known as "Gild Oracle" is described for generating recruiting information on a prospect's psychological state, personality type, and emotional state. Gild Oracle may be used either separately or together with features of the recruitment enhancement system.
[0091] The recruitment enhancement system is configured to gather and analyze information about job applicants. One objective of the analysis is to identify job applicants most likely to successfully perform jobs associated with available job positions. For example, a job applicant's proficiency or knowledge pertaining to a particular aspect of a job can be assessed and scored, such as their knowledge or proficiency of a particular programming language for a programming job. The scoring can be used to rank applicants.
[0092] The information derived from the analyses, such as the scores, can be referred to as job suitability parameters. The job suitability parameters can be designed to aid job recruiters during the manual filtering process of job applicants. The recruitment enhancement system can be configured to generate an interface that allows job recruiters to access the job suitability parameters for various candidates.
[0093] The recruitment enhancement system can be configured to gather information used to derive job suitability parameters directly or indirectly. Direct information gathering can involve configuring an activity and then, getting a job applicant or a potential job applicant to engage in the activity. When the job applicant engages in the activity, information from the user's participation can be gathered and analyzed. For example, the job applicant can be asked to engage in a knowledge based contest related to a skill associated with a job. The job applicant's answers can be scored or evaluated in some manner to provide some indication of the applicant's skill level.
[0094] Indirect information gathering can involve harvesting information from a job candidate's online activities. One source of information that can be gathered an analyzed may be from an applicant's participation in social media activities. Another source of information that can be gathered and analyzed may be from an applicant's participation in professional related activities. Many potential information sources are available and these examples are provided for the purposes of illustration only and are not meant to be limiting.
[0095] As an example of an information source associated with professional related activities that can be used to generate a job suitability parameter, computer programmers often post publically available code related to personal projects. Their projects can be followed by other programmers. A recruitment enhancement system can be configured to gather and analyze the code to assess the programmer's abilities and suitability for various programming jobs. From the analyses, job suitability parameters can be derived that are made available to job recruiters. In one embodiment, a proprietary database can be maintained at the recruitment enhancement system that includes data about individuals whose skills have been analyzed in this manner.
[0096] The job recruiters can use the job suitability parameters, such as job suitability parameters derived for programmers stored in a proprietary database, in the next stages of the recruitment process. For example, the job recruiters can use the job suitability parameters to assemble a list of potential candidates they wish to vet in more detail and then possibly contact. The job suitability parameters can be used to rank each of the potential candidates. Thus, the job recruiters may use the job suitability parameters to establish an order in which to contact potential candidates.
[0097] In particular embodiments, the recruitment enhancement system can be configured to receive identifying information for a group of individuals from a job recruiter. As an example, a plug-in module can be provided for the recruiter's applicant tracking system (ATS) that generates and sends the identifying information related to the individuals to the recruitment enhancement system. In response to receiving the identifying information from the ATS, the recruitment enhancement system can be configured to determine whether one or more of the identified individuals from the ATS can be matched to individuals stored in one or more of the proprietary databases maintained at the recruitment enhancement system. For each of the individuals that are matched, a report can be generated. The report can include job suitability parameters for the matched individuals, including how the individuals rank relative to the other individuals stored in the proprietary database. The report can be returned to the recruiter.
[0098] With respect to the following figures, details of apparatus and methodology for generating and maintaining a proprietary database of skilled individuals in a recruitment enhancement system, where the skilled individuals can be scored or ranked, are described. In addition, recruiting services utilizing these proprietary databases is described. The skilled individuals can be members of a variety of different professions. In one embodiment, the skilled individuals can be software developers. However, the discussion in regards to the software developers is provided for the purpose of illustration only and is not meant to be limiting.
[0099] In particular, with respect to Figure 6, a recruitment ecosystem is described. Within the recruitment ecosystem, interactions involving individuals seeking jobs, recruiters seeking candidates, companies providing jobs and a recruitment enhancement system are discussed. With respect to Figure 7, a recruitment enhancement system is described. In particular, methods and apparatus for formulating a proprietary database of information relating to skilled individuals are discussed.
[0100] With respect to Figure 8, 9A, 9B and 10, methods and apparatus for recruiting services that leverage the proprietary databases maintained within the recruitment enhancement system are discussed. The recruiting services can involve receiving a list of individuals from a recruiter and attempt to match individuals from the list to individuals stored in the proprietary databases. Reports can be generated for matched individuals on the recruiters list and returned to the recruiter. The recruiter can use information in the report to further their recruiting activities.
Recruiting Ecosystem Including Recruitment Enhancement System
[0101] In this section a recruiting ecosystem including a recruitment enhancement system is described. In a recruiting ecosystem, individuals can engage with various electronic systems to supply information that allows them to apply for existing jobs, or to be considered for future jobs when positions open up. In response to the supplied information, recruiters can select from among the individuals and attempt to initiate additional job recruiting related interactions. For instance, based upon the received data, a job recruiter can identify an individual and attempt to contact the person to set up an interview.
[0102] A job prospect can be a person that is interested in working for a company and has agreed to provide information, but may not have applied for a particular job. A job applicant can be a person that has applied for a job, and agreed to provide information as part of the application process. As described herein, a recruitment enhancement system 12 can be provided, that can be used to gather information, and engage job candidates and job applicants alike. [0103] In the recruitment enhancement system, methods and apparatus can be provided for gathering secondary recruitment data that is beyond the resume and other application data that is usually gathered in the job application process, via applicant tracking systems. The secondary recruitment data can be analyzed and scored. Based upon the score, one or more rankings for a job candidate can be generated. In one embodiment, the secondary recruitment data and rankings can be stored in a proprietary database maintained at the recruitment enhancement system.
[0104] As an example of ranking or scoring, an individual in the recruitment enhancement system 12 can be ranked independent of, or relative to, a group of other job candidates or job applicants, such as a group of job applicants applying for the same job, or a group of job candidates within the same profession. An independent rank can involve comparing the individual's performance to some derived scale. A relative rank can involve comparing the individual's performance to other individuals within a group.
[0105] In one scenario, as will be described in more detail with respect to Figures 8, 9A, 9B and 10, a company can send information related to a group of individuals that have recently applied for job, have applied for a job in the past, or have provided information indicating their interest in working for the company. The information associated with these individuals can be stored in a company ATS (applicant tracking system). Upon receiving the information, the recruitment enhancement system can be configured to rank the group of individuals one or more different ways. For instance, individuals recently applying for jobs can be compared against individuals that have been hired by the company. In another example, the group of individuals can be compared against a group of individuals stored in a proprietary database associated with the recruitment enhancement system.
[0106] The recruitment enhancement system 12 can be configured to generate an interface that allows an outside entity, such as a job recruiter, to access the ranking data. The outside entity can use the ranking data as a basis for additional
investigations, such as learning more about particular individuals, and determining whether to contact particular individuals for additional interactions, such as a job interview. In one embodiment, as described in more detail with respect to Figure 8, the interface can be provided as a part of a plug-in application for their ATS. [0107] In particular embodiments, the recruitment enhancement system 12 can be tailored to particular professions. For example, the recruitment enhancement system 12 can be tailored towards the computer programming profession, where the system is configured to gather secondary recruitment data that allows computer programming professionals to be ranked according to some scale, and/or relative to other computer programming professionals that have participated in a common recruitment enhancement activity. A recruitment enhancement system associated with program developers is described below in more detail with respect to Figure 7. In another example, the recruitment enhancement system 12 can be tailored towards medical professionals, such as doctors or nurses, where the system is configured to gather secondary recruitment data that allows the doctors or nurses to be ranked.
[0108] Figure 6 is a block diagram of a recruitment ecosystem 50 including a recruitment enhancement system 12 in accordance with the described embodiments. The ecosystem 50 includes company systems, such as 6, recruiter systems, such as 8, a recruitment enhancement system, such as 12, and 3rd party systems where individuals can engage in various activities, such as 10. Social media sites are one example of 3rd party systems. An individual 2, who may or may not be currently engaged in the job hunting process may interact at various times with each of the systems via a computational device, such as the tablet device 4. Typically, the interactions may involve establishing communications between the computational device and one or more remote servers associated with each of the systems over a local and/or wide area network. For instance, the tablet device 4 can be used to establish communications and interact with a server associated with a social media site.
[0109] Communication devices typically include a processor, a memory, networking capabilities, and a user interface that allows for the input and output of data. Examples of communication devices that can be utilized include, but are not limited to, smartphones, laptop computers, netbook computers, desktop computers and tablet computers. Interface devices that can be utilized include a display, a keyboard, and a microphone in combination with speech recognition, a mouse, a touchpad and a touchscreen.
[0110] The recruitment enhancement system 12 can include components for a) direct gathering of recruitment data 24, b) generating a recruiter interface 26, c) indirect gathering of recruitment data 28, d) scoring and ranking 30, e) data matching 32, and f) data quality assessing. The system 12 can be hosted on one or more servers including processors, memory and network interfaces. Direct recruitment data gathering 24 can involve generating one or more activities in which an individual can directly participate, such as a knowledge-based test, a game, a puzzle or solving a problem. The direct recruitment data gathering can be used to identify and engage individuals that have interest in a company, or encourage individuals to be interested in a company. In addition, the direct recruitment data gathering can be used to possibly assess potential job candidates.
[0111] In general, the capturing of data from an individual directly engaging in an activity can be part of direct recruitment data gathering. For instance, an individual engaging in an activity, such as an interview, can be captured on video data as part of direct recruitment data gathering 24. In another example, when an individual takes a test, information such as their answers, how long they take to answer each question, and their total time can be gathered and analyzed.
[0112] The indirect recruitment data gathering can involve gathering information from an individual's on-line activities, such as participation in social media sites 36, professional sites 38 or even sites related to a user's hobbies. The person's on-line activities can be hosted on 3rd party systems 10, such as a social media site. In one embodiment, when a user agrees to participate in an activity involving direct recruitment data gathering, a user may agree to provide access to their on-line activity information which can be captured by component 28. For example, a user can provide access to their Facebook™ or Linkedln™ profile for a chance to participate in a knowledge-based test related to recruiting. The knowledge-based test can be referred to a recruitment enhancement activity. After access is granted, the indirect recruitment data gathering component 28 can retrieve information about their on-line activities from the 3rd party systems.
[0113] The information gathered by components 24 and 28 can be stored and analyzed. Then, job suitability parameters can be derived. For example, the scoring and ranking components 30 can be used to score information directly gathered during an online test and then rank individuals based upon the scores. In addition, information about an individual's online activities can be gathered indirectly from various third-party systems 10, and job suitability parameters can be derived from this information. For example, based upon software code that a user has posted to a professional site, the individual's coding ability can be scored and ranked. The software code may be executable. However, the software code doesn't have to be executable to generate a score or a ranking. Methods related to scoring code are described below with respect to Figure 7.
[0114] In particular embodiments, scores and ranks can be derived from data that is gathered directly and/or indirectly. For example, as will be described below, a score and/or a rank can be generated based upon an individual's participation in a knowledge-based test. In another example, a score and a rank can be generated based upon an individual's participation in a knowledge based test, and information retrieved from a third-party site, such as social media site. Again, a person can be ranked relative to a group of other individuals, such as a group of individuals participating in a common task, or according to some derived scale or measure.
[0115] The data matching component 32 can be used to match various data sets to a particular individual. For example, an individual may use multiple names in their online activities. As another example, different individuals can share a common name. The data matching component 32 may use information verified from one source to validate another information source. For example, if an individual grants access to their Facebook™ profile and then another source of information is available, such as their profile in a professional organization or data that they have supplied in applying for a job, the information from the two sources can be compared by component 32 to see if they are the same person. When the comparison indicates they are the same person, a portion of the information from the multiple sources can be stored at the system 12 or at least the links to the multiple sources can be stored. This information can be made available in a recruiter interface.
[0116] The data quality assessing component 34 can be used to assess a validity of gathered data. For example, the component 34 might be used to determine whether someone has cheated on a knowledge-based test administered by system 12 or tried to game the system 12 in some other manner. As another example, the component 34 might be used to determine whether a person has falsified data they have provided as part of a job application. Information that is identified as invalid may be flagged. In addition, the system 12 can be configured not to use information identified as possibly invalid for scoring and ranking purposes. [0117] The recruiter interface 26 can allow recruiters to directly access some portion of the information available at system 12. For example, via the interface, recruiters may be able to access scores and ranks about various individuals. Further, via the interface a recruiter may be able to search for individuals with particular skills and assess their job suitability parameters, such as scores and ranking. In another embodiment, as described in more detail with respect to Figure 8, a recruiter can access the system 12 via a plug-in module coupled to their ATS, such as 16 or 20.
[0118] The company 6 and the recruiter 8 can each maintain applicant tracking systems, such as 16 and 20. If a recruiter 8 works for the company 6, then the recruiter 8 may not maintain a separate applicant tracking system. The recruiter 8 may have access to an interface or interfaces that allows them to interact with the recruitment enhancement system 12, an individual 2 via their device 4 or some other mechanism, their own application tracking system 20 and a company application tracking system 16.
[0119] The company 6 can sponsor a job-site interface 16. Via the job site interface 14, an individual, such as 2, may be able to learn about different jobs and optional apply for jobs. Information from the job application process can be stored to the company application tracking system 16. In particular embodiments, the company 6 may allow recruiters, such as 8, and/or the recruitment enhancement system to access the application tracking system 16.
[0120] In one embodiment, one or more direct recruitment data gathering activities can be triggered from the job-site interface 14. For example, when a user applies for a particular job, such as after submitting their information or a part of submitting their information, a link can be established from the job-site interface 14 to the recruitment enhancement system 12. The recruitment enhance system 12 can then generate a recruitment enhancement activity that is used to directly gathered recruitment data. For example, as part of the recruitment enhancement activity an individual may be asked to take a knowledge-based test, play a game, solve a puzzle, solve a problem or combinations thereof.
[0121] Information captured from the recruitment enhancement activity can be scored or ranked and made available to recruiters via the interface generated by the recruiter interface generation component 26. An individual can participate in a number of different recruitment activities. Thus, the interface may allow a recruiter to view descriptions of the recruitment enhancement activities in which an individual has participated and jobs suitability parameters derived from their performance.
[0122] In one embodiment, via the interface, a recruiter may be able to send a request to an individual to participate in a particular recruitment enhancement activity. For instance, the recruiter may be interested in an individual for a particular job position. Via the interface, the recruiter can send a message to the individual indicating their interest in the person and requesting them to participate in a recruitment enhancement activity, such as taking a knowledge-based test associated with the job position. The message received by the individual can include information, such as a link to a web- site, which allows the individual to engage in the recruitment enhancement activity.
[0123] In one embodiment, the recruitment activity that is selected can be related to information associated with an available job. For example, a knowledge-based test can be implemented that is related to a skill needed for the job position. Their answers can be scored and the person can be ranked relative to other job applicants that have taken the test.
[0124] In other embodiment, the recruitment enhancement activity that is selected can be related to information supplied by the individual applying for the job. Via interface 14, an individual can submit information indicating that they possess a particular skill at a particular skill level, such as a number of years of experience in the skill. For instance, the interface can ask the person to indicate how many years of experience the person has with the skill or the person can submit a resume including this information.
[0125] Based upon information supplied by the individual, the system 12 can be configured to select a recruitment enhancement activity that in some way measures or is predictive of the person having the claimed skill. For example, an individual can be asked to take a knowledge-based test associated with their claimed skill. Harder or easier test can be implemented depending upon the person's claimed skill level. The test answers can be received by the system 12, scored and analyzed. The scores and analyses can be used to assess individuals in regards to their proficiency in the claimed skill. In addition, the individual can be ranked relative to other individuals with the same claimed skill level or according to a scale constructed for individuals of the claimed skill level. The system 12 can be configured to make information about an individual's score or rank in the recruitment enhancement activity available to recruiters.
Recruitment Enhancement System
[0126] Figure 7 illustrates in more detail a functional block diagram of a recruitment enhancement system 12 in accordance with an embodiment of the present invention. The recruitment enhancement system may reside on one or more servers with associated processors and memory, wherein the computer code is stored on a computer readable memory. A database memory may be provided to store information for the recruiting service, including candidate profile information.
[0127] In the example below, the generation of a database, which may be a proprietary database, is described. The database is focused on individuals that are software developers. The use of software developers is provided for the purposes of illustration only and databases associated with individuals in other skilled professions can also be generated and maintained at the recruitment enhancement system.
[0128] A crawler 105 is provided to crawl code repositories. For example, the crawler may use an API for code hosting sites such as GitHub. A new candidate profile generation module 1 10 determines whether the crawler has identified a new developer. If so, a profile ID is generated to build a new profile. A code file type analysis module 1 15 determined the file type of files being crawled. After the file type has been determined, the language-specific code analysis module is selected by module 120. Scoring and cheating detection is then performed by module 125.
[0129] Profiles are stored in a profile information database 130. A social media access module 135 provides access to social media information sites and a social media aggregation module 140 correlates aggregated social media information for individual profiles. A messaging interface 145 is included in one embodiment as a mechanism for recruiters to contact individual developers. However it will be understood the messaging interface 145 may be omitted in some implementations. The messaging may, for example, be brokered in the sense of cloaking the user information and email address of the recruiter during initial attempts to contact a developer. A recruiting search engine and graphical user interface (GUI) module 150 is responsible for generating the graphical user interface that is provided for display on a user's computer. In one embodiment, the recruiter's search engine and GUI module can be configured to interact with a plug-in module that is provided to the recruiter's ATS (e.g., see Figure 8).
[0130] The new candidate profile generation module 1 10 utilizes author information from crawled sites to detect that there is a new developer to be added to the system. Code repository sites include author information for each project. This author information is searched by the crawler. Each individual person with a profile has a unique ID. The unique ID is created the first time an individual programmer's name is discovered in crawling author information in code hosting sites. For example, when the crawler finds the names of people that have contributed code to a code hosting site, the system compares the unique ID from the network that the person is found on to the unique IDs in the database of the recruiting service for that network. If an ID doesn't exist, a new user ID is created.
[0131] The crawling process generated project information for each developer. One way to obtain project information for a particular person is specifically ask a code hosting site (or content site like Stack Overflow) for a list of projects for each developer. For example, this may be done through an API for sites such as GitHub and Stack Overflow.
[0132] The crawling is updated regularly and the profiles are refreshed according to a cycle. An exemplary refresh cycle is a two-week profile refresh cycle. That is to say, the update from the crawlers may be constant, but the profiles may be updated according to a schedule, such as every two weeks.
[0133] One aspect of the crawling process is that the source code for a particular project is downloaded for analysis. As illustrative examples, the source may be downloaded using technologies such as Git, SVN, Mercury, and CVS, which are technologies that allow for synchronization with the local computer of a code repository.
[0134] It is preferable to download all of the available information in a repository for analysis. However, note that the source code for a project may be in any one of a variety of different file types.
[0135] Downloaded files are then processed, starting first with the code file type analysis module 1 15. An individual file is analyzed to determine what's in it by looking at file extensions and the binary data or text that the file contains. Specific patterns in the source code are analyzed. For example, specific languages— like Ruby— always start with a certain few lines of code - e.g. the hashbang - so that by looking for the patterns of a specific set of keywords in the file permits the language to be identified. Additionally, the analysis of the code can include looking for the "magic number"— a set of bytes at the beginning of the file that indicates file type. For example, images always start with a specific byte configuration. The pattern associated with each different file type is checked until a match is found. The pattern matching may be performed, for example, using a sequence of if-then clauses to identify the file type of a particular file.
[0136] After the file type is determined, the language specific code analysis selection module 120 makes a selection of an evaluation tool or tools appropriate for the language of the file type. Most software languages have evaluation tools to evaluate the quality and complexity of the coding. The evaluation tools are specific to a particular language and may, for example, look at the length of the code and patterns in the code. For many cases, the evaluation tools for a specific language are open source and/or available from commercial vendors. For example, there is a unique set of tools to evaluate Ruby— tools that differ from those used to evaluate C++. Thus if the recruiting service is designed to analyze code in languages such as Java, Scala, Shell, ActionScript, XML, CSS, HTML, Groovy, PHP, Perl, Python, Lisp, etc. then the system includes the corresponding evaluation tool for each supported language. Thus, the recruiting service includes a wide range of evaluation tools to support different languages and makes the selection of the proper evaluation tool based on the file type. The file is then analyzed using the appropriate selected tool(s) for the language associated with the file type.
[0137] The scoring and cheating detection module 125 utilizes the evaluation of the code and also information from the commit log for the file. To identify the author of the code of a particular file, the commit log is evaluated for the repository. The commit log is a list of who did what for the repository. This permits an evaluation of the developer's specific contribution(s) to that project. For example, by analyzing the commit log an evaluation can be made of the time(s) when the developer made a contribution.
[0138] The process is continued for all of the developer's repositories to permit a determination to be made of what languages that a person has experience in and how much experience the person has in each language. Recruiting service downloads all the developer's repositories and evaluates their contributions to determine the languages they've written code in. To evaluate experience the commit log is examined to look at the date and time of when the developer contributed to the project. Different factors can be used to determine actual experience. One factor is that contributions can be evaluated by their frequency and regularity to weight the actual number of years of experience in a particular language.
[0139] As an illustrative example, consider a developer named Joe. If Joe started contributing to a project 3 years ago, and the commit log shows that he's been contributing regularly to it, that's an indication that he has 3 years of actual experience. Thus, a weighting function can take into account the frequency and regularity of Joe's contributions. For example, if Joe has been making four or more contributions per year, that's an indication that Joe has been regularly working on Java.
[0140] However, if Joe made a single contribution 3 years ago and just contributed again for the first time 2 months ago, then the commit log indicates "episodic" contributions with a wide spacing between contributions. For this second case, the weighting factor can be used to reduce Joe's number of years of experience such that he does not get 3 years of experience credit. The exact weighting function chosen can be empirically determined based on common behavior patterns of software developers. For example, if the commit log shows Joe made a single contribution to a Java project 3 years ago and made a second smaller contribution a month ago there could be a possibility that Joe is either 1) trying to "inflate" his resume about the number of years of experience he has in Java; or 2) may have become aware of the recruiting service and is intentionally trying to trick the recruiting service. In this example, the weighting function may also include one or more rules to discount recent contributions, particularly those of a minor character, such as a minor code tweak or a contribution made with many other contributors.
[0141] Thus, while the raw data provides an indication of a maximum potential number of years of experience, a weighting function may include different factors related to frequency of contribution, size of contribution, and number of co- contributors to perform a weighting function to arrive at a more accurate
interpretation of the number of years of experience for a developer. The weighting function may be determined empirically, based on observations about the way software developers normally work, to optimize different weighting factors and periodically adjusted to discourage gaming of the system. Other types of gaming (such as posting the same code at different times on different sites or plagiarizing code from others) could also, in theory, be checked as part of a larger fraud detection function.
[0142] As an illustrative example, patterns in a commit log may be examined for suspicious factors of how the developer is developing his/her source code. The simplest example of a developer cheating is that the developer downloads someone else's source code, opens their own repository, and submits that same exact code to the new repository. In that case, there would be a huge update all at once, and then nothing else in terms of activity. This is inconsistent with normal commit log behavior in which a user normally makes a series of regular contributions over time. An honest developer would normally (except for perhaps extremely small projects) be consistently committing code they are developing for their project. As a result when a huge aberrant spike occurs in a commit log a presumption can be made that there is a high likelihood that cheating has occurred. In this case, the weighting function can severely or totally discount the project, i.e., give it extremely little or no credit.
[0143] As previously described, in one embodiment there are three kinds of scores that are calculated for each developer. These include knowledge, experience and influence. This level of scoring provides a variety of useful information to evaluate candidates. However it will be understand that the recruiting service could also be implemented with a subset of this set of scores.
[0144] The scores are preferably calculated on a language-specific basis and an overall basis. Language specific scores are useful to evaluate skills in a particular language. However, generating an overall score provides an additional indicator of a developer's talent.
[0145] An exemplary language-specific scoring process will now be described. In one implementation, to determine a knowledge score, an examination is made of lines of code and the number of repositories that the developer has contributed to. The score is then calculated by a function that weights the total number of lines of code in all of the different repositories. That is, a developer who has written more lines of code has more experience and credit is given for contributing to different repositories.
However, the number of lines of code can reach very large numbers. Thus, one way to score knowledge is apply a logarithm function based on the number of lines of code. As one example, a knowledge score for a developer can be generated using a natural log curve: ln(lines) x number of repositories, where "x" is the multiplication operation and this equation is a simplified equation to illustrate a general approach that one of ordinary skill in the art would further optimize for a particular implementation to optimize empirical results. Other variations based on a logarithm function are also possible and other factors could be included in determining a knowledge score.
[0146] An exemplary language-specific scoring process for experience looks at different factors indicative of experience and then weights the factors. For example, to calculate a total experience score the individual experience scores may be combined with work experience. Of course, many variations are possible in terms of weighting individual skills experience with work experience. In one implementation the process looks at the lines of code that have been written with particular attention to the lines of code written per day, in addition to commits per day, and the number of days of activity. A weighted function SUM(r) can be used to perform an initial analysis of experience in different skills. The scoring can be further weighted by work experience, resulting in simplified equation to determine a language-specific experience score: SUM(r) x Work Experience, where this equation is a simplified equation to illustrate a general approach that one of ordinary skill in the art would further optimize for a particular implementation to optimize empirical results.
An overall score can be calculating by weighting individual scores. An overall knowledge score can be determined using different weighting approaches. In one approach a logarithm function is used to weight the sum of different knowledge skill scores so that a high score requires the developer to have a wide variety of skills: Knowledge = ln(SUM(Skill Knowledge)), where this equation is a simplified equation to illustrate a general approach that one of ordinary skill in the art would further optimize for a particular implementation to optimize empirical results.
[0147] The influence score is a measure of the developer's influence in the larger developer community. In one embodiment the influence score includes how much the developer's code influences other developers. Additionally the influence score may include the developer's influence in social media. For example, the influence score may include a component based on how a developer's projects have influenced others, based for example on the number of followers, forks, and contributors, which may be determined from data within code hosting repositories. However, an individual developer may have different influence in different languages, which has to be taken into account in determining an overall influence score. Additionally, the developer's influence in social media may also be considered, such as weighting the influence in social media by a weighting function. For example, one measure of influence in social media is a Klout score. Thus an exemplary overall influence score may be determined as follows:
Influence = f(Klout) + ln(SUM(Skill Influence))
where a developer is given credit for influence in different skills may also be given some credit for social media influence and where this equation is a simplified equation to illustrate a general approach that one of ordinary skill in the art would further optimize for a particular implementation to optimize empirical results.
[0148] The scoring and weighting functions that are applied are determined empirically to give a desired distribution based, for example by examining what weighting functions give the best real-world results at a particular point in time for recruiters. Thus, for example, the actual constants used as weighting factors and aspects of the weighting functions may be varied based on feedback on the usefulness of the scoring for simulated or actual recruiting efforts.
[0149] For example, when calculating knowledge, one approach is to look for extensive experience in several languages. This is because in the real world highly knowledgeable developers have a broad range of experiences to draw upon and are skilled in different languages. Thus, even if a developer has top 10% scores in one or two languages, they cannot get a top 10% overall knowledge score, because that only happens when the developer has top-tier scores in several languages. That is to say, breadth counts.
[0150] The public code repositories are crawled on a regular basis. In one embodiment all the raw data obtained from a crawl of the public code repositories is saved, except for the source code. That is, it is preferable to save the information obtained by analyzing the code, in addition to the source log itself. The next time the crawler encounters that repository, the source code is downloaded again, and a
"refresh" is made based on new contributions. Additionally, the logs are checked to determine the individual(s) that made the new contribution. Thus, if Michael has a repository with a project, the system will also confirm from the log entries which contributors made any new contributions. Thus if Luca makes a follow-on
contribution to Michael's project, the follow-on contribution will be credited to Luca. This cross-checking of which individual made which new contribution to a project is useful to improve accuracy and reliability of the scoring.
[0151] The social media access module 135 and the social media aggregation module 140 provide a comprehensive set of social media links for each profile. The author information obtained from public repository sites such as GitHub and Stack Overflow may be incomplete or contain inaccuracies. However typically the author information will include at least an email address and perhaps also a name. This information can then be used to obtain additional social media information using commercial services such as Full Contact, Inc. of Denver, Colorado, Fliptop, Inc. of San Francisco, California and Rap Leaf of San Francisco, California. Many commercial services check by unique information, like email address, or a hash of the email address (a hash is a unique number generated by an email address. That way, companies can match users by email addresses, but protect their privacy by looking at hash numbers). In one embodiment a search of social media sites is performed of all of the sites listed under Full Contact's set of Social Network Types. From this information profile information identifying the names of developers may be generated along with associated information. For example, work history may also be scraped from social networking sites.
[0152] Direct scanning of social media sites is also an option, such as the option of scanning sites such as Linkedln and Google Plus. However, there's usually not a one- to-one results process. For example, if a developer has a common name, such as "John Smith," a scan based on their name may turn up more than one hit. To find additional social media links for a particular profile it is thus desirable to look for multiple matching factors (location, title, company, name, etc.), and then calculate the probability that it's a match. If the probability is higher than a certain number, the system automatically merges the profiles. If the probability is less than that threshold, the system sends a notification that there needs to be a manual review process.
[0153] In one embodiment, an individual may have granted the recruitment enhancement system access to one of their profiles at a social media site. For example, to participate in contest, which is an example of a recruitment enhancement activity described with respect to Figure 6, the individual may have given the recruitment enhancement system permission to access their profile at Linkedln™. When permission is granted in this way, the recruitment enhancement system may be able to access more information about the individual than when social media sites are directly scanned. Once links to social media are identified for a developer they can be refreshed at a rate slower than other information in the public code repositories. Individuals typically add new social networks infrequently and the URLs of social media sites are generally static.
Recruiting Services Using a Recruitment Enhancement System
[0154] In this section, recruitment services that leverage information gathered and analyzed at the recruitment enhancement system are described. In addition, interfaces that allow outside entities to access these services are described. For instance, an interface can be provided for an applicant tracking system that allows a recruiter to access recruitment services, such as candidate scoring, from a recruitment enhancement system.
[0155] Figure 8 is a block diagram of recruitment ecosystem 200. The recruitment ecosystem 200 includes two applicant tracking systems (ATSs), 202 and 208 and a recruitment enhancement system 12 including a developer database 206. The developer database 206 includes information about software developers. This example is provided for the purposes of illustration only as the recruitment ecosystem 200 can include a plurality of different applicant tracking systems. In addition, as is described in more detail below, the recruitment enhancement system 12 can be configured to provide access interfaces that don't require an ATS. Further, database 206 can include information about individuals from other professions other than software developers and/or separate databases (not shown) can be maintained for individuals in other professions.
[0156] The applicant tracking systems, 202 and 208, can include data associated with a plurality of individuals that have applied for jobs over some period of time. In applying for a job, each individual may have supplied information that is stored to an ATS. For example, an individual may have electronic uploaded a resume in some format. After the resume is uploaded, the resume may be processed. For example, optical character recognition can be applied to a scanned in image of the resume. [0157] After processing, information from the resume can be parsed and stored to an ATS database. The format of the database including the record structure and the information stored in the database can vary from ATS system to ATS system. For example, ATS 202 can store a first set of applicant information in a first order and ATS 208 can store a second set of applicant information in a second order.
[0158] As another example, the ATSs may also include information about individuals that have expressed interest in the company through some mechanism, such as a social media application. For example, via a social media application a user may have requested interest in a company and supplied a name and e-mail to receive some type of information. As another example, via a social media application, a user may have provided a social media profile to participate in a company related activity.
Information obtained via a mechanism, such as a social media application, can be stored to the ATS. Thus, in general, there are many different scenarios in which information can be gathered about an individual and then entered into an ATS and information obtained via the job application process is only one example in which information about an individual can be provided to an ATS.
[0159] The ATSs can be associated with different entities. For example, ATS 202 can be associated with a first company where it includes information for applicants that have applied for jobs at the first company and ATS 202 can be associated with a second company where it includes information for applicants that have applied for jobs at the second company. As another example, ATS 202 can be associated with a first recruiter that has performed recruitment searches for a first group of companies and ATS 208 can be associated with a second recruiter that has performed recruitment searches for a second group of companies different from the first group. Thus, individuals in one ATS may have applied for jobs at different companies serviced by a recruiter or group of recruiters.
[0160] In some instances, information stored in each of the ATSs can overlap. For instance, when ATS 202 and ATS 208 are associated with companies in the same field, it is possible that the same individual has applied for jobs at each of the companies. Thus, ATS 202 and ATS 208 can include information, such as resume information associated with the same individual. However, because companies and recruiters usually closely guard the information stored in their ATS and don't generally share the information, neither of the entities controlling ATS 202 or ATS 208 is likely to know about the overlaps between their ATS databases.
[0161] Using an ATS database, such as 202 and 208, a recruiter can perform searches over the records in the ATS database. A search can allow a recruiter to identify a group of records that satisfy particular search criteria. For example, if the database stores information related to an individual's experience level with a particular programming language, then a recruiter can perform a search for candidates satisfying the experience level requirements for a particular job. The outcome of the search can be a list of candidates meeting the search criteria and information about each candidate.
[0162] After searching, a recruiter can choose to contact various individuals for additional scrutiny, such as interviews. The recruiter may continue to contact individuals represented in the ATS until a job position is filled. After the job position is filled, the information about individuals that were not hired may still remain in the ATS. Thus, search results can include information about job applicants that have applied for a particular job opening, information about individuals that have previously applied for other jobs and individuals that have been entered into the ATS through other mechanisms, such as via a social media application.
[0163] In particular embodiments, information about all or a portion of the candidates in an ATS can be sent to the recruitment enhancement system 12. As an example, in 210, information about one or more individuals stored in ATS 202 can be sent to recruitment enhancement system 12. As another example, in 214, information about one or more individuals stored in ATS 208 can be sent to the recruitment enhancement system 12.
[0164] The information sent from the ATS to the recruitment enhancement system 12 can be used to provide a recruitment enhancement service. In various embodiments, as will be described in more detail below, one recruitment enhancement service can involve scoring or ranking individuals, such as software developers, which are identified from the information received from ATS in a recruitment service request. Some examples of information that can be used to identify an individual that may be stored in an ATS and sent to system 12 are described with respect to Figure 9A. Some examples of information that can be returned about an individual from system 12, such as but not limited to a score or a rank are described with respect to Figure 9B. [0165] Next, some of the information that can be sent to system 12 as part of a recruitment service request is described with respect to Figure 1 OA. Then, additional details about Figure 8 are described followed by a description of Figure 10B. The individual in Figure 9A can be one among group of individuals whose information is sent in 210 or 214. In a communication between an ATS and system 12, identification information about each individual, such as a first name and a last name 302, can be sent. In addition, one or more e-mail addresses, 304 and 306 can be sent that are believed to be associated with the identified individual. In one embodiment, the recruitment enhancement system requires a first name, last name and at least one e- mail address to provide a recruitment service involving assessing a skill level of the individual.
[0166] For competitive purposes and for privacy reasons, a company may not wish to release too much information about an individual. An individual name and e-mail address or addresses are likely to be publically available. Thus, an advantage of sending only a name and email addresses is it allows the company to maintain that it is protecting the privacy of individuals because only a limited amount of information is sent and what is sent is likely to be publically available.
[0167] Besides the name and the e-mail address or addresses, one or more additional types of information can be sent. The recruitment service may involve determining whether the recruitment enhancement system 12 has additional information about the named individual. Individuals can share names. Thus, the additional information may allow the system to confirm with a higher level of confidence that a named individual from an outside entity is properly identified by system 12.
[0168] Some example of additional information include but are not limited to location information 308, blog information 310, social profile information 312, nicknames 314, user names 316 and profile names. The location information 308 may be an address where the person lives or works. In one embodiment, the location information 308 may be a partial address, such as a city where the individual lives or works. The blog information 310 is example of information associated with an individual's on- line activities. For example, information 310 can be associated with a user's online blog, such as link (e.g., a URL) to the blog.
[0169] In another example, information 312 may be associated with an individual's on-line profile. For example, a link to a Linkedin™ profile or information obtained from the on-line profile can be sent. Multiple instances of information 312 can be sent. For instance, a first link to a first profile and a second link to a second profile can be sent. The system 12 can be configured to parse the received information when multiple instances of particular type of information or no instances of a particular type of information, such as 312, are received.
[0170] In other examples, information, such as nicknames 314, usernames 316 or profile names 318, associated with a user's on-line activities can be provided to system 12. As will be described in more detail as follows, the received information can be used to determine whether the named individual also has a record in the developer database 206 maintained by the recruitment enhancement system 12. In general, any information available at the ATS that can help to locate information about the individual in the developer database 206 or other database maintained at the recruitment enhancement system can be sent.
[0171] The information that is sent can vary from individual to individual. For example, for a first individual, a name and one e-mail address can be sent. For a second individual, a name and two e-mail addresses can be sent. For a third individual, a name, one e-mail address and a link to a social profile. For a fourth individual, a name, two e-mail addresses and a number of different user names can be sent. Thus, the recruitment enhancement system 12 can be configured parse and utilize different combinations of information that vary from individual to individual.
[0172] In a particular embodiment, the recruitment system can be configured to receive information about an unnamed individual. For example, the first name and last name 302 may not be known. Based upon the received information, the system 12 can be configured to determine a possible name of the individual and return the information to the entity that sent the information about the unnamed individual.
[0173] The information described above doesn't necessarily have be stored in or sent from an ATS. In one embodiment, the received information is used to locate information about individuals in a database maintained by the recruitment enhancement system 12. For this purpose, it doesn't matter where the information resides before it sent as long as it is suitable for locating records of individuals maintained in the database or databases of system 12. Thus, the example of an ATS as a source of the information sent to the recruitment enhancement system is for the purposes of illustration only and is not meant to be limiting. [0174] In one embodiment, besides information about the individuals, information about a job position for which candidates are desired can be sent. For example, a portion of a job description can be sent in a recruitment service request. In one embodiment, when the system 12 receives a job description and a list of a number of individuals, the system 12 can be configured to store information about the individual and information about the job position for which they are being considered to a database maintained at system 12.
[0175] Returning to Figure 8, in one embodiment, a plug-in module, such as 204 and 206, can be provided for an ATS. The plug-in module can be configured to be compatible with the recruitment enhancement system 12. The plug-in modules can be configured to extract information from an associated ATS and send it to the recruitment enhancement system in format that is recognizable by system 12.
[0176] In one embodiment, the plug-in module may provide an export feature. For instance, an export button can be located within an interface provided with the search functions of the ATS. In response to a search, a selection of the export feature can cause an electronic data file to be generated that is compatible with the recruitment enhancement system 12. The electronic data file can include information consistent with the search, such as the names and e-mail addresses of the individuals identified in the search. As described above with respect to Figure 9A, additional information related to a particular individual can also be included.
[0177] An API (Application Program Interface) can be provided with system 12. The API can describe formats and particular data types that system 12 can accept. The API can be used by third-party developers to construct a plug-in module for an ATS.
[0178] The extracted information can be sent to the recruitment enhancement system in 210 or 214. The information can be sent as part of a recruitment service request. The plug-in module may require a user to provide an additional input before the data is sent, such as a selection of a send button. In one embodiment, the plug- in module, such as 204 or 206, can be configured to establish electronic communications with the recruitment enhancement system 12 and then send data to the recruitment enhancement system 12 that has been extracted from an ATS. The data can be sent electronically via a secure connection between the ATS and the recruitment enhancement system 12. The information that allows the secure connection to be established may have already been exchanged by the systems. [0179] In one embodiment, entities can be charged according to their utilization of the recruitment enhancement system 12. Thus, the data sent from the ATS, such as 202 or 208, can include identification and authorization information for an entity requesting a service. The authorization information can be used to determine whether the entity sending the data is authorized to obtain a requested service from the recruitment enhancement system 12. The identification information may allow the recruitment enhancement system to attribute its utilization to the entity and bill it for the services.
[0180] Although not shown, the recruitment enhancement system 12 can be configured to provide an interface, such as web-based interface, that allows an entity to upload information that is compatible for receiving a service from system 12, such as a file including a list of names and e-mail addresses to the system. The system 12 can be configured to process the file and generate a report. Upon payment, entity can receive the report. The report can be emailed to an address specified by the entity or a link can be provided that allows the entity to download the report. Thus, a plug-in module that provides an interface to an ATS may not be required.
[0181] After receiving the individual information in 210 and 214, the system 12 is configured to determine whether it has information about the received individual's in its developer database 206. Possible results of this determination are shown graphically in Figure 8. The largest circle represents individuals in the developer database 206. Group 218 can include one or more individuals from ATS 202. It can be seen that a circle associated with group 218 partially lies within the large circle associated with 206 and partially outside of it. The portion inside the circle represents individuals in group 218 that can be matched to individuals in the developer database 216. The portion outside the circle 206 represents the portion of individuals that can't be matched to individuals within the developer database 206 with an acceptable degree of confidence.
[0182] In this example, most of the individuals in group 218 were matched to individuals described in database 206. In other embodiments, it is possible no matches may be found. In yet other embodiments, it is possible that all of the individuals in a group, such as 218, can be matched to individuals in the database 206. Thus, the fractions shown in Figure 8 are for the purposes of illustration only and are not meant to be limiting. [0183] The system 12 can maintain a number of databases that identify individuals. The individuals can have different job skills and may be members of different professions. At some point, the system 12 may have received information that allows a score or a rank to be developed for the individual. In various embodiments, the scores or ranks can be an indicator of such factors, as the quality of their one or more job skills, their influence within their profession and their effectiveness at their profession. Details of generating these scores and ranks for software developers are described above with respect to Figure 7.
[0184] When reporting the results of a search, the system 12 can report which individuals received in a group, such as 218, were matched to an individual in at least one database maintained at 12 and which individuals were not matched. For named individuals in a group that are matched to someone in the developer database 206 a confidence measure can be determined. The confidence measure may be based upon the amount of information that is matched. For example, a number of points can be given for a last name match, a number of points can be given for a first and last name match and a number of points can be given for each e-mail address match. The more information that is matched the higher score and hence the higher level of confidence that the named individual is a match for an individual in the developer database 206.
[0185] In one embodiment, when an individual is not matched or the confidence measure is determined to below a certain threshold, the system 12 can be configured to initiate a search and attempt to locate information about a named individual in a group that allows them to be scored or ranked or raises the confidence measure of the match to an individual in the database 206. A named individual newly scored or ranked can be added to the database 206 and their score/rank can be reported in reply to the service request, such as in 212 or 216. When a confidence measure of a match has been increased as a result of search such that it exceeds threshold value for reporting the result, then the result can be reported in a reply to the recruitment service request.
[0186] In one embodiment, when a named individual can't be located in the database 206 and information about the named individual can't be found which allows the individual to be scored or ranked, the system 12 can still be configured to keep a record of the individual. The system may keep the information so that future searches can be carried out in which attempts are made to gather information on the named individual where the gathered information can be used to develop a score or rank for the individual. Further, if the named individual shows up again as a result of a request from another entity to score the individual, then the system 12 can report that an individual may be actively looking for a job based upon the fact that the system 12 has received information about the individual from multiple entities. This information can be provided even if a score or rank is not available for the named individual.
[0187] In one embodiment, when a match is not found for a named individual in a service request and a record of the unmatched named individual is stored, the system can be configured to attempt to match the named individual at a later time. It is possible a match may be subsequently found and a score or ranking may become available for the named individual that was not previously matched. The system 12 can be configured to report the score or ranking to one or more entities that requested a score or ranking for the named individual that originally were notified that a score or ranking was not available for the individual or the confidence level of the match was below a threshold value when a score or ranking becomes subsequently available. This feature may be a system option in that a service requester can specify whether they wish to be updated or not if additional information is subsequently discovered about a named individual, such as whether a score or ranking has been developed for the name individual that was previously not available.
[0188] In general, the system can be configured to allow the recruitment service requester to specify whether or not they want updates regarding named individuals that have been previously submitted in a service request, such as 210 or 214. For example, the system can be configured to maintain a watch list for an entity including a number of named individuals in a group. In response to specified triggers, the system 12 can be configured to notify the entity. For instance, the system can notify an entity, such as recruiter, when it is determined a named individual on the watch list appears to be searching for a job. In another example, the system can be configured to notify an entity, when a score or ranking of an individual on the watch list has changed in some manner, such as exceeding some threshold value or changing by some percent amount.
[0189] The notifications from system 12 can occur through one or more different communications channels. For example, an e-mail can be sent to a recruiter. In another example, a text message can be sent to a recruiter's portable electronic device. In yet another example, an e-mail and text message can both be sent.
[0190] The system 12 can be configured to receive in a request for a recruitment service subsets of named individuals grouped in some manner. The named groups may be used for comparison purposes. These different subsets are represented by the circles in Figure 8 that are smaller than the largest circle. As described above, the largest circle can represent individuals in the developer database 206 and portions of the smaller circles lying outside of the largest circle represent named individuals in a group that have not been matched to an individual in the developer database 206.
[0191] In one example, the recruitment service request 210 for ranking/scoring of individuals can include groups 218, 220, 230 and 232. The named individuals in group 218 are possible job candidates. These candidates may have applied for a job over a first time period, such as in the last year. Group 220 may be individuals that are distinct from other members of group 218 in some manner, such as having applied for a job in the last month or having applied for a job multiple times. Group 230 can include individuals that have been offered a job. Group 232 can include individuals that have been hired.
[0192] In another example, the recruiting service request 214 can include three specified groups, 224, 226 and 228. Group 224 can be a group of individuals that have applied for jobs in between 6 months to a year ago. Group 226 can be individuals that have applied more recently for a job or jobs, such as within the last month. Group 228 can represent a current employees of a company that perform a job or jobs that are similar to the job or jobs for which the named individuals in group 228 are applying.
[0193] Scores/ranks can be provided for each individual in a group as well as for the group as whole (e.g., an average score for the named individuals in a group can be provided). The scores/ranks can be used for various comparison purposes. For example, the score of a named individual in the recent 220 group can be compared to the average score or rank of the hired individuals in group 232 or the average score or rank of the individuals that were offered jobs in group 230. Similar comparisons can also be made between individuals in groups 224, 226 and 228. For example, scores/ranks of a named individual in group 226 that has recently applied for a job can be compared to scores/ranks of individuals in group 228 or an average of scores/ranks for individuals in group 228. In another example, the average rank/scores of the individuals in group 224 that have applied 6 to 12 months ago can be compared to another group, such as individuals in group 220 that have applied for a job in the last month.
[0194] In one embodiment, this comparison feature can be used as a filter. For example, the system 12 can be configured to only return information on named individuals in a group whose score or rank is above the average of the named individuals of a comparison group. As another example, the system 12 can be configured to only return information on named individuals in a group whose score/rank is above some threshold value, such as the average score/rank of the named individuals in a group.
[0195] In the examples in Figure 8, some of the named individuals that are submitted in different recruitment service requests can overlap. For instance, some of the named individuals in group 218 can also be named individuals in group 224. In Figure 8, the overlap of members for groups 218 and 224 is represented by the intersections of the circle 222. In example in Figure 8, the entity associated 202 and the entity associated with 208 may not be aware of their individual recruitment service requests and the fact one or more individuals are in each request as different entities, such as companies don't share this information. However, when a named individual is identified from multiple recruitment service requests, the system 12 can be configured to keep track of and leverage this information.
[0196] As an example, the system can be configured to record each time an individual appears in a recruitment service request. If a named individual appears multiple times over some period, it can indicate the candidate is looking for a job. If the named individual is highly ranked, this factor can be used as a trigger to notify a recruiter. For instance, as described above, a recruiter can place an individual on a watch list and request to be notified under some conditions. One of the conditions can be if the individual appears to be looking for a job which may be determined from their appearance in multiple recruitment service requests. The system 12 can be configured to allow a user to specify parameters that determine under what conditions they are to be notified.
[0197] In a recruitment service request, such as 210 or 214, a large number of names can be submitted. For example, tens of thousands of names can be submitted. The processing of a large group may take a significant amount of time. In one embodiment, the system 12 can be configured to provide results of a recruitment service request all at once. For example, the system 12 can be configured to process all of the named individuals in a recruitment service request and then return the results in a batch mode when request is completed. In another embodiment, the system 12 can be configured to send a portion of the results in real-time before the request is completed. For example, when a match is made for a named individual in a recruitment service request, the results can be returned as soon as the match has been made. As another example, when a match is made for a named individual meeting certain criteria (e.g., a ranking or score about some specified value), the results can be returned as soon as the match is made. The remaining results including or not including individual results previously sent can be returned in a batch mode when the matching process for the recruitment service request is completed.
[0198] In one embodiment, the plug-in, such as 204 or 206, can be configured to automatically submit recruitment service requests to the system 12. For example, each time an individual applies for a job and their information is entered into the ATS 202, plug-in 204 can be configured to submit a recruitment service request to system 12. When a match is found, the system 12 can return information, such as a score or ranking, about the individual. The returned information may be stored to the ATS. In another example, the recruitment service request can be triggered after some number of applicants have applied, such as after receiving 10 applications. Then, the plug-in module can send a recruitment service request including ten names to system 12. In another example, the recruitment service request from the ATS can be triggered at some time interval, such as once a day.
[0199] In the example above, one or more named individuals are described as being sent in a recruitment service request to system 12, the system 12 tries to match the named individual to an individual stored in a database, such as database 206, and then information, such as scores for matched individuals can be returned. The use of the service is described with respect to recruiting individual candidates for jobs.
However, the use of the system is not limited to recruiting individual candidates for jobs.
[0200] For example, in one embodiment, the system 12 can be used to rank or score the strength of a team, such as a software development team. An employer may have a number of software development teams for different products. The individuals on each team can be submitted to system 12 and the development team can be scored. Based upon the scores, a manager may decide to reformulate the groups. For instance, some development teams have much higher scores than other development teams and a manager may reshuffle the team members such that the scores for the teams are more balanced.
[0201] In another example, a company may wish to hire a team of developers for a project. The company may receive bids from different contractors. The company can request the names of individuals that are going to be working the project for each contractor and submit their names to the recruitment enhancement system 12. The system can return scores for each of the development teams for each contractor. Based upon the scores/rankings provided for each of the team members as well as the scores as a group, the company can decide which contractor to hire for the project.
[0202] In yet another example, a company, such as a software contractor, may have their software developers ranked/scored by system 12. The system 12 can generate individual scores. The individual scores can be used to develop company scores.
Then, the company may use individual scores and/or the company scores to promote the technical expertise of their employees.
[0203] As described above in response to a recruitment service request including a number of named individuals, such as 210 or 214, results which may include scores and/or ranks can be returned to the requester in 212 and or 216. An example of what information can be returned is described as follows with respect to Figure 9B. Other types of information can be returned and the example provided in Figure 9B is for the purposes of illustration only.
[0204] In one embodiment, the individual output data 350 can include some set of baseline information and optional information. As an example, the baseline information can include a link to a profile 352 or the information contained in a profile. The profile 352 can be a profile collated from various information sources to which system 12 has access, such as various on-line sources. The baseline information can include one or more scores, such as 354 and 356 (e.g., see description of Figure 7 above for more details), skills possessed by the individual and an indicator 360 of the confidence measure of the match. The confidence measure can be indicated as a score, such as a numerical score, or according to some scale, such as high, medium or low. [0205] In one embodiment, the system 12 can be configured to provide standard scores. In another embodiment, the system 12 can be configured to provide customized scores. As described above, the scores can be determined using a number of different weighting factors. The system can be configured with specifiable parameters that allow a person using the system, such as a recruiter, to influence the scores in some manner. For instance, more weight can be put on experience as opposed to some other factor, such as experience. The system 12 may allow a recruiter to vary the weights from search to search but may report for each search what weights were used.
[0206] In another example, a company may wish to develop custom scores to suit specifics of their recruitment needs. The custom scores can be utilized for all of the company's reports from the system 12. In one embodiment, a plug-in module for an ATS, described above with respect to Figure 8, can be configured to implement custom scoring for a company such that all of the searches involving the ATS will use the custom scoring.
[0207] Returning to Figure 9B, some examples of optional information include one or more social profile links 362, additional contact information 364 and a current job status. The formulation of the baseline information is provided for illustrative purposes only and different combinations of baseline information and optional information can be provided. Additional information that can be provided, an interface for outputting the information and example of a profile that can be returned is described in more detail in U.S. patent application no. 13/499,791, entitled RECRUITING SERVICE GRAPHICAL USER INTERFACE, incorporated herein by reference.
[0208] In one embodiment, the system 12 can be configured to receive parameters that allow a user to customize the information and the format of the information that is received in 350. The customization may be implemented on a request by request basis, implemented globally as part of the plug-in module or implemented on a user by user basis For example, a first user can specify one set of preferences and a second user may specify a second set of preferences for receiving information 350 where the preferences can affect the type of information that is received and its format.
[0209] In one embodiment, for a large job including information about many individuals, the system 12 can electronically send a report or a link to location that allows the report to be downloaded when the report is ready. In addition, the plug-in modules, such as 204 and 206, can be configured to add all or a portion of the information in the report to an ATS. For example, scoring results can be integrated into the ATS such that future searches involving the ATS can return scoring information without having to contact the recruitment enhancement system 12. The plug-in module can be configured to periodically communicate with the recruitment enhancement system to update the scores or ranks stored to the ATS.
[0210] Figure 10 is a flow chart of a method 400 of a recruitment enhancement system. In 402, the system can analyze and score developers as was described above with respect to Figure 7. The analysis and scoring process can be an on-going activity where scores are updated with some frequency. The results from the scoring and analysis process for various individuals can be stored to a developer database, such as 206 in Figure 8.
[0211] In 404, the system can receive a recruitment service request. The request can include information, such as a name and an e-mail address, for a number of different individuals. Different groups of individuals can be specified in the request. The specified grouping can be used for comparison purposes. For example, individuals in a first group can be compared to individuals in a second group. In 406, a search can be carried out to determine whether individuals named in the recruitment service request can be matched to individuals in a database, such as a developer database that was populated in 402. In one embodiment, the database can be a proprietary database.
[0212] In 408, in response to the recruitment service request, the database can be updated. For example, the fact that a match was determined for an individual in the database, when the match occurred and the entity requesting information about the individual can be stored. If a job description was included in the request, then information about the position can be stored. If many matches for an individual occur over some period as a result of the system receiving different recruitment service requests, the system may determine that the matched individual is looking for a job. In another example, even if a match is not found, a record can be created in the database that includes information about the individual for which a match was not found, such as their name, e-mail address and the entity the provided the information.
[0213] In 410, a notification event can be triggered. For example, a recruiter may place an individual on a watch list where if the system determines the individual is looking for a job then the recruiter is to be notified. Other parameters can be utilized to define a triggering event and the example of determining that the individual is applying for jobs is provided for illustrative purpose only. For example, another triggering event for notification can be if a score or ranking associated with the individual reaches some threshold value or changes by some amount.
[0214] In 412, the system can perform any requested comparisons and scorings, such as a team scoring or individual to group comparisons. In 414, the system can determine confidence measures for any determined matches. In one embodiment, individual results may only be reported when the confidence measure exceeds some threshold. In 416, the system can compile and send a response to the recruitment service request including the results, such as scores. For example, a report can be generated and a link to the report can be sent to a recruiter. The report can include overall statistics associated with the request, such as but not limited to a total number of matches, a total number of non-matches and an average rank/score of the matched individuals.
[0215] In 418, the system can tally billable events and bill the client accordingly. For example, in one embodiment, a billable event can be a match with a confidence level above a certain threshold. A client can be charged some amount for each match satisfying these criteria. Thus, in response to recruitment service requests involving one or more successful matches, a client can be billed according to the number of successful matches.
Generating Recruiting Information On A Candidate's Psychological State, Personality Type, and Emotional State
[0216] Figure 1 1 illustrates aspects of a user interface 600 for an embodiment of the invention in which recruiting information is supplemented with psychological, emotional, and personality factors determined from online content sources, such as social media. Social media broadly includes any content authored by a candidate that is posted online and which is publicly available, including blogging and personal websites.
[0217] The user interface and associated back-end process for generating the information in user interface 600 is currently known by the name of "Gild Oracle." Gild Oracle may be used by itself or its information may be used in combination with the other types of information described in the previous examples discussed in this patent application. ,Links to the candidate's social media or other Internet works authored by the candidate may be listed. The user interface may, for example, be output in response to a query from a recruitment service requester, either alone or in combination with ranking information. In one implementation, the user interface 600 preferably includes a candidate's name. Additionally, another option is to include other information about the candidate's work related skills and experience, which may be generated using any of the previously described techniques.
[0218] The candidate's self-authored writings (in social media or other online sources) is analyzed to identify psychological, emotional, and personality factors relevant to recruiting the candidate. This additional information may be used to determine an optimum time with respect to the candidate's state of mind to reach out to recruit the individual. Additionally, this information may be used to determine personality information that may be relevant to how a recruiter reaches out to the candidate in terms of approach or in terms of selecting a job that is most likely to be of interest to the candidate in terms of a fit for their personality.
[0219] In one embodiment, the recruitment enhancement system 12 generates the psychological, emotional, and personality factors by accessing and downloading the content of the social media or other links obtained for a candidate. In one embodiment this additional information is used to augment conventional "hard" recruiting information, based on skill sets and work experience. More generally, however, a candidate could be identified and the candidate's social media links and other online content links identified using a variety of techniques, such as receiving a candidate's name and then, crawling the Internet to search for links to sites having materials authored by the candidate and downloading content authored by the individual.
Examples of additional information that may be determined from a candidate's writings include a Meyer-Briggs personality analysis 605, other personality analysis measures (e.g., psychological age, masculine/feminine attributes) 610,
formal/informal writing percentage 615, recurring words 620 in the candidate's writings, recurring bigrams 630 in the candidate's writings, topics 635 in the candidate's writings (e.g., the topics may represent the individual's interests as determined from their own writings), languages used 640 in the candidate's writings, openness/interest in seeking a new job or a particular type of job 645, and sentiment 650. [0220] Generally speaking, Figure 1 1 merely illustrates an exemplary listing of psychological, emotional, and personality aspects derived from a candidate's social media or other online media. Any subset could be used. However, providing a combination of different types of information greatly aids a recruiter. For example, providing information on the topics of interest to the candidate may indicate interest in hobbies, technical interests, or other information that helps to round out a fuller picture of the candidate. For example, the topics may include technical or professional interests relevant to whether a candidate might be interested in a particular job opening.
[0221] Additionally, the listing of recurring words, bigrams, and topics also provides an indication of the candidate's psychological, emotional, and personality aspects. For example, a candidate may have topics indicative of a psychologically healthy, outgoing extroversive individual, such as a listing of charities and social activities that the candidate writes about (e.g., "horse riding," "cancer society walk," etc.).
Alternatively, a candidate may have topics indicative of psychological introversion. As another example, a list of topics may include interests indicative of a risk taking personality (e.g, "skydiving"), or those more consistent with a risk-averse personality. As other examples, the topics may provide indications of other aspects of the psychological state of a candidate including changes in the ranking of certain topics over time. Thus, the topics of interest provide a wealth of information about a candidate's psychological, emotional, and personality aspects.
[0222] As previously mentioned, in many recruitment scenarios, a candidate is a prospect in that the person has not formally applied for a job. The recruiter in such a scenario is thus searching for individuals to contact. Note that under the labor laws of many countries, the regulations for the type of information used to identify a prospect are generally less stringent than using acquired information after an individual has applied for a job. Even stricter standards may apply in some countries regarding the types of information that may be used to evaluate current employees. Moreover, individual countries have different laws regarding how self-authored writings of an individual may be used in different employment scenarios. Additionally, there have historically been certain professions in which a condition of employment included the expectation that a very wide range of personal information could be collected and evaluated (e.g., psychological testing for firefighters). Thus, the information types displayed in Figure 1 1 are merely illustrative. The example of Figure 1 1 may be tailored to comply with all local labor laws, not only for identifying prospects but also for other applications such as evaluating an individual who has applied for a job. Moreover, the example of Figure 1 1 may be tailored for evaluating a current employee in an organization.
[0223] The analysis of content preferably uses filtering techniques to verify authorship, such as filtering out re-posts on social media (e.g., filtering out "retweets" of Twitter® postings). Additionally, the filtering may include verification tests to verify that the content is associated with a single individual and not misattributed from other individuals having similar names. The analysis of the content may be performed over a selected time range and trend analysis may also be performed if there is sufficient historical data.
[0224] Referring to Figure 12, in one embodiment a portion of the recruitment enhancement system 12 (or other computer system including a processor and a non- transitory computer readable memory storing instructions) utilizes a spider to crawl links to content authored by the candidate (e.g., social media) in step 705. The Internet is crawled with a spider and content is downloaded to a database (DB) and associated with an individual user in step 710. As examples, the content may include text extracted from social media, such as blogs and Twitter® posts. Text equivalents of video or audio posts may also be generated. Additionally, the content may include articles available on the Internet written by the individual.
[0225] The collection of content may be performed in view of other contextual information, such as crawling the Internet searching for individuals having certain educational qualifications, job-related qualifications, or business opportunity qualifications to form an initial pool of potential candidates from which additional information for specific individuals is desired. Alternately, the collection of data may be performed for a specific individual whose identity is already known.
[0226] The individual's choice of words in an individual piece of content has many aspects related to the psychological, emotional, and personality characteristics of the user. For example, the frequency with which certain words are used and certain patterns of words may be indicative of the personality of the user or the current mood of the user. That is, whenever an individual creates media content the user makes conscious and unconscious choices of words such that the associated text tends to reflect aspects of the individual's mood, personality type, temperament, and way of viewing and interacting with the world.
[0227] A natural language module may be used in step 715 to process the text. The text can be manipulated using natural language processing, such as determining a frequency of detected keywords and correlations and semantic associations between words. An exemplary tool for performing natural language processing of text is the Natural Language Toolkit (NLTK) from the NLTK Group (via the Internet organization http://nltk.org/) for use with Python® programming. NLTK provides a suite of processing libraries for performing computational linguistics for linguistic classification, tokenization, stemming, tagging, parsing, and semantic reasoning.
[0228] A Machine Learning (ML) module is used in step 720 to classify the text associated with a particular individual and obtain additional information to assess the potential interest and potential fit of an individual for a job. ML is branch of artificial intelligence which includes algorithms enabling software to learn. ML is used to classify content associated with a particular individual. ML programs perform classification based on training to find a pattern based on the training. ML typically generates results in which there is certain probability of a particular classification being correct. If the result is over a specific threshold, a particular classification is assigned.
[0229] The ML module can use commercially available software programs adapted to be specifically trained to perform the desired classification function for job recruiting. One example of Machine Learning software is the Apache Mahout® software, which includes a ML library. A variety of psychological, emotional, and personality attributes can be determined by ML techniques. As an example uClassify® is a web service that includes APIs and to classify text based on classifications such as sentiment (positive or negative), mood (happy or upset), age, sex, and Meyer-Brigg Personality type.
[0230] A training phase is used to train the ML module. This may, for example, be performed by experts in specific linguistic or psychological areas to train the ML to perform classification of a particular factor relevant to making a recruiting decision. Alternatively, crowd sourcing technologies may be used. For example, the Amazon Mechanical Turk (MTurk) is a crowd sourcing marketplace that allows requesters to coordinate the use of human intelligence to perform tasks. Using a crowd sourcing technology, such as Mechanical Turk, tweets or sentences taken from blogs may be shown to Mechanical Turk workers and then the workers asked to classify (rate) the sentences, such as whether the sentence expresses happiness, sadness, or rage.
[0231] As illustrated in step 725, the machine learning can classify the information into a variety of different classifications. As illustrative examples, the choice of words that an individual uses in their writings may be linguistically processed and then classified for mood (e.g., happy or upset), sentiment (positive or negative
(pessimistic) changes in state of mind), style (formal or informal writing style), psychological age (e.g., as an indicator of emotional maturity), sex (e.g., masculine or feminine psychological attributes), and topics of interest.
[0232] Additionally, the classification may include an analysis of the languages of the content (e.g., English, German, French), which may permit a determination of the native language of the candidate and whether the individual is multi-lingual.
Additionally, the classification may include psychological personality type (e.g., Myers-Briggs MBTI, such as Extrovert/Introvert, Judging/Perceiving,
Sensing/Intuition, and Feeling/Thinking). The classification may also be performed for specific business issues, such as whether the individual is looking for a new job, open to new opportunities, or is interested in a specific opportunity (e.g., corporate job or startup job).
[0233] Long term trend analysis may be performed by analyzing how classification information changes over time, as illustrated in step 730. Some psychological attributes change slowly over an adult's lifetime. However, other psychological attributes, such as mood and sentiment, can change rapidly over time. Analyzing trends can be useful in making recruiting decisions. An alarm threshold may be set for one or more parameters. In one implementation mood, either alone or in combination with sentiment, is calculated over time. Individuals are particularly likely to seek new jobs when they have shifted from being happy and positive (optimistic) to being unhappy and pessimistic. As a result, an alarm threshold can be set to indicate when an individual is particularly likely to be open to changing jobs.
[0234] Another option is to determine if the classification information matches a desired profile, as indicated by step 735. The classification information may be used directly, or alternatively, the classification information may be compared to a selection profile. In one embodiment, a selection profile may include psychological and personality factors related to a particular type of job. For example, a job position in which the prospect would be doing marketing work might favor the candidate being extroverted. Additionally, in some embodiments, the profile may also include skill rankings or work experience. For example, in some situations it may be desirable to determine both if an individual satisfies objective job skill qualification and also has emotional and psychological attributes indicating that they are likely to consider changing jobs.
[0235] Additionally, the classification information may also be correlated with the content or other information in step 740. In one embodiment, an analysis of recurring topics and/or recurring bigrams in the content may be included, which optionally may include links to portions of the content discussing the topic or bigram. As another example, the classification information may be correlated to information about the company the individual currently works at, such as information on stock price or acquisition rumors.
[0236] The information that is generated using the method may be stored in a memory for future use or dynamically updated for display in response to a particularly query. As previously discussed, the information may be displayed on a user interface either alone or in combination with other recruiting information in response to a query from a requester.
[0237] Figure 13 illustrates a user interface 800 providing psychological, emotional, or personality aspects relevant to networking with a prospect for social, business, or professional reasons. For example, an entity may be interested in seeking work or forming business ventures or collaborations with a decision maker in another organization. Thus, Figure 13 and 14 broaden the concept from recruitment to a variety of networking applications such as identifying psychological characteristics, mood and sentiment of a person that is a prospect for social, business, or professional interaction. As illustrative examples, an individual may want to acquire information to determine the best person or optimum time to make a business introduction or business pitch. In the non-profit arena, an individual may want to acquire information to determine a person or optimum time to solicit a charitable donation for a charity or a political donation for a political campaign.
[0238] Figure 14 illustrates a method, analogous to that of Figure 12, for using machine learning techniques to classify the writings of a networking prospect. Machine learning techniques are used to analyze social media or other content available for the prospect. For example, if the prospect is a decision maker at a company, the machine learning utilizes social media, or other online writings authored by the decision maker, to acquire information, such as mood or
psychological personality type, to determine whether and when to contact the decision maker.
[0239] Referring to Figure 14, as illustrated in step 905, content is acquired for the prospect and identified with them (step 910). Natural language processing is performed in step 915. Machine learning classification is performed in step 920 to classify the content for information relevant to potential networking with the prospect. The initial classification information 925 may include information to that previously described, except that it may also classify the content for information specific to classifications related to networking, such as looking for networking opportunities, openness to new collaborations, etc. Long term trend analysis may be performed in step 930. A determination may be made in step 935 if the prospect matched a desired profile. Additionally, an optional correlation with the original content may be performed.
III. RECRUITMENT ENHANCEMENT SYSTEM INCLUDING A
MODULE FOR GENERATING AND ENCOURAGING INDIVIDUALS TO PARTICIPATE IN RECRUITMENT ENHANCEMENT ACTIVITIES, SUCH AS KNOWLEDGE-BASED CONTESTS
[0240] As will be described in more detail as follows, a recruitment enhancement system for electronic job recruiting system is described. The recruitment enhancement system is configured to gather and analyze information about job applicants or prospective candidates for jobs. One objective is to engage individuals to promote knowledge and interest in a company. The engaged individuals may eventually become prospective job candidates. Another objective of the analysis is to identify job applicants most suitable for available job positions. For example, a job applicant's proficiency or knowledge pertaining to a particular aspect of a job can be assessed and scored, such as their knowledge or proficiency of a particular programming language for a programming job. The scoring can be used to rank applicants. [0241] The information derived from the analyses, such as the scores, can be referred to as job suitability parameters. The job suitability parameters can be designed to aid job recruiters during the manual filtering process of job applicants. The recruitment enhancement system can be configured to generate an interface that allows job recruiters to access the job suitability parameters for various candidates.
[0242] The recruitment enhancement system can be configured to gather information used to derive job suitability parameters directly or indirectly. Direct information gathering can involve configuring an activity and then getting an individual, such as a job applicant or a potential job applicant to engage in the activity. During the engagement process, the individual may learn about and become aware of a company as a result of their participation in the activity. Further, the individual may provide information that allows the company to learn more about the individual. Based upon the learned information, a decision can be made as to whether to attempt to further engage the individual.
[0243] A general trivia contest that is sponsored by a company is an example of a direct information gathering activity. The contest can be branded with information about the company, such as a company name and logo, so that the participant becomes aware of the company. Before beginning the contest, the participant can be required to provide identification information, such as logging in with their
Facebook™ or Linkedln™ account. Thus, the contest sponsor may be able to gather additional information about the participant.
[0244] Some direct information gathering activities may allow the skills of an applicant to be assessed. When the individual, such as a job applicant, engages in the activity, information from the user's participation can be gathered and analyzed. For example, the job applicant can be asked to engage in a knowledge based contest related to a skill associated with a job. The job applicant's answers can be scored or evaluated in some manner to provide some indication of the applicant's skill level. At the very least, as described above with respect to the general trivia contest, the activity may have increased awareness about the sponsor and the sponsor may have gained information about individuals that are potentially interested in them.
[0245] Indirect information gathering can involve harvesting information from a job applicant's online activities. One source of information that can be gathered an analyzed may be from an applicant's participation in social media activities. Another source of information that can be gathered and analyzed may be from an applicant's participation in professional related activities. Many potential information sources are available and these examples are provided for the purposes of illustration only and are not meant to be limiting.
[0246] As an example of an information source associated with professional related activities that can be used to generate a job suitability parameter, computer programmers often post publically available code related to personal projects. Their projects can be followed by other programmers. Via the recruitment enhancement system, the code can be gathered and analyzed as a component of a recruitment enhancement system to assess the programmer's abilities and suitability for various programming jobs. From the analyses, job suitability parameters can be derived that are made available to job recruiters.
[0247] The description to follow is primarily focused on direct information gathering and its analysis as part of a recruitment enhancement system. For example, methods and apparatus for providing knowledge based contests involving direct participation of job applicants are described. Information derived from the contests can be used to assess a job applicant's suitability for a particular job. Methods and apparatus associated with indirect information gathering that can be used in a recruitment enhancement system are described in more detail with respect to co-pending U.S. application no. 13/499,791, entitled "Recruiting Service Graphical User Interface," by Bonmassar, et al., filed June 1 1, 2012, which is incorporated herein in its entirety and for all purposes.
[0248] In particular, a block diagram of a recruitment ecosystem including a recruitment enhancement system is described with respect to Figure 15. Various scenarios in which information is directly gathered by the recruitment enhancement system for the purposes of generating job suitability parameters are discussed. With respect to Figure 16, a method of job recruiting using a recruitment enhancement system configured to generate activities used to gather information about potential job applicants is described. With respect to Figures 17- 19, a number of pages of an administrative interface for configuring a knowledge-based recruitment activity used in the recruitment enhancement system are discussed. Finally, with respect to [0249] Figures 20-27, a number of different interface states involving a user participating in a recruitment enhancement activity implemented on a portable electronic are described. Recruiting Ecosystem
[0250] In this section a recruiting ecosystem including a recruitment enhancement system is a described. In a recruiting ecosystem, individuals can engage with various electronic systems to supply information that allows them to apply for existing jobs or to be considered for future jobs when positions open up. In response to the supplied information, recruiters can select from among the individuals and attempt to initiate additional job recruiting related interactions. For instance, based upon the received data, a job recruiter can identify an individual and attempt to contact the person for additional interactions, such as to set up an interview.
[0251] A job candidate can be a person that is interested in working for company and has agreed to provide information. A job applicant can be a person that has applied for a job and agreed to provide information as part of the application process. As described herein, a recruitment enhancement system 12 can be provided that can be used to gather information and engage job candidates and job applicants alike. In addition, the recruitment enhancement system 12 can be configured to gather information about individuals that have engaged with a company in some manner. For example, the individual may have learned about the company through participation in a direct information gathering activity sponsored by the company, such as a contest of some type.
[0252] In the recruitment enhancement system, methods and apparatus can be provided for gathering secondary recruitment data that is beyond the resume and other application data that is usually gathered in the job application process. The secondary recruitment data can be analyzed and scored.
[0253] Based upon the score, one or more rankings for a job candidate can be generated. For example, an individual in the recruitment enhancement system 12 can be ranked independent of or relative to a group of other job candidates or job applicants, such as a group of job applicants applying for the same job or a group of job candidates within the same profession. An independent rank can involve comparing the individual's performance to some derived scale. A relative rank can involve comparing the individual's performance to other individuals within a group.
[0254] The recruitment enhancement system 12 can be configured to generate an interface that allows an outside entity such as a job recruiter to access the ranking data. The outside entity can use the ranking data as a basis for additional
investigations, such as learning more about particular individuals, and determining whether to contact particular individuals for additional interactions, such as a job interview.
[0255] In particular embodiments, the recruitment enhancement system 12 can be tailored to particular professions. For example, the recruitment enhancement system 12 can be tailored towards the computer programming profession where the system is configured to gather secondary recruitment data that allows computer programming professionals to be ranked according to some scale and/or relative to other computer programming professionals that have participated in a common recruitment enhancement activity. In another example, the recruitment enhancement system 12 can be tailored towards medical professionals, such as doctors or nurses, where the system is configured to gather secondary recruitment data that allows the doctors or nurses to be ranked.
[0256] Figure 15 is a block diagram of a recruitment ecosystem 50 including a recruitment enhancement system 12 in accordance with the described embodiments. The ecosystem 50 includes company systems, such as 6, recruiter systems, such as 8, a recruitment enhancement system, such as 12 and 3rd party systems where individuals can engage in various activities, such as 10,. Social media sites are one example of 3rd party systems. An individual 2, who may or may not be currently engaged in the job hunting process may interact at various times with each of the systems via a computational device, such as the tablet device 4. Typically, the interactions may involve establishing communications between the computational device and one or more remote servers associated with each of the systems over a local and/or wide area network. For instance, the tablet device 4 can be used to establish communications and interact with a server associated with a social media site.
[0257] Communication devices typically include a processor, a memory, networking capabilities and a user interface that allows for the input and output of data. Examples of communication devices that can be utilized include but are not limited to smartphones, laptop computers, netbook computers, desktop computers and tablet computers. Interface devices that can be utilized include a display, a keyboard, a microphone in combination with speech recognition, a mouse, a touchpad and a touchscreen.
[0258] The recruitment enhancement system 12 can include components for a) direct gathering of recruitment data 24, b) generating a recruiter interface 26, c) indirect gathering of recruitment data 28, d) scoring and ranking 30, e) data matching 32 and f) data quality assessing. The system 12 can be hosted on one or more servers including processors, memory and network interfaces. Direct recruitment data gathering 24 can involve generating one or more activities in which an individual can directly participate, such as a knowledge-based test, a game, a puzzle or solving a problem.
[0259] In general, the capturing of data from an individual engaging in an activity can be part of direct recruitment data gathering. For instance, an individual engaging in an activity, such as an interview, can be captured on video data as part of direct recruitment data gathering 24. In another example, when an individual takes a test, information such as their answers, how long they take to answer each question and their total time can be gathered and analyzed.
[0260] The indirect recruitment data gathering can involve gathering information from an individual's on-line activities, such as participation in social media sites 36, professional sites 38 or even sites related to a user's hobbies. The person's on-line activities can be hosted on 3rd party systems 10, such as a social media site. In one embodiment, as is described in more detail below with respect to Figures 20-27, when a user agrees to participate in an activity involving direct recruitment data gathering, a user may agree to provide access to their on-line activity information which can be captured by component 28. For example, a user can provide access to their
Facebook™ or Linkedln™ profile for a chance to participate in a knowledge-based test related to recruiting. The knowledge-based test can be referred to a recruitment enhancement activity. After access is granted, the indirect recruitment data gathering component 28 can retrieve information about their on-line activities from the 3rd party systems. [0261] The information gathered by components 24 and 28 can be stored and analyzed. Then, job suitability parameters can be derived. For example, the scoring and ranking components 30 can be used to score information directly gathered during an online test and then rank individuals based upon the scores. In addition, information about an individual's online activities can be gathered indirectly from various third-party systems 10 and job suitability parameters can be derived from this information. For example, based upon executable code that a user has posted to a professional site, the individual's coding ability can be scored and ranked.
[0262] In particular embodiments, scores and ranks can be derived from data that is gathered directly and/or indirectly. For example, as will be described below, a score and/or a rank can be generated based upon an individual's participation in a knowledge-based test. In another example, a score and a rank can be generated based upon an individual's participation in a knowledge based test and information retrieved from a third-party site, such as social media site. Again, a person can be ranked relative to a group of other individuals, such as a group of individuals participating in a common task, or according to some derived scale or measure.
[0263] The data matching component 32 can be used to match various data sets to a particular individual. For example, an individual may use multiple names in their online activities. As another example, different individuals can share a common name. The data matching component 32 may use information verified from one source to validate another information source. For example, if an individual grants access to their Facebook™ profile and then another source of information is available, such as their profile in a professional organization or data that they have supplied in applying for a job, the information from the two sources can be compared by component 32 to see if they are the same person. When the comparison indicates they are the same person, a portion of the information from the multiple sources can be stored at the system 12 or at least the links to the multiple sources can be stored. This information can be made available in a recruiter interface.
[0264] The data quality assessing component 34 can be used to assess a validity of gathered data. For example, the component 34 might be used to determine whether someone has cheated on a knowledge-based test administered by system 1. As another example, the component 34 might be used to determine whether a person has falsified data they have provided as part of a job application. Information that is identified as invalid may be flagged. In addition, the system 12 can be configured not to use information identified as possibly invalid for scoring and ranking purposes.
[0265] The recruiter interface 26 can allow recruiters to access some portion of the information available at system 12. For example, via the interface, recruiters may be able to access scores and ranks about various individuals. Further, via the interface a recruiter may be able to search for individuals with particular skills and assess their job suitability parameters, such as scores and ranking.
[0266] The company 6 and the recruiter 8 can each maintain applicant tracking systems, such as 16 and 20. If a recruiter 8 works for the company 6, then the recruiter 8 may not maintain a separate applicant tracking system. The recruiter 8 may have access to an interface or interfaces that allows them to interact with the recruitment enhancement system 12, an individual 2 via their device 4 or some other mechanism, their own application tracking system 20 and a company application tracking system 16.
[0267] The company 6 can sponsor a job-site interface 16. Via the job site interface 14, an individual, such as 2, may be able to learn about different jobs and optional apply for jobs. Information from the job application process can be stored to the company application tracking system 16. In particular embodiments, the company 6 may allow recruiters, such as 8, and/or the recruitment enhancement system to access the application tracking system 16.
[0268] In one embodiment, one or more direct recruitment data gathering activities can be triggered from the job-site interface 14. For example, when a user applies for a particular job, such as after submitting their information or a part of submitting their information, a link can be established from the job-site interface 14 to the recruitment enhancement system 12. The recruitment enhance system 12 can then generate a recruitment enhancement activity that is used to directly gathered recruitment data. For example, as part of the recruitment enhancement activity an individual may be asked to take a knowledge-based test, play a game, solve a puzzle, solve a problem or combinations thereof.
[0269] Information captured from the recruitment enhancement activity can be scored or ranked and made available to recruiters via the interface generated by the recruiter interface generation component 26. An individual can participate in a number of different recruitment activities. Thus, the interface may allow a recruiter to view descriptions of the recruitment enhancement activities in which an individual has participated and jobs suitability parameters derived from their performance.
[0270] In one embodiment, via the interface, a recruiter may be able to send a request to an individual to participate in a particular job enhancement activity. For instance, the recruiter may be interested in an individual for a particular job position. Via the interface, the recruiter can send a message to the individual indicating their interest in the person and requesting them to participate in a recruitment enhancement activity, such as taking a knowledge-based test associated with the job position. The message received by the individual can include information, such as a link to a web-site, which allows the individual to engage in the recruitment enhancement activity.
[0271] In one embodiment, the recruitment activity that is selected can be related to information associated with an available job. For example, a knowledge-based test can be implemented that is related to a skill needed for the job position. Their answers can be scored and the person can be ranked relative to other job applicants that have taken the test.
[0272] In other embodiment, the recruitment enhancement activity that is selected can be related to information supplied by the individual applying for the job. Via interface 14, an individual can submit information indicating that they possess a particular skill at a particular skill level, such as a number of years of experience in the skill. For instance, the interface can ask the person to indicate how many years of experience the person has with the skill or the person can submit a resume including this information.
[0273] Based upon information supplied by the individual, the system 12 can be configured to select a recruitment enhancement activity that in some way measures or is predictive of the person having the claimed skill. For example, an individual can be asked to take a knowledge-based test associated with their claimed skill. Harder or easier test can be implemented depending upon the person's claimed skill level. The test answers can be received by the system 12, scored and analyzed. The scores and analyses can be used to assess individuals in regards to their proficiency in the claimed skill. In addition, the individual can be ranked relative to other individuals with the same claimed skill level or according to a scale constructed for individuals of the claimed skill level. [0274] The system 12 can also be configured to implement recruitment enhancement activities in scenarios other than applying for a job. For example, contests involving recruitment enhancement activities, such as knowledge based tests can be implemented as part of a job fair. Individuals can be enticed to participate in the contest via some method, such as for a prize. Information about the individual's participation in the activity associated with the contest can be captured and evaluated by system 12. Based upon the evaluation, individuals can be scored or ranked. Prizes can be awarded based upon their achieved score or rank.
[0275] The system 12 can be configured to make information about an individual's score or rank in the recruitment enhancement activity available to recruiters. For instance, the information can be made available via an interface generated by the system. Recruiters, such as 8, may subsequently use this information to determine whether to contact the individual, such as in regards to their interest in an internship or a job at the company. Additional details of scenarios involving the recruitment enhancement system are described below with respect to Figures 17-27.
Method of Job Recruiting in a Recruitment Enhancement System
[0276] With respect to Figure 16, a method 100 of job recruiting in a recruitment enhancement system is described. In 102, a recruitment enhancement activity can be configured. The recruitment enhancement activity can be configured to generate one or more job suitability parameters that allow the suitability of different individuals for a particular job to be assessed. For example, the recruitment enhancement activity can be scored and individuals that have participated can be ranked relative to one another.
[0277] The scoring and ranking information can serve as job suitability parameters that allow a recruiter to filter potential to be recruited candidates for a job. One example of recruitment enhancement activity can be a knowledge-based test. An example of administration interface for configuring a knowledge-based test is described as follows with respect to Figures 17-19.
[0278] In one embodiment, as part of a configuration process, a user may have to download an application. For instance, a user can download an application from an app store to their smart phone. In other embodiments, the user may simply navigate to a web-site to participate in a recruitment enhancement activity. The web-site can be configured for generating output on a mobile device, such as via a browser application executing on the mobile device.
[0279] In 104, a determination can be made that recruitment enhancement activity has been triggered. For example, an individual may simply make a request to engage in the activity, such as to take a knowledge-based test. In another example, as described above, an individual can apply for job, such as at a company web-site. At some point during the application process, a recruitment activity can be triggered, such as in response to the user submitting particular information. In yet another example, a recruitment enhancement activity can be triggered via some interaction performed with an electronic device. For instance, a user can scan a QR code or place their device near a wireless interface, such as a near field communication interface to trigger the activity.
[0280] In 106, a recruitment enhancement activity can be selected for the user. For instance, if an individual is applying for a job, a recruitment enhancement activity can be selected that is related to the job where different jobs may utilize different enhancement activities. For example, a first knowledge-based test can be
implemented in response to an application for a first job and a second knowledge- based test can be implemented in response to an application for a second job. In other embodiments, a user may select a particular recruitment enhancement activity in which they wish to engage.
[0281] In yet another embodiment, a recruitment enhancement activity can be selected for an individual based upon information the individual has provided. For example, an individual can provide information about particular skills. The recruitment enhancement system can be configured to select or generate a recruitment enhancement activity that can be used to measure whether the person actually possess the skill they claim. For instance, if an individual claims to have a particular programming skill a knowledge-based test related to a particular programming language can be selected.
[0282] In 108, the recruitment enhancement activity can be instantiated and the system can monitor user interactions during the activity. In 110, the participant interactions can be scored and assessed. The scores can be used to rank participants in some manner. In some instances, the participant can be notified of their score and a rank, such as a rank relative to other participants. For example, as described below with respect to Figures 20-27, in 1 12, a leader board can be generated. In other instances, the participant may not be told how well they performed individually or how well they performed relative to other participants. However, this information can be generated and made available to other entities, such as job recruiters.
[0283] The system can keep track of whether an individual has already participated in activity. In some instances, depending on the activity the user may only be allowed to participate in a recruitment enhancement once. For example, an individual may only be allowed to take a knowledge-based test once. In other examples, an individual may be allowed to partake in a particular recruitment enhancement activity multiple times and potentially improve their score or rank.
[0284] In 1 14, the system can gather indirect recruitment enhancement data. For instance, to participate in the recruitment enhancement activity, a user may have been required to grant access to some information at a social media site, such as a profile at Linkedln™, which can be gathered. In 116, recruitment enhancement data gathered directly or indirectly can be stored. Optionally, links to some of the data can be stored. For example, rather than storing an entire profile at a social media site. A link to the profile can be stored.
[0285] In 1 18, based upon a participants score, a recruiter can be notified. For example, if an individual does well at recruitment enhancement activity at a conference. A recruiter, also at the conference, can be notified immediately so that they may potentially meet with the individual. In another example, a recruiter can receive a list of individuals that have participated in a recruitment enhancement activity involving a contest. The recruiter can receive a list of individuals that did well in a contest that was held over a particular time period. An example of such a contest is described as follows with respect to Figures 14-27.
Interface for Configuring Recruitment Enhancement Activity
[0286] In this section, an interface for configuring a recruitment enhancement activity that can be used in a recruitment enhancement system is described. The interface can be used to configure a knowledge based contest. Job applicants or potential job applicants can participate in the knowledge based contest. Other types of recruitment enhancement activities are possible and the example of a knowledge based contest is provided for the purposes of illustration only and is not meant to be limiting. For example, as part of a recruitment activity used to gather information, a participant can be requested to engage in a game, solve a puzzle or perform a specified task, such as program a solution to a problem.
[0287] Information gathered directly from the knowledge-based contest can be used to derive job suitability parameters, such as score or ranking. Further, when participants agree to participate in the recruitment activity, the participant may grant to access to their personal information, such as information stored at a social media account (e.g., Facebook™, Linkedln™ or Google+™). Information gathered from the social media account can trigger searches for information from other sources. The job suitability parameters and other information gathered directly or indirectly about participants in the knowledge-based contest can be made available to job recruiters via an interface generated from the recruitment enhancement system.
[0288] Figures 17-19 are examples of pages, 200, 250 and 270 that can be provided as part of an administrative interface for configuring a recruitment activity including a knowledge-based contest. An example of an implementation of a configured recruitment on a portable electronic device is described with respect to Figures 20-27. In Figure 17, interface page 200 includes a selectable menu button 202. The selectable menu button may allow an admin to access other pages of the interface, such as 250 and 270. The view contest components 240 and edit/back buttons 242a can also be selected to access different pages of the admin interface.
[0289] Via the interface, an admin can specify a contest name 204. For instance, the admin can enter the name 204, "Are you the best at a particular event?" The event might be a conference or exposition where the contest is to be implemented. The contest can be sponsored by a particular entity, such as an entity hosting the conference or a company participating in the contest. In one embodiment, the interface can be configured to receive a specification of the sponsor, such as a name 208.
[0290] If a sponsor is specified, the interface can be configured to allow an image associated with the sponsor to be uploaded. For example, an image of a company logo, such as 210, can be uploaded. The uploaded image can be used as part of a description of the contest and/or advertising indicating the existence of the contest.
[0291] The interface can be configured to receive a brief description of the contest. The description may be limited to a maximum number of characters. For example, in 212, the brief description is "Prove you have the most knowledge at iOS." The brief description 212 can be used in advertising for the contest.
[0292] The terms 214 can define a criterion for winning the contest. The interface can be configured to receive a selection of a contest criterion. For example, in 214, the criterion is the first four people to score 90% or better win. As another example, the four highest scoring individuals can win. In one embodiment, when the criterion is met, the contest can end and additional participants may no longer be allowed to participate in the contest.
[0293] In another embodiment, even after the winners have been determined, individuals may still be allowed to participate in the contest. The contest may be scored and a leader board can be posted. Thus, participants may wish to see how they scored relative to other participants even if they are no longer eligible for a prize.
[0294] The interface can be configured to receive a specification of a prize 214.
Prizes can be items, such as a cash, goods or services, such as goods or services associated with the sponsor of the contest. In addition, prizes can be an experience, such as a chance to meet a particular person at a company or a chance to visit and receive a tour of a company. In 220, the prize is a chance to win a lunch with John
Smith who is the VP of engineering at the company.
[0295] In particular embodiments, the prize can be a recruiting related prize. For instance, as described above, an opportunity to visit or meet someone at a company that is recruiting. In another example, the prize can be a chance to meet with a recruiter at the company or even receive an interview at the company.
[0296] The interface can be configured to receive a number of prizes that are to be awarded. In Figure 17, the number of prizes 222 is four which corresponds to the number of people that can have lunch with the special person at the company. The interface can be configured to allow an admin to upload prize associated content. For instance, the prize associated content can be an image, a video and/or audio data associated with the prize. For example, the prize associated content 224 is an image selected to indicate the prize is associated with dining. As another example, the prize associated content could be a video and/or audio snippet of describing information about the person (e.g., John Smith) at the company who is going to attend the lunch.
[0297] In particular embodiments, contests can be time limited. For example, a contest may be made available during a particular event that has a beginning and an end, such as 3 days. Thus, the interface can be configured to receive an input of a start time 226 and an end time 228 for the contest.
[0298] In one embodiment, the contest can be location limited. For instance, the contest may only be available to individuals within a radius of a certain location, such as around a particular convention center. A server remotely hosting the contest or an application executing on a user's device generating the contest may attempt to verify an individual's location before allowing the individual to participate in the contest. For example, a GPS sensor on a user's smart phone can be used to determine a location that is verified by a contest application executing on their device. The location information can also be sent to a remote server that is hosting the contest which can verify location eligibility. In one embodiment, the location 230 can be entered GPS coordinates, such as a longitude and latitude.
[0299] In one embodiment, the interface can be configured to accept social media information. The social media information can be used to allow a participant to post some information associated with their participation in the contest to a social media site, such as but not limited to Facebook™, Google+™, Twitter™ or Linkedln™. In 200, the interface is configured to receive a URL 232 for a twitter post. Further, a format of a twitter message 234 can be specified. In 200, the twitter message includes information about the participants score, a company's name and a job. Further details related to tweeting a score are described below with respect to Figs. 25 and 26.
[0300] Lastly, the interface can be configured to receive a specification of a winner value 90 and its format (Winner Rule 238). In this example, 90% of the questions need to be answered correctly. Another way of specifying the winner could be specifying a number of questions that need to be answered correctly.
[0301] In Figure 18, an interface page 250 for specifying contest components is shown. In this example, a name 252 associated with the contest components is provided. A selectable link 254 allows the user to return to a different page, such as the page illustrated in Figure 17. In one embodiment, a database of content that can be used in a contest, such as a database including different question can be provided with the system. On page 250, a search feature 256 is provided. The search feature may allow a user to input one or more searchable terms. The searchable terms can be used to locate questions that can be used in the contest. [0302] When a question is found via searching or after a user manually inputs a question, the interface can assign an ID 258 to the question. The questions that have been generated are listed according to their ID number. For each question ID number, a question body is specified, such as 262a, 262b and 262c. A number of responses 260 can be tabulated with each question. The number of responses is the number of individuals that have answered the question. This number is updated as the number of participants in the contest increase. Selectable links are provided that allow each question associated with a question ID to be edited or deleted. An example of editing a question is described with respect as follows with respect to Figure 19.
[0303] Figure 19 shows an interface page where a question with a particular question id 275 can be edited. In this example, the question id number is one. When a question is edited the body of the question 274 is shown. In this example, the body of the question is associated with a logic question, "Jack is a hacker. Some hackers are smart. Therefore Jack must be very smart." Via the interface, in some instances, the user may be able to edit the body of the question. In other embodiments, the body of the question can be fixed. Thus, the user can delete the question but may not be able to edit the body of the question 240.
[0304] In some embodiments, via the interface, a user can specify a plurality of answers to a question. For example, for a true false question, the user can specify answer one 276 as "true" and answer two 278 as "false." For multiple choice questions, the user may be able to specify a number of different multiple choice answers, such as two, three, four, five, etc. or more multiple choice answers. Where the number of choices can be varied from question to question. The interface may allow a user to specify which answer is the correct answer. In this example, answer two 278 is the correct answer and is shown as highlighted in the interface page 270.
[0305] In various embodiments, the interface may allow a user to specify more questions than are presented to a user. For instance, the user can specify fifty questions for a contest where only fifty questions are presented to a participant in the contest. In this embodiment, the system can be configured to select ten questions randomly from the pool of questions each time a contest is instantiated for a particular participant. Thus, the questions administered to each participant can vary from participant to participant. The interface can be configured to allow a number of questions administered and the total number of questions from which the administered questions are selected to be specified.
[0306] In other embodiments, each participant can answer the same set of question the questions can be presented in a randomized order such that they vary from person to person. In addition, the order in which the answers appear to the questions can be randomized such that the order of the answers that appear for a particular question vary from participant to participant. In yet another embodiment, a degree of difficulty can be assigned to each question. The degree of difficulty can be used to score the contest such that more difficult questions are given a higher weight in the score.
Implementation of a Knowledge-Based Contest for Recruiting on a Portable Electronic Device
[0307] In this section, an implementation of knowledge-based contest for job recruiting on a portable electronic device, such as smart phone. The knowledge-based contest can be implemented as part of a recruitment enhancement system described above. Other types of electronic devices, such as desktop or tablet computers, can be used to participate in a contest via an associated interface and the example of smart phone is provided for the purposes and is not meant to be limiting. Figures 20-27 depict a number of different interface states involving a user participating in a recruitment enhancement activity, such as a knowledge-based contest, implemented on a portable electronic device are described.
[0308] In Figure 20, an image of a front face of a portable electronic device
(iPhone™, Apple, Cupertino, CA) is shown. A first state of an interface is shown on a display device. In one embodiment, an application may have been downloaded to the device to allow access to the recruitment enhancement activity. In another embodiment, a web-browser can be instantiated on the device and a participant can navigate to a particular web-site via the browser. For example, a participant can scan a QR code or other optically formatted image data that can result in navigation to a particular web-site or a user can manually type in a web-address to arrive at the web- site.
[0309] The initial page of an interface that appears in an application or a web-site can indicate a sponsor of the recruitment enhancement activity. In 300, a logo 302 for Gild™ (San Francisco, CA) is displayed on the page. In one embodiment, a participant can be required to login via a social media site 304, such as Facebook™ or Linkedln™. A login via one of these sites can allow access to information in the person's social media account, such as their profile on the account. In alternate embodiment, an interface page can be generated that allows a user to sign up for an account and enter basic information, such as a profile. For example, the user can signup for an account with Gild™ that allows the user to access various recruitment enhancement activities, such as knowledge-based contests.
[0310] Figure 21 shows an image 310 of an interface page after a user has selected a social media site. In this example, the user has chosen to login via Linkedln™. In response, the user can be requested to provide information 312 needed to login via the social media site, such as an account identifier (e.g., email address) and a password. A widget can be generated for accepting this information. The user can provide an indication 314 that they grant permission to allow access to their social media account and then select a button to trigger the login. When future logins occur via the same social media site, the user can again login via the social media site but may not be asked for permission to access their data. A user may choose to login at a first time using a first social media site and a second time via a second social media site. The user can be requested to grant access to their information from each site, respectively.
[0311] In response to receiving the login information and permission to access account information, such as a profile information, information can be downloaded from the social media site. As described above, the information can be used to indirectly gather information about the individual. The information gathered indirectly can be used to determine one or more job suitability parameters that can be used by a job recruiter to select a candidate for a job position. After a user has granted access to their profile information, the recruitment enhancement system can periodically check for changes in their information. The changes can be used to update job suitability parameters. The checks can occur independently of the user's participation in recruitment enhancement activities, such as knowledge-based contests, used to directly gather information and possibly rank the user.
[0312] In one embodiment, as described above, this information can be used to affect a recruitment enhancement activity. For example, if it is learned from a social media account that a user claims to have a particular job related skill, such as a skill appearing on their Linkedln™ profile, a knowledge-based test can be selected for the user related to the skill and the user can be given the option of taking the knowledge- based test. As an example, if a user claims to have experience in a programming language then a knowledge -based test related to the programming language can be selected for the person. As another example, if a user claims to have knowledge of accounting, a knowledge-based test related to account can be selected for the person. One or more tests can be selected for a person in this manner.
[0313] In additional embodiments, when it can be determined how much experience the user claims to have in a job related skill, a recruitment enhancement activity, such as a knowledge-based test, can be adjusted to be consistent the claimed level of a job related skill. For example, a different test, such as a test with harder test questions, can be implemented to a person claiming to have five years of experience versus a person claiming to have one year of experience. In addition or alternatively, a test can be scored differently depending on a claimed skill level. For instance, an individual with more experience in a skill can be expected to finish a test faster than an individual with less experience.
[0314] As previously described, participants in recruitment enhancement activities, such as knowledge based test can be ranked relative to one another. In one embodiment, participants can be ranked against other participants with a similar level of experience. For example, participants with five to ten years of experience can be grouped together and participants with one to five years of experience can be grouped together for ranking purposes. In another example, a scale on which the participant's performance can be placed can be developed to reflect a specific experience level.
[0315] In yet other embodiments, a user can request to participate in a particular recruitment enhancement activity, such as a knowledge based contest. For example, at a job fair or conference, a number of recruitment enhancement activities in which a person may participate, such as knowledge based contests, may be made available. After a user logs into the system, a list of available knowledge-based contests can be presented to the user. The number of contests that are available may relate to skills from many different professions. In one embodiment, the system can be configured to filter recruitment enhancement activities, such as knowledge-based contests, so that only ones of interest to the user are presented. For example, the interface can be configured to receive user preferences in regards to knowledge-based contests or the system can be configured to automatically filter the recruitment enhancement activities based upon information learned from their social media profile.
[0316] In Figure 22, a state 320 of an electronic device showing a list 320 of available contests is shown. The list includes knowledge-based contests for jQuery, Ruby on Rails, iOS, system administration or start-up knowledge. In 324, the iOS contest has been selected. In response, the system can initiate the contest. The interface can include a selectable button 326 that allows all contests to be displayed, such as all contests available at a job fair, and a selectable button 328 that causes contests that have been filtered for a user according to their preferences to be displayed and/or causes contests in which a user has participated to appear. For example, the user may select button 328 to view contest in which they have participated to learn their current status on a contest leader board.
[0317] In one embodiment, the interface can be configured to indicate sponsors of each knowledge-based contest and possibly job positions associated with each contest. For example, via the interface, the user may be able to learn about companies that have job positions requiring iOS skills. As another example, the user may be able learn about companies that have job positions requiring system administration skills.
[0318] In a particular embodiment, a user can gain access to a particular contest only in response to a particular interaction. For example, the user can scan a QR code on a poster at a booth at a job fair to gain access to the contest where the QR code may only be available at the job fair. In another example, an interface, such as a near-field interface, can be provided. The user can pass a place a device, such as a smart phone, in proximity to the near field interface in response the user can gain access to one or more different contests. In yet another example, a local area network may be made available, such as a local Wi-Fi hot spot. Access to one or more different contests may only be granted when a user's device is able to log into the local hot spot.
[0319] As will be described in more detail with respect to Figure 27, contest access can be location based. Thus, a person may only be able to participate in a contest when they are within a particular geographic area, such as near a conference site. The user's geographic location can be verified based upon GPS information received from their electronic device or based upon the fact that the user's device has established a connection with a local area network at the geographic location. [0320] In Figure 23, a state 330 of the portable electronic device is shown after a user has selected a particular contest in which to participant. A description 321 of the contest is provided. In this example, the contest is related to Apple's iOS™. Rules 333 for the contest are specified. In this example, the rules are to answer as many questions as possible in three minutes. As described above, a prize or prizes for the contest (not shown) can also be listed.
[0321] A question database can be generated to account for the fact that different participants can be answer a different amount of questions. In one embodiment, a maximum number of questions that are available can be specified. Thus, if the participant answers all of the questions prior to the three minutes ending, the contest is ended and a score is generated. The score can be adjusted to account for the amount of time it took for the participant to finish the contest. In another embodiment, the question database can be sufficiently large such that a participant can be allowed to answer questions until the contest is completed.
[0322] A current leader board 334 can be displayed prior to a user starting a contest. The leader board 334 can include a list of people according to their rank 335. The rank 335 can be determined by their score 338. In one embodiment, the ranks and scores can be presented anonymously. In other embodiments, the ranks and scores can be presented with user identification information, such as information obtained from a profile associated with a social media account.
[0323] In 330, an image of each participant 336 and their name 337 shown for the purposes of illustration. In one embodiment, a participant can be given an option of posting their score anonymously or not posting it at all. Further, the user can be given the option of removing a posting of their score after it has been posted. However, even when a score is not posted or removed from posting, the system can keep a record of the participant's score.
[0324] When a participant is ready to start the contest, they can select the start contest button 332. In response, the system and/or the application can start generating the activities associated with the contest. For example, if the contest involves playing a game, the game can begin. If the contest involves solving a puzzle, the puzzle can be presented. In this example, the contest includes answering a number of questions and the first question can be presented. As part of a contest, a combination of games, puzzles and questions can be presented. [0325] In Figure 24, a state 340 of the electronic device is shown where a question is output. The time remaining 348 in the contest is presented. A question 344 is displayed and four selectable answers are displayed. In this example, a user has selected one of the answers 346 as their choice which is highlighted. The application or system may or may not indicate whether the answer was correct before moving on to the next question. In this example, the system is configured to indicate whether a response is correct or not. The participant has selected the correct answer and thus, an indication 342 that the answer is correct has been provided.
[0326] After it is determined that conditions associated with the contest ending have been met, the contest can end. For example, a time limit may have been reached, all of the questions in the contest may have been or a limit on the number of wrong questions may have been exceeded. In response to the contest ending, the participant's results can be scored 352 and the person can be provided a rank 354 in the contest. As described above, the participant's results can be posted to a leader board. Thus, the leader board can be updated to reflect the participant's results. In this example, the participant's score resulted in their being tied for first place.
[0327] In one embodiment, the participant may be allowed only to take the contest one time. After taking the contest, a record of their participation may be maintained. Thus, the person may be to check on the contests in which they have participated and view the current leader board. When the overall contest has ended, winners of the contests can be notified and/or recognized. For example, winners of a prize can be notified.
[0328] In one embodiment, the system can be configured to notify a job recruiter based upon the results a contest. For example, if the participant is at a job fair and scores above a certain threshold, a recruiter can be notified. The notification can include contact information for the participant in the contest. The recruiter can then attempt contact the participant, such as for an interview at the job fair. In another example, after the contest has ended, one or more recruiters can receive a list of participants that are ranked above a certain threshold. For example, the recruiters can receive contact information for the participants ranked in the top ten of the contest. After receiving this information, the recruiters can attempt to contact the participants that were ranked in the top ten. [0329] In yet another embodiment, the application can be configured to allow the user to post their score or information about their participation to a social media site, such as Twitter™, Facebook,™ Google+™ or Linkedln™. For example, in 350, a button 356 is provided that allows a user to post tweet their score to twitter. When the button 356 is selected, a tweet can be instantiated.
[0330] Figure 26 is a state 360 of a portable device on which the participant has participated in the contest is shown. The state 360 shows an indication 362 of the social media site that is being utilized. In one embodiment, a message including its content and format can be automatically generated. In this example, the message 364, which reflects the participants score, contest description and the sponsor is "I scored 29.2% on the iOS on the @gild arena mobile application." As described with respect to Figure 17, a format of an automated message and its contents can be specified by the administrator for the contest via the administration interface.
[0331] If a participant subsequently wins a prize or ranks highly in a contest, an opportunity can be provided for the user to automatically post a message related to a win of the prize or their rank after the contest has ended. For example, the application may be configured to automatically generate a message, such as "I finished first in the iOS contest and won a free iPad at the developer's conference" or "I finished in the top 1% in the iOS contest at the developer's conference." A selectable button can be provided that allows the participant to automatically post the message to a social media site.
[0332] As previously described contests can be geographically limited. In one embodiment, the recruiting application can be configured to generate a map including locations where particular recruiting contests are available. Information on the map can include the location of contests and the contests that are open for the user's participation based on their current location. For example, in Figure 27, a device state 370 including contest locations is shown. In state 370, a map 372 is shown. A circle 374 is shown around the user's current location is generated on the map. In one embodiment, all contests within the circle 374 may be available to a user. As described above, an account can be maintained for a user that keeps track of all the available contests. In the example, one contest 376 is open to the user. When the button 378 is selected, the user can learn more details about the contest. [0333] In 370, a map is shown of a city. In other embodiments, the map can be of an indoor location. For example, a map of a convention center can be provided. A user might be required to travel to various locations within the convention center floor, such as different booths to access different contests. The map might show the locations of the different contests and/or the user's location. Selectable indicators can be provided on the interface which allow a user to learn additional information about each contest.
[0334] It will also be understood that the scoring techniques described are exemplary. As software evaluation tools increase in their capabilities it will also be understood that other metrics of coding quality and/or complexity could be utilized.
IV. RECRUITMENT ENHANCEMENT SYSTEM INCLUDING
APTITUDE ASSESSMENT OF PROFESSIONAL SKILLS
[0335] As will be described in more detail as follows, a recruitment enhancement system for electronic job recruiting system is described. The recruitment enhancement system is configured to gather and analyze information about job applicants. One objective of the analysis is to identify job applicants most likely to successfully perform jobs associated with available job positions. For example, a job applicant's proficiency or knowledge pertaining to a particular aspect of a job can be assessed and scored, such as their knowledge or proficiency of a particular programming language for a programming job. The scoring can be used to rank applicants.
[0336] The information derived from the analyses, such as the scores, can be referred to as job suitability parameters. The job suitability parameters can be designed to aid job recruiters during the manual filtering process of job applicants. The recruitment enhancement system can be configured to generate an interface that allows job recruiters to access the job suitability parameters for various candidates.
[0337] The recruitment enhancement system can be configured to gather information used to derive job suitability parameters directly or indirectly. Direct information gathering can involve configuring an activity and then getting a job applicant or a potential job applicant to engage in the activity. When the job applicant engages in the activity, information from the user's participation can be gathered and analyzed. For example, the job applicant can be asked to engage in a knowledge based contest related to a skill associated with a job. The job applicant's answers can be scored or evaluated in some manner to provide some indication of the applicant's skill level.
[0338] Indirect information gathering can involve harvesting information from a job applicant's online activities. One source of information that can be gathered an analyzed may be from an applicant's participation in social media activities. Another source of information that can be gathered and analyzed may be from an applicant's participation in professional related activities. Many potential information sources are available and these examples are provided for the purposes of illustration only and are not meant to be limiting.
[0339] As an example of an information source associated with professional related activities that can be used to generate a job suitability parameter, computer programmers often post publically available code related to personal projects. Their projects can be followed by other programmers. A recruitment enhancement system can be configured to gather and analyze the code to assess the programmer's abilities and suitability for various programming jobs. From the analyses, job suitability parameters can be derived that are made available to job recruiters. In one embodiment, a proprietary database can be maintained at the recruitment enhancement system that includes data about individuals whose skills have been analyzed in this manner.
[0340] The job recruiters can use the job suitability parameters, such as job suitability parameters derived for programmers stored in a proprietary database, in the next stages of the recruitment process. For example, the job recruiters can use the job suitability parameters to assemble a list of potential candidates they wish to vet in more detail and then possibly contact. The job suitability parameters can be used to rank each of the potential candidates. Thus, the job recruiters may use the job suitability parameters to establish an order in which to contact potential candidates.
[0341] In particular embodiments, the recruitment enhancement system can be configured to receive identifying information for a group of individuals from a job recruiter. As an example, a plug- in module can be provided for the recruiter's applicant tracking system (ATS) that generates and sends the identifying information related to the individuals to the recruitment enhancement system. In response to receiving the identifying information from the ATS, the recruitment enhancement system can be configured to determine whether one or more of the identified individuals from the ATS can be matched to individuals stored in one or more of the proprietary databases maintained at the recruitment enhancement system. For each of the individuals that are matched, a report can be generated. The report can include job suitability parameters for the matched individuals including how the individuals rank relative to the other individuals stored in the proprietary database. The report can be returned to the recruiter.
[0342] With respect to the following figures, details of apparatus and methodology for generating and maintaining a proprietary database of skilled individuals in a recruitment enhancement system where the skilled individuals can be scored or ranked are described. In addition, recruiting services utilizing these proprietary databases is described. The skilled individuals can be members of a variety of different professions. In one embodiment, the skilled individuals can be software developers. However, the discussion in regards to the software developers is provided for the purpose of illustration only and is not meant to be limiting.
[0343] In particular with respect to Figure 28, a recruitment ecosystem is described. Within the recruitment ecosystem, interactions involving individuals seeking jobs, recruiters seeking candidates, companies providing jobs and a recruitment enhancement system are discussed. With respect to Figure 29, a recruitment enhancement system is described. In particular, methods and apparatus for formulating a proprietary database of information relating to skilled individuals are discussed.
[0344] With respect to Figure 30, 31 A, 32B, 33, methods and apparatus for recruiting services that leverage the proprietary databases maintained within the recruitment enhancement system are discussed. The recruiting services can involve receiving a list of individuals from a recruiter and attempting to match individuals from the list to individuals stored in the proprietary databases. Reports can be generated for matched individuals on the recruiters list and returned to the recruiter. The recruiter can use information in the report to further their recruiting activities. Recruiting Ecosystem Including Recruitment Enhancement System
[0345] In this section a recruiting ecosystem including a recruitment enhancement system is a described. In a recruiting ecosystem, individuals can engage with various electronic systems to supply information that allows them to apply for existing jobs or to be considered for future jobs when positions open up. In response to the supplied information, recruiters can select from among the individuals and attempt to initiate additional job recruiting related interactions. For instance, based upon the received data, a job recruiter can identify an individual and attempt to contact the person to set up an interview.
[0346] A job candidate can be a person that is interested in working for company and has agreed to provide information but may not have applied for a particular job. A job applicant can be a person that has applied for a job and agreed to provide information as part of the application process. As described herein, a recruitment enhancement system 12 can be provided that can be used to gather information and engage job candidates and job applicants alike.
[0347] In the recruitment enhancement system, methods and apparatus can be provided for gathering secondary recruitment data that is beyond the resume and other application data that is usually gathered in the job application process via applicant tracking systems. The secondary recruitment data can be analyzed and scored. Based upon the score, one or more rankings for a job candidate can be generated. In one embodiment, the secondary recruitment data and rankings can be stored in a proprietary database maintained at the recruitment enhancement system.
[0348] As an example of ranking or scoring, an individual in the recruitment enhancement system 12 can be ranked independent of or relative to a group of other job candidates or job applicants, such as a group of job applicants applying for the same job or a group of job candidates within the same profession. An independent rank can involve comparing the individual's performance to some derived scale. A relative rank can involve comparing the individual's performance to other individuals within a group.
[0349] In one scenario, as will be described in more detail with respect to Figures 30, 31 A, 3 IB and 32, a company can send information related to a group of individuals that have recently applied for job, have applied for a job in the past or have provided information indicating their interest in working for the company. The information associated with these individuals can be stored in a company ATS (applicant tracking system). Upon receiving the information, the recruitment enhancement system can be configured to rank the group of individuals one or more different ways. For instance, individuals recently applying for jobs can be compared against individuals that have been hired by the company. In another example, the group of individuals can be compared against a group of individuals stored in a proprietary database associated with the recruitment enhancement system.
[0350] The recruitment enhancement system 12 can be configured to generate an interface that allows an outside entity such as a job recruiter to access the ranking data. The outside entity can use the ranking data as a basis for additional investigations, such as learning more about particular individuals, and determining whether to contact particular individuals for additional interactions, such as a job interview. In one embodiment, as described in more detail with respect to Figure 30, the interface can be provided as a part of a plug-in application for their ATS.
[0351] In particular embodiments, the recruitment enhancement system 12 can be tailored to particular professions. For example, the recruitment enhancement system 12 can be tailored towards the computer programming profession where the system is configured to gather secondary recruitment data that allows computer programming professionals to be ranked according to some scale and/or relative to other computer programming professionals that have participated in a common recruitment enhancement activity. A recruitment enhancement system associated with program developers is described below in more detail with respect to Figure 29. In another example, the recruitment enhancement system 12 can be tailored towards medical professionals, such as doctors or nurses, where the system is configured to gather secondary recruitment data that allows the doctors or nurses to be ranked.
[0352] Figure 28 is a block diagram of a recruitment ecosystem 50 including a recruitment enhancement system 12 in accordance with the described embodiments. The ecosystem 50 includes company systems, such as 6, recruiter systems, such as 8, a recruitment enhancement system, such as 12 and 3rd party systems where individuals can engage in various activities, such as 10. Social media sites are one example of 3rd party systems. An individual 2, who may or may not be currently engaged in the job hunting process may interact at various times with each of the systems via a computational device, such as the tablet device 4. Typically, the interactions may involve establishing communications between the computational device and one or more remote servers associated with each of the systems over a local and/or wide area network. For instance, the tablet device 4 can be used to establish communications and interact with a server associated with a social media site.
[0353] Communication devices typically include a processor, a memory, networking capabilities and a user interface that allows for the input and output of data. Examples of communication devices that can be utilized include but are not limited to smartphones, laptop computers, netbook computers, desktop computers and tablet computers. Interface devices that can be utilized include a display, a keyboard, and a microphone in combination with speech recognition, a mouse, a touchpad and a touchscreen.
[0354] The recruitment enhancement system 12 can include components for a) direct gathering of recruitment data 24, b) generating a recruiter interface 26, c) indirect gathering of recruitment data 28, d) scoring and ranking 30, e) data matching 32 and f) data quality assessing. The system 12 can be hosted on one or more servers including processors, memory and network interfaces. Direct recruitment data gathering 24 can involve generating one or more activities in which an individual can directly participate, such as a knowledge-based test, a game, a puzzle or solving a problem. The direct recruitment data gathering can be used to identify and engage individuals that have interest in a company or encourage individuals to be interested in a company. In addition, the direct recruitment data gathering can be used to possibly assess potential job candidates.
[0355] In general, the capturing of data from an individual directly engaging in an activity can be part of direct recruitment data gathering. For instance, an individual engaging in an activity, such as an interview, can be captured on video data as part of direct recruitment data gathering 24. In another example, when an individual takes a test, information such as their answers, how long they take to answer each question and their total time can be gathered and analyzed.
[0356] The indirect recruitment data gathering can involve gathering information from an individual's on-line activities, such as participation in social media sites 36, professional sites 38 or even sites related to a user's hobbies. The person's on-line activities can be hosted on 3rd party systems 10, such as a social media site. In one embodiment, when a user agrees to participate in an activity involving direct recruitment data gathering, a user may agree to provide access to their on-line activity information which can be captured by component 28. For example, a user can provide access to their Facebook™ or Linkedln™ profile for a chance to participate in a knowledge-based test related to recruiting. The knowledge-based test can be referred to a recruitment enhancement activity. After access is granted, the indirect recruitment data gathering component 28 can retrieve information about their on-line activities from the 3rd party systems.
[0357] The information gathered by components 24 and 28 can be stored and analyzed. Then, job suitability parameters can be derived. For example, the scoring and ranking components 30 can be used to score information directly gathered during an online test and then rank individuals based upon the scores. In addition, information about an individual's online activities can be gathered indirectly from various third-party systems 10 and job suitability parameters can be derived from this information. For example, based upon software code that a user has posted to a professional site, the individual's coding ability can be scored and ranked. The software code may be executable. However, the software code doesn't have to be executable to generate a score or a ranking. Methods related to scoring code are described below with respect to Figure 29.
[0358] In particular embodiments, scores and ranks can be derived from data that is gathered directly and/or indirectly. For example, as will be described below, a score and/or a rank can be generated based upon an individual's participation in a knowledge-based test. In another example, a score and a rank can be generated based upon an individual's participation in a knowledge based test and information retrieved from a third-party site, such as social media site. Again, a person can be ranked relative to a group of other individuals, such as a group of individuals participating in a common task, or according to some derived scale or measure.
[0359] The data matching component 32 can be used to match various data sets to a particular individual. For example, an individual may use multiple names in their online activities. As another example, different individuals can share a common name. The data matching component 32 may use information verified from one source to validate another information source. For example, if an individual grants access to their Facebook™ profile and then another source of information is available, such as their profile in a professional organization or data that they have supplied in applying for a job, the information from the two sources can be compared by component 32 to see if they are the same person. When the comparison indicates they are the same person, a portion of the information from the multiple sources can be stored at the system 12 or at least the links to the multiple sources can be stored. This information can be made available in a recruiter interface.
[0360] The data quality assessing component 34 can be used to assess a validity of gathered data. For example, the component 34 might be used to determine whether someone has cheated on a knowledge-based test administered by system 12 or tried to game the system 12 in some other manner. As another example, the component 34 might be used to determine whether a person has falsified data they have provided as part of a job application. Information that is identified as invalid may be flagged. In addition, the system 12 can be configured not to use information identified as possibly invalid for scoring and ranking purposes.
[0361] The recruiter interface 26 can allow recruiters to directly access some portion of the information available at system 12. For example, via the interface, recruiters may be able to access scores and ranks about various individuals. Further, via the interface a recruiter may be able to search for individuals with particular skills and assess their job suitability parameters, such as scores and ranking. In another embodiment, as described in more detail with respect to Figure 30, a recruiter can access the system 12 via a plug-in module coupled to their ATS, such as 16 or 20.
[0362] The company 6 and the recruiter 8 can each maintain applicant tracking systems, such as 16 and 20. If a recruiter 8 works for the company 6, then the recruiter 8 may not maintain a separate applicant tracking system. The recruiter 8 may have access to an interface or interfaces that allows them to interact with the recruitment enhancement system 12, an individual 2 via their device 4 or some other mechanism, their own application tracking system 20 and a company application tracking system 16.
[0363] The company 6 can sponsor a job-site interface 16. Via the job site interface 14, an individual, such as 2, may be able to learn about different jobs and optional apply for jobs. Information from the job application process can be stored to the company application tracking system 16. In particular embodiments, the company 6 may allow recruiters, such as 8, and/or the recruitment enhancement system to access the application tracking system 16.
[0364] In one embodiment, one or more direct recruitment data gathering activities can be triggered from the job-site interface 14. For example, when a user applies for a particular job, such as after submitting their information or a part of submitting their information, a link can be established from the job-site interface 14 to the recruitment enhancement system 12. The recruitment enhance system 12 can then generate a recruitment enhancement activity that is used to directly gathered recruitment data. For example, as part of the recruitment enhancement activity an individual may be asked to take a knowledge-based test, play a game, solve a puzzle, solve a problem or combinations thereof.
[0365] Information captured from the recruitment enhancement activity can be scored or ranked and made available to recruiters via the interface generated by the recruiter interface generation component 26. An individual can participate in a number of different recruitment activities. Thus, the interface may allow a recruiter to view descriptions of the recruitment enhancement activities in which an individual has participated and jobs suitability parameters derived from their performance.
[0366] In one embodiment, via the interface, a recruiter may be able to send a request to an individual to participate in a particular recruitment enhancement activity. For instance, the recruiter may be interested in an individual for a particular job position. Via the interface, the recruiter can send a message to the individual indicating their interest in the person and requesting them to participate in a recruitment enhancement activity, such as taking a knowledge-based test associated with the job position. The message received by the individual can include information, such as a link to a website, which allows the individual to engage in the recruitment enhancement activity.
[0367] In one embodiment, the recruitment activity that is selected can be related to information associated with an available job. For example, a knowledge-based test can be implemented that is related to a skill needed for the job position. Their answers can be scored and the person can be ranked relative to other job applicants that have taken the test.
[0368] In other embodiment, the recruitment enhancement activity that is selected can be related to information supplied by the individual applying for the job. Via interface 14, an individual can submit information indicating that they possess a particular skill at a particular skill level, such as a number of years of experience in the skill. For instance, the interface can ask the person to indicate how many years of experience the person has with the skill or the person can submit a resume including this information. [0369] Based upon information supplied by the individual, the system 12 can be configured to select a recruitment enhancement activity that in some way measures or is predictive of the person having the claimed skill. For example, an individual can be asked to take a knowledge-based test associated with their claimed skill. Harder or easier test can be implemented depending upon the person's claimed skill level. The test answers can be received by the system 12, scored and analyzed. The scores and analyses can be used to assess individuals in regards to their proficiency in the claimed skill. In addition, the individual can be ranked relative to other individuals with the same claimed skill level or according to a scale constructed for individuals of the claimed skill level. The system 12 can be configured to make information about an individual's score or rank in the recruitment enhancement activity available to recruiters.
Recruitment Enhancement System
[0370] Figure 29 illustrates in more detail a functional block diagram of a recruitment enhancement system 12 in accordance with an embodiment of the present invention. The recruitment enhancement system may reside on one or more servers with associated processors and memory, wherein the computer code is stored on a computer readable memory. A database memory may be provided to store information for the recruiting service, including candidate profile information.
[0371] In the example below, the generation of a database, which may be a proprietary database, is described. The database is focused on individuals that are software developers. The use of software developers is provided for the purposes of illustration only and databases associated with individuals in other skilled professions can also be generated and maintained at the recruitment enhancement system.
[0372] A crawler 105 is provided to crawl code repositories. For example, the crawler may use an API for code hosting sites such as GitHub. A new candidate profile generation module 1 10 determines whether the crawler has identified a new developer. If so, a profile ID is generated to build a new profile. A code file type analysis module 1 15 determined the file type of files being crawled. After the file type has been determined, the language-specific code analysis module is selected by module 120. Scoring and cheating detection is then performed by module 125. [0373] Profiles are stored in a profile information database 130. A social media access module 135 provides access to social media information sites and a social media aggregation module 140 correlates aggregated social media information for individual profiles. A messaging interface 145 is included in one embodiment as a mechanism for recruiters to contact individual developers. However it will be understood the messaging interface 145 may be omitted in some implementations. The messaging may, for example, be brokered in the sense of cloaking the user information and email address of the recruiter during initial attempts to contact a developer. A recruiting search engine and graphical user interface (GUI) module 150 is responsible for generating the graphical user interface that is provided for display on a user's computer. In one embodiment, the recruiter's search engine and GUI module can be configured to interact with a plug-in module that is provided to the recruiter's ATS (e.g., see Figure 30).
[0374] The new candidate profile generation module 1 10 utilizes author information from crawled sites to detect that there is a new developer to be added to the system. Code repository sites include author information for each project. This author information is searched by the crawler. Each individual person with a profile has a unique ID. The unique ID is created the first time an individual programmer's name is discovered in crawling author information in code hosting sites. For example, when the crawler finds the names of people that have contributed code to a code hosting site, the system compares the unique ID from the network that the person is found on to the unique IDs in the database of the recruiting service for that network. If an ID doesn't exist, a new user ID is created.
[0375] The crawling process generated project information for each developer. One way to obtain project information for a particular person is specifically ask a code hosting site (or content site like Stack Overflow) for a list of projects for each developer. For example, this may be done through an API for sites such as GitHub and Stack Overflow.
[0376] The crawling is updated regularly and the profiles are refreshed according to a cycle. An exemplary refresh cycle is a two-week profile refresh cycle. That is to say, the update from the crawlers may be constant, but the profiles may be updated according to a schedule, such as every two weeks. [0377] One aspect of the crawling process is that the source code for a particular project is downloaded for analysis. As illustrative examples, the source may be downloaded using technologies such as Git, SVN, Mercury, and CVS, which are technologies that allow for synchronization with the local computer of a code repository.
[0378] It is preferable to download all of the available information in a repository for analysis. However, note that the source code for a project may be in any one of a variety of different file types.
[0379] Downloaded files are then processed, starting first with the code file type analysis module 1 15. An individual file is analyzed to determine what's in it by looking at file extensions and the binary data or text that the file contains. Specific patterns in the source code are analyzed. For example, specific languages— like Ruby— always start with a certain few lines of code - e.g. the hashbang - so that by looking for the patterns of a specific set of keywords in the file permits the language to be identified. Additionally, the analysis of the code can include looking for the "magic number"— a set of bytes at the beginning of the file that indicates file type. For example, images always start with a specific byte configuration. The pattern associated with each different file type is checked until a match is found. The pattern matching may be performed, for example, using a sequence of if-then clauses to identify the file type of a particular file.
[0380] After the file type is determined, the language specific code analysis selection module 120 makes a selection of an evaluation tool or tools appropriate for the language of the file type. Most software languages have evaluation tools to evaluate the quality and complexity of the coding. The evaluation tools are specific to a particular language and may, for example, look at the length of the code and patterns in the code. For many cases, the evaluation tools for a specific language are open source and/or available from commercial vendors. For example, there is a unique set of tools to evaluate Ruby— tools that differ from those used to evaluate C++. Thus if the recruiting service is designed to analyze code in languages such as Java, Scala, Shell, ActionScript, XML, CSS, HTML, Groovy, PHP, Perl, Python, Lisp, etc. then the system includes the corresponding evaluation tool for each supported language. Thus, the recruiting service includes a wide range of evaluation tools to support different languages and makes the selection of the proper evaluation tool based on the file type. The file is then analyzed using the appropriate selected tool(s) for the language associated with the file type.
[0381] The scoring and cheating detection module 125 utilizes the evaluation of the code and also information from the commit log for the file. To identify the author of the code of a particular file, the commit log is evaluated for the repository. The commit log is a list of who did what for the repository. This permits an evaluation of the developer's specific contribution(s) to that project. For example, by analyzing the commit log an evaluation can be made of the time(s) when the developer made a contribution.
[0382] The process is continued for all of the developer's repositories to permit a determination to be made of what languages that a person has experience in and how much experience the person has in each language. Recruiting service downloads all the developer's repositories and evaluates their contributions to determine the languages they've written code in. To evaluate experience the commit log is examined to look at the date and time of when the developer contributed to the project. Different factors can be used to determine actual experience. One factor is that contributions can be evaluated by their frequency and regularity to weight the actual number of years of experience in a particular language.
[0383] As an illustrative example, consider a developer named Joe. If Joe started contributing to a project 3 years ago, and the commit log shows that he's been contributing regularly to it, that's an indication that he has 3 years of actual experience. Thus, a weighting function can take into account the frequency and regularity of Joe's contributions. For example, if Joe has been making four or more contributions per year that's an indication that Joe has been regularly working on Java.
[0384] However, if Joe made a single contribution 3 years ago and just contributed again for the first time 2 months ago, then the commit log indicates "episodic" contributions with a wide spacing between contributions. For this second case, the weighting factor can be used to reduce Joe's number of years of experience such that he does not get 3 years of experience credit. The exact weighting function chosen can be empirically determined based on common behavior patterns of software developers. For example, if the commit log shows Joe made a single contribution to a Java project 3 years ago and made a second smaller contribution a month ago there could be a possibility that Joe is either 1) trying to "inflate" his resume about the number of years of experience he has in Java; or 2) may have become aware of the recruiting service and is intentionally trying to trick the recruiting service. In this example, the weighting function may also include one or more rules to discount recent contributions, particularly those of a minor character, such as a minor code tweak or a contribution made with many other contributors.
[0385] Thus, while the raw data provides an indication of a maximum potential number of years of experience, a weighting function may include different factors related to frequency of contribution, size of contribution, and number of co- contributors to perform a weighting function to arrive at a more accurate
interpretation of the number of years of experience for a developer. The weighting function may be determined empirically, based on observations about the way software developers normally work, to optimize different weighting factors and periodically adjusted to discourage gaming of the system. Other types of gaming (such as posting the same code at different times on different sites or plagiarizing code from others) could also, in theory, be checked as part of a larger fraud detection function.
[0386] As an illustrative example, patterns in a commit log may be examined for suspicious factors of how the developer is developing his/her source code. The simplest example of a developer cheating is that the developer downloads someone else's source code, opens their own repository, and submits that same exact code to the new repository. In that case, there would be a huge update all at once, and then nothing else in terms of activity. This is inconsistent with normal commit log behavior in which a user normally makes a series of regular contributions over time. An honest developer would normally (except for perhaps extremely small projects) be consistently committing code they are developing for their project. As a result when a huge aberrant spike occurs in a commit log a presumption can be made that there is a high likelihood that cheating has occurred. In this case, the weighting function can severely or totally discount the project, i.e., give it extremely little or no credit.
[0387] As previously described, in one embodiment there are three kinds of scores that are calculated for each developer. These include knowledge, experience and influence. This level of scoring provides a variety of useful information to evaluate candidates. However it will be understand that the recruiting service could also be implemented with a subset of this set of scores.
[0388] The scores are preferably calculated on a language-specific basis and an overall basis. Language specific scores are useful to evaluate skills in a particular language. However, generating an overall score provides an additional indicator of a developer's talent.
[0389] An exemplary language-specific scoring process will now be described. In one implementation, to determine a knowledge score, an examination is made of lines of code and the number of repositories that the developer has contributed to. The score is then calculated by a function that weights the total number of lines of code in all of the different repositories. That is, a developer who has written more lines of code has more experience and credit is given for contributing to different repositories.
However, the number of lines of code can reach very large numbers. Thus, one way to score knowledge is apply a logarithm function based on the number of lines of code. As one example, a knowledge score for a developer can be generated using a natural log curve: ln(lines) x number of repositories, where "x" is the multiplication operation and this equation is a simplified equation to illustrate a general approach that one of ordinary skill in the art would further optimize for a particular implementation to optimize empirical results. Other variations based on a logarithm function are also possible and other factors could be included in determining a knowledge score.
[0390] An exemplary language-specific scoring process for experience looks at different factors indicative of experience and then weights the factors. For example, to calculate a total experience score the individual experience scores may be combined with work experience, i.e., Total Experience = Individual experience skill experience scores x work experience. Of course, many variations are possible in terms of weighting individual skills experience with work experience. In one implementation the process looks at the lines of code that have been written with particular attention to the lines of code written per day, in addition to commits per day, and the number of days of activity. A weighted function SUM(r) can be used to perform an initial analysis of experience in different skills. The scoring can be further weighted by work experience, resulting in simplified equation to determine a language-specific experience score: SUM(r) x Work Experience, where this equation is a simplified equation to illustrate a general approach that one of ordinary skill in the art would further optimize for a particular implementation to optimize empirical results.
[0391] An overall score can be calculating by weighting individual scores. An overall experience score can be determined as follows:
Experience = Individual experience skill experience scores x work experience.
An overall knowledge score can be determined using different weighting approaches. In one approach a logarithm function is used to weight the sum of different knowledge skill scores so that a high score requires the developer to have a wide variety of skills: Knowledge = ln(SUM(Skill Knowledge)), where this equation is a simplified equation to illustrate a general approach that one of ordinary skill in the art would further optimize for a particular implementation to optimize empirical results.
[0392] The influence score is a measure of the developer's influence in the larger developer community. In one embodiment the influence score includes how much the developer's code influences other developers. Additionally the influence score may include the developer's influence in social media. For example, the influence score may include a component based on how a developer's projects have influenced others, based for example on the number of followers, forks, and contributors, which may be determined from data within code hosting repositories. However, an individual developer may have different influence in different languages, which has to be taken into account in determining an overall influence score. Additionally, the developer's influence in social media may also be considered, such as weighting the influence in social media by a weighting function. For example, one measure of influence in social media is a Klout score. Thus an exemplary overall influence score may be determined as follows:
Influence = f(Klout) + ln(SUM(Skill Influence))
where a developer is given credit for influence in different skills may also be given some credit for social media influence and where this equation is a simplified equation to illustrate a general approach that one of ordinary skill in the art would further optimize for a particular implementation to optimize empirical results.
[0393] The scoring and weighting functions that are applied are determined empirically to give a desired distribution based, for example by examining what weighting functions give the best real-world results at a particular point in time for recruiters. Thus, for example, the actual constants used as weighting factors and aspects of the weighting functions may be varied based on feedback on the usefulness of the scoring for simulated or actual recruiting efforts.
[0394] For example, when calculating knowledge, one approach is to look for extensive experience in several languages. This is because in the real world highly knowledgeable developers have a broad range of experiences to draw upon and are skilled in different languages. Thus, even if a developer has top 10% scores in one or two languages, they cannot get a top 10% overall knowledge score, because that only happens when the developer has top-tier scores in several languages. That is to say, breadth counts.
[0395] The public code repositories are crawled on a regular basis. In one
embodiment all the raw data obtained from a crawl of the public code repositories is saved, except for the source code. That is, it is preferable to save the information obtained by analyzing the code, in addition to the source log itself. The next time the crawler encounters that repository, the source code is downloaded again, and a "refresh" is made based on new contributions. Additionally, the logs are checked to determine the individual(s) that made the new contribution. Thus, if Michael has a repository with a project, the system will also confirm from the log entries which contributors made any new contributions. Thus if Luca makes a follow-on contribution to Michael's project, the follow-on contribution will be credited to Luca. This cross-checking of which individual made which new contribution to a project is useful to improve accuracy and reliability of the scoring.
[0396] The social media access module 135 and the social media aggregation module 140 provide a comprehensive set of social media links for each profile. The author information obtained from public repository sites such as GitHub and Stack Overflow may be incomplete or contain inaccuracies. However typically the author information will include at least an email address and perhaps also a name. This information can then be used to obtain additional social media information using commercial services such as Full Contact, Inc. of Denver, Colorado, Fliptop, Inc. of San Francisco, California and Rap Leaf of San Francisco, California. Many commercial services check by unique information, like email address, or a hash of the email address (a hash is a unique number generated by an email address. That way, companies can match users by email addresses, but protect their privacy by looking at hash numbers). In one embodiment a search of social media sites is performed of all of the sites listed under Full Contact's set of Social Network Types. From this information profile information identifying the names of developers may be generated along with associated information. For example, work history may also be scraped from social networking sites.
[0397] Direct scanning of social media sites is also an option, such as the option of scanning sites such as Linkedln and Google Plus. However, there's usually not a one- to-one results process. For example, if a developer has a common name, such as "John Smith," a scan based on their name may turn up more than one hit. To find additional social media links for a particular profile it is thus desirable to look for multiple matching factors (location, title, company, name, etc.), and then calculate the probability that it's a match. If the probability is higher than a certain number, the system automatically merges the profiles. If the probability is less than that threshold, the system sends a notification that there needs to be a manual review process.
[0398] In one embodiment, an individual may have granted the recruitment enhancement system access to one of their profiles at a social media site. For example, to participate in contest, which is an example of a recruitment enhancement activity described with respect to Figure 28, the individual may have given the recruitment enhancement system permission to access their profile at Linkedln™. When permission is granted in this way, the recruitment enhancement system may be able to access more information about the individual than when social media sites are directly scanned. Once links to social media are identified for a developer they can be refreshed at a rate slower than other information in the public code repositories. Individuals typically add new social networks infrequently and the URLs of social media sites are generally static.
Recruiting Services Using a Recruitment Enhancement System
[0399] In this section, recruitment services that leverage information gathered and analyzed at the recruitment enhancement system are described. In addition, interfaces that allow outside entities to access these services are described. For instance, an interface can be provided for an applicant tracking system that allows a recruiter to access recruitment services, such as candidate scoring, from a recruitment enhancement system. [0400] Figure 30 is a block diagram of recruitment ecosystem 200. The recruitment ecosystem 200 includes two applicant tracking systems (ATSs), 202 and 208 and a recruitment enhancement system 12 including a developer database 206. The developer database 206 includes information about software developers. This example is provided for the purposes of illustration only as the recruitment ecosystem 200 can include a plurality of different applicant tracking systems. In addition, as is described in more detail below, the recruitment enhancement system 12 can be configured to provide access interfaces that don't require an ATS. Further, database 206 can include information about individuals from other professions other than software developers and/or separate databases (not shown) can be maintained for individuals in other professions.
[0401] The applicant tracking systems, 202 and 208, can include data associated with a plurality of individuals that have applied for jobs over some period of time. In applying for a job, each individual may have supplied information that is stored to an ATS. For example, an individual may have electronic uploaded a resume in some format. After the resume is uploaded, the resume may be processed. For example, optical character recognition can be applied to a scanned in image of the resume.
[0402] After processing, information from the resume can be parsed and stored to an ATS database. The format of the database including the record structure and the information stored in the database can vary from ATS system to ATS system. For example, ATS 202 can store a first set of applicant information in a first order and ATS 208 can store a second set of applicant information in a second order.
[0403] As another example, the ATSs may also include information about individuals that have expressed interest in the company through some mechanism, such as a social media application. For example, via a social media application a user may have requested interest in a company and supplied a name and e-mail to receive some type of information. As another example, via a social media application, a user may have provided a social media profile to participate in a company related activity.
Information obtained via a mechanism, such as a social media application, can be stored to the ATS. Thus, in general, there are many different scenarios in which information can be gathered about an individual and then entered into an ATS and information obtained via the job application process is only one example in which information about an individual can be provided to an ATS. [0404] The ATSs can be associated with different entities. For example, ATS 202 can be associated with a first company where it includes information for applicants that have applied for jobs at the first company and ATS 202 can be associated with a second company where it includes information for applicants that have applied for jobs at the second company. As another example, ATS 202 can be associated with a first recruiter that has performed recruitment searches for a first group of companies and ATS 208 can be associated with a second recruiter that has performed recruitment searches for a second group of companies different from the first group. Thus, individuals in one ATS may have applied for jobs at different companies serviced by a recruiter or group of recruiters.
[0405] In some instances, information stored in each of the ATSs can overlap. For instance, when ATS 202 and ATS 208 are associated with companies in the same field, it is possible that the same individual has applied for jobs at each of the companies. Thus, ATS 202 and ATS 208 can include information, such as resume information associated with the same individual. However, because companies and recruiters usually closely guard the information stored in their ATS and don't generally share the information, neither of the entities controlling ATS 202 or ATS 208 is likely to know about the overlaps between their ATS databases.
[0406] Using an ATS database, such as 202 and 208, a recruiter can perform searches over the records in the ATS database. A search can allow a recruiter to identify a group of records that satisfy particular search criteria. For example, if the database stores information related to an individual's experience level with a particular programming language, then a recruiter can perform a search for candidates satisfying the experience level requirements for a particular job. The outcome of the search can be a list of candidates meeting the search criteria and information about each candidate.
[0407] After searching, a recruiter can choose to contact various individuals for additional scrutiny, such as interviews. The recruiter may continue to contact individuals represented in the ATS until a job position is filled. After the job position is filled, the information about individuals that were not hired may still remain in the ATS. Thus, search results can include information about job applicants that have applied for a particular job opening, information about individuals that have previously applied for other jobs and individuals that have been entered into the ATS through other mechanisms, such as via a social media application.
[0408] In particular embodiments, information about all or a portion of the candidates in an ATS can be sent to the recruitment enhancement system 12. As an example, in 210, information about one or more individuals stored in ATS 202 can be sent to recruitment enhancement system 12. As another example, in 214, information about one or more individuals stored in ATS 208 can be sent to the recruitment enhancement system 12.
[0409] The information sent from the ATS to the recruitment enhancement system 12 can be used to provide a recruitment enhancement service. In various embodiments, as will be described in more detail below, one recruitment enhancement service can involve scoring or ranking individuals, such as software developers, which are identified from the information received from ATS in a recruitment service request. Some examples of information that can be used to identify an individual that may be stored in an ATS and sent to system 12 are described with respect to Figure 31A.
Some examples of information that can be returned about an individual from system 12, such as but not limited to a score or a rank are described with respect to Figure 31B.
[0410] Next, some of the information that can be sent to system 12 as part of a recruitment service request is described with respect to Figure 31 A. Then, additional details about Figure 30 are described followed by a description of Figure 3 IB. The individual in Figure 31 A can be one among group of individuals whose information is sent in 210 or 214. In a communication between an ATS and system 12, identification information about each individual, such as a first name and a last name 302, can be sent. In addition, one or more e-mail addresses, 304 and 306 can be sent that are believed to be associated with the identified individual. In one embodiment, the recruitment enhancement system requires a first name, last name and at least one e- mail address to provide a recruitment service involving assessing a skill level of the individual.
[0411] For competitive purposes and for privacy reasons, a company may not wish to release too much information about an individual. An individual name and e-mail address or addresses are likely to be publically available. Thus, an advantage of sending only a name and email addresses is it allows the company to maintain that it is protecting the privacy of individuals because only a limited amount of information is sent and what is sent is likely to be publically available.
[0412] Besides the name and the e-mail address or addresses, one or more additional types of information can be sent. The recruitment service may involve determining whether the recruitment enhancement system 12 has additional information about the named individual. Individuals can share names. Thus, the additional information may allow the system to confirm with a higher level of confidence that a named individual from an outside entity is properly identified by system 12.
[0413] Some example of additional information include but are not limited to location information 308, blog information 310, social profile information 312, nicknames 314, user names 316 and profile names. The location information 308 may be an address where the person lives or works. In one embodiment, the location information 308 may be a partial address, such as a city where the individual lives or works. The blog information 310 is example of information associated with an individual's on- line activities. For example, information 310 can be associated with a user's online blog, such as link (e.g., a URL) to the blog.
[0414] In another example, information 312 may be associated with an individual's on-line profile. For example, a link to a Linkedln™ profile or information obtained from the on-line profile can be sent. Multiple instances of information 312 can be sent. For instance, a first link to a first profile and a second link to a second profile can be sent. The system 12 can be configured to parse the received information when multiple instances of particular type of information or no instances of a particular type of information, such as 312, are received.
[0415] In other examples, information, such as nicknames 314, usernames 316 or profile names 318, associated with a user's on-line activities can be provided to system 12. As will be described in more detail as follows, the received information can be used to determine whether the named individual also has a record in the developer database 206 maintained by the recruitment enhancement system 12. In general, any information available at the ATS that can help to locate information about the individual in the developer database 206 or other database maintained at the recruitment enhancement system can be sent.
[0416] The information that is sent can vary from individual to individual. For example, for a first individual, a name and one e-mail address can be sent. For a second individual, a name and two e-mail addresses can be sent. For a third individual, a name, one e-mail address and a link to a social profile. For a fourth individual, a name, two e-mail addresses and a number of different user names can be sent. Thus, the recruitment enhancement system 12 can be configured parse and utilize different combinations of information that vary from individual to individual.
[0417] In a particular embodiment, the recruitment system can be configured to receive information about an unnamed individual. For example, the first name and last name 302 may not be known. Based upon the received information, the system 12 can be configured to determine a possible name of the individual and return the information to the entity that sent the information about the unnamed individual.
[0418] The information described above doesn't necessarily have be stored in or sent from an ATS. In one embodiment, the received information is used to locate information about individuals in a database maintained by the recruitment enhancement system 12. For this purpose, it doesn't matter where the information resides before it sent as long as it is suitable for locating records of individuals maintained in the database or databases of system 12. Thus, the example of an ATS as a source of the information sent to the recruitment enhancement system is for the purposes of illustration only and is not meant to be limiting.
[0419] In one embodiment, besides information about the individuals, information about a job position for which candidates are desired can be sent. For example, a portion of a job description can be sent in a recruitment service request. In one embodiment, when the system 12 receives a job description and a list of a number of individuals, the system 12 can be configured to store information about the individual and information about the job position for which they are being considered to a database maintained at system 12.
[0420] Returning to Figure 30, in one embodiment, a plug-in module, such as 204 and 206, can be provided for an ATS. The plug-in module can be configured to be compatible with the recruitment enhancement system 12. The plug-in modules can be configured to extract information from an associated ATS and send it to the recruitment enhancement system in format that is recognizable by system 12.
[0421] In one embodiment, the plug-in module may provide an export feature. For instance, an export button can be located within an interface provided with the search functions of the ATS. In response to a search, a selection of the export feature can cause an electronic data file to be generated that is compatible with the recruitment enhancement system 12. The electronic data file can include information consistent with the search, such as the names and e-mail addresses of the individuals identified in the search. As described above with respect to Figure 31 A, additional information related to a particular individual can also be included.
[0422] An API (Application Program Interface) can be provided with system 12. The API can describe formats and particular data types that system 12 can accept. The API can be used by third-party developers to construct a plug-in module for an ATS.
[0423] The extracted information can be sent to the recruitment enhancement system in 210 or 214. The information can be sent as part of a recruitment service request. The plug-in module may require a user to provide an additional input before the data is sent, such as a selection of a send button. In one embodiment, the plug- in module, such as 204 or 206, can be configured to establish electronic communications with the recruitment enhancement system 12 and then send data to the recruitment enhancement system 12 that has been extracted from an ATS. The data can be sent electronically via a secure connection between the ATS and the recruitment enhancement system 12. The information that allows the secure connection to be established may have already been exchanged by the systems.
[0424] In one embodiment, entities can be charged according to their utilization of the recruitment enhancement system 12. Thus, the data sent from the ATS, such as 202 or 208, can include identification and authorization information for an entity requesting a service. The authorization information can be used to determine whether the entity sending the data is authorized to obtain a requested service from the recruitment enhancement system 12. The identification information may allow the recruitment enhancement system to attribute its utilization to the entity and bill it for the services.
[0425] Although not shown, the recruitment enhancement system 12 can be configured to provide an interface, such as web-based interface, that allows an entity to upload information that is compatible for receiving a service from system 12, such as a file including a list of names and e-mail addresses to the system. The system 12 can be configured to process the file and generate a report. Upon payment, entity can receive the report. The report can be emailed to an address specified by the entity or a link can be provided that allows the entity to download the report. Thus, a plug- in module that provides an interface to an ATS may not be required. [0426] After receiving the individual information in 210 and 214, the system 12 is configured to determine whether it has information about the received individual's in its developer database 206. Possible results of this determination are shown graphically in Figure 30.The largest circle represents individuals in the developer database 206. Group 218 can include one or more individuals from ATS 202. It can be seen that a circle associated with group 218 partially lies within the large circle associated with 206 and partially outside of it. The portion inside the circle represents individuals in group 218 that can be matched to individuals in the developer database 216. The portion outside the circle 206 represents the portion of individuals that can't be matched to individuals within the developer database 206 with an acceptable degree of confidence.
[0427] In this example, most of the individuals in group 218 were matched to individuals described in database 206. In other embodiments, it is possible no matches may be found. In yet other embodiments, it is possible that all of the individuals in a group, such as 218, can be matched to individuals in the database 206. Thus, the fractions shown in Figure 30 are for the purposes of illustration only and are not meant to be limiting.
[0428] The system 12 can maintain a number of databases that identify individuals. The individuals can have different job skills and may be members of different professions. At some point, the system 12 may have received information that allows a score or a rank to be developed for the individual. In various embodiments, the scores or ranks can be an indicator of such factors, as the quality of their one or more job skills, their influence within their profession and their effectiveness at their profession. Details of generating these scores and ranks for software developers are described above with respect to Figure 29.
[0429] When reporting the results of a search, the system 12 can report which individuals received in a group, such as 218, were matched to an individual in at least one database maintained at 12 and which individuals were not matched. For named individuals in a group that are matched to someone in the developer database 206 a confidence measure can be determined. The confidence measure may be based upon the amount of information that is matched. For example, a number of points can be given for a last name match, a number of points can be given for a first and last name match and a number of points can be given for each e-mail address match. The more information that is matched the higher score and hence the higher level of confidence that the named individual is a match for an individual in the developer database 206.
[0430] In one embodiment, when an individual is not matched or the confidence measure is determined to below a certain threshold, the system 12 can be configured to initiate a search and attempt to locate information about a named individual in a group that allows them to be scored or ranked or raises the confidence measure of the match to an individual in the database 206. A named individual newly scored or ranked can be added to the database 206 and their score/rank can be reported in reply to the service request, such as in 212 or 216. When a confidence measure of a match has been increased as a result of search such that it exceeds threshold value for reporting the result, then the result can be reported in a reply to the recruitment service request.
[0431] In one embodiment, when a named individual can't be located in the database 206 and information about the named individual can't be found which allows the individual to be scored or ranked, the system 12 can still be configured to keep a record of the individual. The system may keep the information so that future searches can be carried out in which attempts are made to gather information on the named individual where the gathered information can be used to develop a score or rank for the individual. Further, if the named individual shows up again as a result of a request from another entity to score the individual, then the system 12 can report that an individual may be actively looking for a job based upon the fact that the system 12 has received information about the individual from multiple entities. This information can be provided even if a score or rank is not available for the named individual.
[0432] In one embodiment, when a match is not found for a named individual in a service request and a record of the unmatched named individual is stored, the system can be configured to attempt to match the named individual at a later time. It is possible a match may be subsequently found and a score or ranking may become available for the named individual that was not previously matched. The system 12 can be configured to report the score or ranking to one or more entities that requested a score or ranking for the named individual that originally were notified that a score or ranking was not available for the individual or the confidence level of the match was below a threshold value when a score or ranking becomes subsequently available. This feature may be a system option in that a service requester can specify whether they wish to be updated or not if additional information is subsequently discovered about a named individual, such as whether a score or ranking has been developed for the name individual that was previously not available.
[0433] In general, the system can be configured to allow the recruitment service requester to specify whether or not they want updates regarding named individuals that have been previously submitted in a service request, such as 210 or 214. For example, the system can be configured to maintain a watch list for an entity including a number of named individuals in a group. In response to specified triggers, the system 12 can be configured to notify the entity. For instance, the system can notify an entity, such as recruiter, when it is determined a named individual on the watch list appears to be searching for a job. In another example, the system can be configured to notify an entity, when a score or ranking of an individual on the watch list has changed in some manner, such as exceeding some threshold value or changing by some percent amount.
[0434] The notifications from system 12 can occur through one or more different communications channels. For example, an e-mail can be sent to a recruiter. In another example, a text message can be sent to a recruiter's portable electronic device. In yet another example, an e-mail and text message can both be sent.
[0435] The system 12 can be configured to receive in a request for a recruitment service subsets of named individuals grouped in some manner. The named groups may be used for comparison purposes. These different subsets are represented by the circles in Figure 30 that are smaller than the largest circle. As described above, the largest circle can represent individuals in the developer database 206 and portions of the smaller circles lying outside of the largest circle represent named individuals in a group that have not been matched to an individual in the developer database 206.
[0436] In one example, the recruitment service request 210 for ranking/scoring of individuals can include groups 218, 220, 230 and 232. The named individuals in group 218 are possible job candidates. These candidates may have applied for a job over a first time period, such as in the last year. Group 220 may be individuals that are distinct from other members of group 218 in some manner, such as having applied for a job in the last month or having applied for a job multiple times. Group 230 can include individuals that have been offered a job. Group 232 can include individuals that have been hired. [0437] In another example, the recruiting service request 214 can include three specified groups, 224, 226 and 228. Group 224 can be a group of individuals that have applied for jobs in between 6 months to a year ago. Group 226 can be individuals that have applied more recently for a job or jobs, such as within the last month. Group 228 can represent a current employees of a company that perform a job or jobs that are similar to the job or jobs for which the named individuals in group 228 are applying.
[0438] Scores/ranks can be provided for each individual in a group as well as for the group as whole (e.g., an average score for the named individuals in a group can be provided). The scores/ranks can be used for various comparison purposes. For example, the score of a named individual in the recent 220 group can be compared to the average score or rank of the hired individuals in group 232 or the average score or rank of the individuals that were offered jobs in group 230. Similar comparisons can also be made between individuals in groups 224, 226 and 228. For example, scores/ranks of a named individual in group 226 that has recently applied for a job can be compared to scores/ranks of individuals in group 228 or an average of scores/ranks for individuals in group 228. In another example, the average rank/scores of the individuals in group 224 that have applied 6 to 12 months ago can be compared to another group, such as individuals in group 220 that have applied for a job in the last month.
[0439] In one embodiment, this comparison feature can be used as a filter. For example, the system 12 can be configured to only return information on named individuals in a group whose score or rank is above the average of the named individuals of a comparison group. As another example, the system 12 can be configured to only return information on named individuals in a group whose score/rank is above some threshold value, such as the average score/rank of the named individuals in a group.
[0440] In the examples in Figure 30, some of the named individuals that are submitted in different recruitment service requests can overlap. For instance, some of the named individuals in group 218 can also be named individuals in group 224. In Figure 30, the overlap of members for groups 218 and 224 is represented by the intersections of the circle 222. In example in Figure 30, the entity associated 202 and the entity associated with 208 may not be aware of their individual recruitment service requests and the fact one or more individuals are in each request as different entities, such as companies don't share this information. However, when a named individual is identified from multiple recruitment service requests, the system 12 can be configured to keep track of and leverage this information.
[0441] As an example, the system can be configured to record each time an individual appears in a recruitment service request. If a named individual appears multiple times over some period, it can indicate the candidate is looking for a job. If the named individual is highly ranked, this factor can be used as a trigger to notify a recruiter. For instance, as described above, a recruiter can place an individual on a watch list and request to be notified under some conditions. One of the conditions can be if the individual appears to be looking for a job which may be determined from their appearance in multiple recruitment service requests. The system 12 can be configured to allow a user to specify parameters that determine under what conditions they are to be notified.
[0442] In a recruitment service request, such as 210 or 214, a large number of names can be submitted. For example, tens of thousands of names can be submitted. The processing of a large group may take a significant amount of time. In one
embodiment, the system 12 can be configured to provide results of a recruitment service request all at once. For example, the system 12 can be configured to process all of the named individuals in a recruitment service request and then return the results in a batch mode when request is completed. In another embodiment, the system 12 can be configured to send a portion of the results in real-time before the request is completed. For example, when a match is made for a named individual in a recruitment service request, the results can be returned as soon as the match has been made. As another example, when a match is made for a named individual meeting certain criteria (e.g., a ranking or score about some specified value), the results can be returned as soon as the match is made. The remaining results including or not including individual results previously sent can be returned in a batch mode when the matching process for the recruitment service request is completed.
[0443] In one embodiment, the plug- in, such as 204 or 206, can be configured to automatically submit recruitment service requests to the system 12. For example, each time an individual applies for a job and their information is entered into the ATS 202, plug-in 204 can be configured to submit a recruitment service request to system 12. When a match is found, the system 12 can return information, such as a score or ranking, about the individual. The returned information may be stored to the ATS. In another example, the recruitment service request can be triggered after some number of applicants have applied, such as after receiving 10 applications. Then, the plug-in module can send a recruitment service request including ten names to system 12. In another example, the recruitment service request from the ATS can be triggered at some time interval, such as once a day.
[0444] In the example above, one or more named individuals are described as being sent in a recruitment service request to system 12, the system 12 tries to match the named individual to an individual stored in a database, such as database 206, and then information, such as scores for matched individuals can be returned. The use of the service is described with respect to recruiting individual candidates for jobs.
However, the use of the system is not limited to recruiting individual candidates for jobs.
[0445] For example, in one embodiment, the system 12 can be used to rank or score the strength of a team, such as a software development team. An employer may have a number of software development teams for different products. The individuals on each team can be submitted to system 12 and the development team can be scored. Based upon the scores, a manager may decide to reformulate the groups. For instance, some development teams have much higher scores than other development teams and a manager may reshuffle the team members such that the scores for the teams are more balanced.
[0446] In another example, a company may wish to hire a team of developers for a project. The company may receive bids from different contractors. The company can request the names of individuals that are going to be working the project for each contractor and submit their names to the recruitment enhancement system 12. The system can return scores for each of the development teams for each contractor. Based upon the scores/rankings provided for each of the team members as well as the scores as a group, the company can decide which contractor to hire for the project.
[0447] In yet another example, a company, such as a software contractor, may have their software developers ranked/scored by system 12. The system 12 can generate individual scores. The individual scores can be used to develop company scores. Then, the company may use individual scores and/or the company scores to promote the technical expertise of their employees. [0448] As described above in response to a recruitment service request including a number of named individuals, such as 210 or 214, results which may include scores and/or ranks can be returned to the requester in 212 and or 216. An example of what information can be returned is described as follows with respect to Figure 3 IB. Other types of information can be returned and the example provided in Figure 3 IB is for the purposes of illustration only.
[0449] In one embodiment, the individual output data 350 can include some set of baseline information and optional information. As an example, the baseline information can include a link to a profile 352 or the information contained in a profile. The profile 352 can be a profile collated from various information sources to which system 12 has access, such as various on-line sources. The baseline information can include one or more scores, such as 354 and 356 (e.g., see description of Figure 29 above for more details), skills possessed by the individual and an indicator 360 of the confidence measure of the match. The confidence measure can be indicated as a score, such as a numerical score, or according to some scale, such as high, medium or low.
[0450] In one embodiment, the system 12 can be configured to provide standard scores. In another embodiment, the system 12 can be configured to provide customized scores. As described above, the scores can be determined using a number of different weighting factors. The system can be configured with specifiable parameters that allow a person using the system, such as a recruiter, to influence the scores in some manner. For instance, more weight can be put on experience as opposed to some other factor, such as experience. The system 12 may allow a recruiter to vary the weights from search to search but may report for each search what weights were used.
[0451] In another example, a company may wish to develop custom scores to suit specifics of their recruitment needs. The custom scores can be utilized for all of the company's reports from the system 12. In one embodiment, a plug-in module for an ATS, described above with respect to Figure 30, can be configured to implement custom scoring for a company such that all of the searches involving the ATS will use the custom scoring.
[0452] Returning to Figure 3 IB, some examples of optional information include one or more social profile links 362, additional contact information 364 and a current job status. The formulation of the baseline information is provided for illustrative purposes only and different combinations of baseline information and optional information can be provided. Additional information that can be provided, an interface for outputting the information and example of a profile that can be returned is described in more detail in U.S. patent application no. 13/499,791, entitled
RECRUITING SERVICE GRAPHICAL USER INTERFACE, incorporated herein by reference.
[0453] In one embodiment, the system 12 can be configured to receive parameters that allow a user to customize the information and the format of the information that is received in 350. The customization may be implemented on a request by request basis, implemented globally as part of the plug-in module or implemented on a user by user basis For example, a first user can specify one set of preferences and a second user may specify a second set of preferences for receiving information 350 where the preferences can affect the type of information that is received and its format.
[0454] In one embodiment, for a large job including information about many individuals, the system 12 can electronically send a report or a link to location that allows the report to be downloaded when the report is ready. In addition, the plug-in modules, such as 204 and 206, can be configured to add all or a portion of the information in the report to an ATS. For example, scoring results can be integrated into the ATS such that future searches involving the ATS can return scoring information without having to contact the recruitment enhancement system 12. The plug-in module can be configured to periodically communicate with the recruitment enhancement system to update the scores or ranks stored to the ATS.
[0455] Figure 32 is a flow chart of a method 400 of a recruitment enhancement system. In 402, the system can analyze and score developers as was described above with respect to Figure 29. The analysis and scoring process can be an on-going activity where scores are updated with some frequency. The results from the scoring and analysis process for various individuals can be stored to a developer database, such as 206 in Figure 30.
[0456] In 404, the system can receive a recruitment service request. The request can include information, such as a name and an e-mail address, for a number of different individuals. Different groups of individuals can be specified in the request. The specified grouping can be used for comparison purposes. For example, individuals in a first group can be compared to individuals in a second group. In 406, a search can be carried out to determine whether individuals named in the recruitment service request can be matched to individuals in a database, such as a developer database that was populated in 402. In one embodiment, the database can be a proprietary database.
[0457] In 408, in response to the recruitment service request, the database can be updated. For example, the fact that a match was determined for an individual in the database, when the match occurred and the entity requesting information about the individual can be stored. If a job description was included in the request, then information about the position can be stored. If many matches for an individual occur over some period as a result of the system receiving different recruitment service requests, the system may determine that the matched individual is looking for a job. In another example, even if a match is not found, a record can be created in the database that includes information about the individual for which a match was not found, such as their name, e-mail address and the entity the provided the information.
[0458] In 410, a notification event can be triggered. For example, a recruiter may place an individual on a watch list where if the system determines the individual is looking for a job then the recruiter is to be notified. Other parameters can be utilized to define a triggering event and the example of determining that the individual is applying for jobs is provided for illustrative purpose only. For example, another triggering event for notification can be if a score or ranking associated with the individual reaches some threshold value or changes by some amount.
[0459] In 412, the system can perform any requested comparisons and scorings, such as a team scoring or individual to group comparisons. In 414, the system can determine confidence measures for any determined matches. In one embodiment, individual results may only be reported when the confidence measure exceeds some threshold. In 416, the system can compile and send a response to the recruitment service request including the results, such as scores. For example, a report can be generated and a link to the report can be sent to a recruiter. The report can include overall statistics associated with the request, such as but not limited to a total number of matches, a total number of non-matches and an average rank/score of the matched individuals.
[0460] In 418, the system can tally billable events and bill the client accordingly. For example, in one embodiment, a billable event can be a match with a confidence level above a certain threshold. A client can be charged some amount for each match satisfying these criteria. Thus, in response to recruitment service requests involving one or more successful matches, a client can be billed according to the number of successful matches.
V. SOCIAL MEDIA DATA FILTERING FOR ELECTRONIC JOB RECRUITING
[0461] As will be described in more detail as follows, a recruitment enhancement system for electronic job recruiting system is described. The recruitment enhancement system is configured to gather and analyze information about potential employees. For example, the system can be configured to gather information about individuals from publically available sources, such as a social media sites. The aggregated individual information can be made available to users of the recruitment enhancement system.
[0462] To prevent use of the aggregated individual information in a manner that is considered illegal for recruiting purposes, the aggregated individual information can be filtered before being output by the system. For example, the aggregated individual information can be filtered for protected class information, such as race, gender, age, etc., before it is presented to a user. Filtering the information in this manner may prevent someone from eliminating someone for a job position in an illegal manner or asking inappropriate questions that could indicate discriminatory intent. In one embodiment, a Bayesian type data filter that is trained to recognize protected class information can be employed.
[0463] One objective of the analyzing the aggregated individual information is to identify job applicants most likely to successfully perform jobs associated with available job positions. For example, a job applicant's proficiency or knowledge pertaining to a particular aspect of a job can be assessed and scored, such as their knowledge or proficiency of a particular programming language for a programming job. The scoring can be used to rank applicants which can be used to screen or filter job applications.
[0464] The information derived from the analyses, such as the scores, can be referred to as job suitability parameters. The job suitability parameters can be designed to aid job recruiters during the manual selection process of job applicants. If desired, the job suitability parameters can be used in conjunction with the filtered aggregated individual information to make recruiting decisions. The recruitment enhancement system can be configured to generate an interface that allows job recruiters to view the job suitability parameters and filtered aggregated individual information for various candidates. There are various local and national laws and regulations that govern recruiting decisions including what information can be used to the make the decisions. Thus, the system can be configured to use various techniques that attempt to identify and present only information that is allowable according to these laws and regulations.
[0465] With respect to the following figures, details of a recruitment enhancement system that includes one or more databases of information about skilled individuals and recruiting services provided by the system using these databases are described. One type of information that can be stored to the databases can be self-generated information, such as blog posts or tweets, made by the individuals and submitted to publically accessible web-sites. As another example, this type of information can be electronically submitted in a resume when the individual applies for a job. An interface can be provided with the system that allows the self-generated information to be accessed by job recruiters for the purposes of making recruiting decisions. The system can be configured to apply one or more filters to the information before it is presented to the job recruiters. In one embodiment, the system can be configured to identify and remove information that may possibly contravene recruiting laws and regulations.
[0466] In particular, with respect to Figure 33, a recruitment ecosystem is described. Within the recruitment ecosystem, interactions involving individuals seeking jobs, recruiters seeking candidates, companies providing jobs and a recruitment enhancement system are discussed. With respect to Figure 34, a recruitment enhancement system is described. In particular, methods and apparatus for formulating a proprietary database of information relating to skilled individuals are discussed. With respect to Figure 35, 36A, 36B, methods and apparatus for recruiting services that leverage the proprietary databases maintained within the recruitment enhancement system are discussed. The recruiting services can involve receiving a list of individuals from a recruiter and attempt to match individuals from the list to individuals stored in the proprietary databases. Reports can be generated for matched individuals on the recruiters list and returned to the recruiter. The recruiter can use information in the report to further their recruiting activities.
[0467] With respect to Figure 37, a method of obtaining and aggregating publically available information about individuals, such as information available from social media sites, is described. With respect to Figure 38, methods are described for filtering the aggregated individual information obtained using the method of Figure 5. In one embodiment, the filtering can involve identifying and removing protected class information to prevent it from being used in a manner that could be inferred to be discriminatory. With respect to Figure 39, methods for initializing and maintaining data filters are discussed. In a particular embodiment, a Bayesian type filter can be trained to estimate a probability of whether a particular unit of data is protected class information. Finally, with respect to Figures 40A and 40B, an example of individual information that has been gathered, aggregated and then filtered by the system is described.
Recruiting Ecosystem Including Recruitment Enhancement System
[0468] In this section a recruiting ecosystem including a recruitment enhancement system is described. In a recruiting ecosystem, individuals can engage with various electronic systems to supply information that allows them to apply for existing jobs, or to be considered for future jobs when positions open up. In response to the supplied information, recruiters can select from among the individuals and attempt to initiate additional job recruiting related interactions. For instance, based upon the received data, a job recruiter can identify an individual and attempt to contact the person to set up an interview.
[0469] A job prospect can be a person that is interested in working for a company and has agreed to provide information, but may not have applied for a particular job. A job applicant can be a person that has applied for a job, and agreed to provide information as part of the application process. As described herein, a recruitment enhancement system 12 can be provided, that can be used to gather information, and engage job candidates and job applicants alike.
[0470] In the recruitment enhancement system, methods and apparatus can be provided for gathering secondary recruitment data that is beyond the resume and other application data that is usually gathered in the job application process, via applicant tracking systems. The secondary recruitment data can be analyzed and scored. Based upon the score, one or more rankings for a job candidate can be generated. In one embodiment, the secondary recruitment data and rankings can be stored in a proprietary database maintained at the recruitment enhancement system.
[0471] As an example of ranking or scoring, an individual in the recruitment enhancement system 12 can be ranked independent of, or relative to, a group of other job candidates or job applicants, such as a group of job applicants applying for the same job, or a group of job candidates within the same profession. An independent rank can involve comparing the individual's performance to some derived scale. A relative rank can involve comparing the individual's performance to other individuals within a group.
[0472] In addition, the recruitment enhancement system can provide an interface for presenting secondary recruitment data. One example of secondary recruitment data may be self-generated information, such as blog posts, tweets, pictures, or profile data, generated and posted to publically accessible sources, such as web-sites. In addition, a job candidate may submit secondary recruitment data as part of a job application process, such as when the job candidate electronically submits a resume as part of an Internet-based application process. The recruitment enhancement system can be configured to gather the secondary recruitment data, aggregate it and generate an interface that allows the aggregated information to be viewed by a recruiter.
[0473] In one embodiment, the system can apply a number of different types of filters to the secondary recruitment data before it is presented to a job recruiter. One type of filter may be configured to identify and remove before it is presented, individual information that could be used to make an "illegal" recruiting decision. As described above and as will be described in more detail with respect to Figures 38, 39, 40A and 40B, one type of filter can be configured to detect and remove protected class information.
[0474] In one scenario, as will be described in more detail with respect to Figures 35, 36A, 36B and 37, a company can send information related to a group of individuals that have recently applied for job, have applied for a job in the past, or have provided information indicating their interest in working for the company. The information associated with these individuals can be stored in a company ATS (applicant tracking system). Upon receiving the information, the recruitment enhancement system can be configured to rank the group of individuals one or more different ways. For instance, individuals recently applying for jobs can be compared against individuals that have been hired by the company. In another example, the group of individuals can be compared against a group of individuals stored in a proprietary database associated with the recruitment enhancement system.
[0475] As described above, the recruitment enhancement system can be configured to receive information from the company ATS. During a job application process, a job candidate may submit information, such as protected class information, which a recruiter may not wish to view because of legal liability reasons associated with complying with hiring laws and regulations. This information may be stored in the company's ATS. In one embodiment, after receiving the information the system can be configured to apply one or more filters to the information, aggregate it with information gathered by the recruitment enhancement system and then present it to the job recruiter. Thus, the company's ATS information and the information gathered by the recruitment enhancement system can be filtered in a consistent manner.
[0476] The recruitment enhancement system 12 can be configured to generate an interface that allows an outside entity, such as a job recruiter, to access the ranking data and/or the filtered secondary recruitment data about candidates. The outside entity can use the ranking data and secondary recruitment data as a basis for additional investigations, such as learning more about particular individuals, and determining whether to contact particular individuals for additional interactions, such as a job interview. In one embodiment, as described in more detail with respect to Figure 3, the interface can be provided as a part of a plug-in application for their ATS.
[0477] In particular embodiments, the recruitment enhancement system 12 can be tailored to particular professions. For example, the recruitment enhancement system 12 can be tailored towards the computer programming profession, where the system is configured to gather secondary recruitment data that allows computer programming professionals to be ranked according to some scale, and/or relative to other computer programming professionals that have participated in a common recruitment enhancement activity. A recruitment enhancement system associated with program developers is described below in more detail with respect to Figure 2. In another example, the recruitment enhancement system 12 can be tailored towards medical professionals, such as doctors or nurses, where the system is configured to gather secondary recruitment data that allows the doctors or nurses to be ranked.
[0478] Figure 33 is a block diagram of a recruitment ecosystem 50 including a recruitment enhancement system 12 in accordance with the described embodiments. The ecosystem 50 includes company systems, such as 6, recruiter systems, such as 8, a recruitment enhancement system, such as 12, and 3rd party systems where individuals can engage in various activities, such as 10. Social media sites are one example of 3rd party systems. An individual 2, who may or may not be currently engaged in the job hunting process may interact at various times with each of the systems via a computational device, such as the tablet device 4. Typically, the interactions may involve establishing communications between the computational device and one or more remote servers associated with each of the systems over a local and/or wide area network. For instance, the tablet device 4 can be used to establish communications and interact with a server associated with a social media site.
[0479] Communication devices typically include a processor, a memory, networking capabilities, and a user interface that allows for the input and output of data. Examples of communication devices that can be utilized include, but are not limited to, smartphones, laptop computers, netbook computers, desktop computers and tablet computers. Interface devices that can be utilized include a display, a keyboard, and a microphone in combination with speech recognition, a mouse, a touchpad and a touchscreen.
[0480] The recruitment enhancement system 12 can include components for a) direct gathering of recruitment data 24, b) generating a recruiter interface 26, c) indirect gathering of recruitment data 28, d) scoring and ranking 30, e) data matching 32, f) data quality assessing 34, g) data aggregating 40, h) data filtering 42 and i) search logging 44. The system 12 can be hosted on one or more servers including processors, memory and network interfaces. Direct recruitment data gathering 24 can involve generating one or more activities in which an individual can directly participate, such as a knowledge-based test, a game, a puzzle or solving a problem. The direct recruitment data gathering can be used to identify and engage individuals that have interest in a company, or encourage individuals to be interested in a company. In addition, the direct recruitment data gathering can be used to possibly assess potential job candidates.
[0481] In general, the capturing of data from an individual directly engaging in an activity can be part of direct recruitment data gathering. For instance, an individual engaging in an activity, such as an interview, can be captured on video data as part of direct recruitment data gathering 24. In another example, when an individual takes a test, information such as their answers, how long they take to answer each question, and their total time can be gathered and analyzed.
[0482] The indirect recruitment data gathering can involve gathering information from an individual's on-line activities, such as participation in social media sites 36, professional sites 38 or even sites related to a user's hobbies. The person's on-line activities can be hosted on 3rd party systems 10, such as a social media site. In one embodiment, when a user agrees to participate in an activity involving direct recruitment data gathering, a user may agree to provide access to their on-line activity information which can be captured by component 28. For example, a user can provide access to their Facebook™ or Linkedln™ profile for a chance to participate in a knowledge-based test related to recruiting. The knowledge-based test can be referred to a recruitment enhancement activity. After access is granted, the indirect recruitment data gathering component 28 can retrieve information about their on-line activities from the 3rd party systems.
[0483] The information gathered by components 24 and 28 can be stored and analyzed. Then, job suitability parameters can be derived. For example, the scoring and ranking components 30 can be used to score information directly gathered during an online test and then rank individuals based upon the scores. In addition, information about an individual's online activities can be gathered indirectly from various third-party systems 10, and job suitability parameters can be derived from this information. For example, based upon software code that a user has posted to a professional site, the individual's coding ability can be scored and ranked. The software code may be executable. However, the software code doesn't have to be executable to generate a score or a ranking. Methods related to scoring code are described below with respect to Figure 34.
[0484] In particular embodiments, scores and ranks can be derived from data that is gathered directly and/or indirectly. For example, as will be described below, a score and/or a rank can be generated based upon an individual's participation in a knowledge-based test. In another example, a score and a rank can be generated based upon an individual's participation in a knowledge based test, and information retrieved from a third-party site, such as social media site. Again, a person can be ranked relative to a group of other individuals, such as a group of individuals participating in a common task, or according to some derived scale or measure.
[0485] The data matching component 32 can be used to match various data sets to a particular individual. For example, an individual may use multiple names in their online activities. As another example, different individuals can share a common name. The data matching component 32 may use information verified from one source to validate another information source. For example, if an individual grants access to their Facebook™ profile and then another source of information is available, such as their profile in a professional organization or data that they have supplied in applying for a job, the information from the two sources can be compared by component 32 to see if they are the same person. When the comparison indicates they are the same person, a portion of the information from the multiple sources can be stored at the system 12 or at least the links to the multiple sources can be stored. This information can be made available in a recruiter interface.
[0486] The data quality assessing component 34 can be used to assess a validity of gathered data. For example, the component 34 might be used to determine whether someone has cheated on a knowledge-based test administered by system 12 or tried to game the system 12 in some other manner. As another example, the component 34 might be used to determine whether a person has falsified data they have provided as part of a job application. Information that is identified as invalid may be flagged. In addition, the system 12 can be configured not to use information identified as possibly invalid for scoring and ranking purposes.
[0487] The data aggregating component 40 can be used to aggregate information received from a number of different sources, such as via the direct recruitment data gathering 24 and the indirect recruitment data gathering components 28. As described above and in more detail with respect to Figure 35, information can be obtained from crawling or scraping publically available web-sites or from a direct download from an ATS. Before the aggregated data for an individual is presented via the recruiter interface 26, the data filtering 42 can be applied. [0488] The data filtering 42 can be used to identify certain types of data that may be of interest to a user of the system. The types of data of interest can be provided as search parameters input by the user. The identified information that satisfies the search parameters can then be output via the interface 26. In one embodiment, the data filtering 42 can be configured identify certain types of information that are to be blocked from output by the system. For example, in one embodiment, a filter can be provided which removes protected class information from the aggregated data. Then, a set of data filtered for this information can be output via the interface 26.
[0489] In particular embodiments, the system can be configured to filter externally provided information. For example, a recruiter wishing to view information in their company ATS can submit the information to the recruitment enhancement system. The system 12 can apply one or more filters, such as a filter to remove protected class information, and then return the filtered information to the ATS. The ATS can then store and/or output the filter information to the recruiter according to an interface provided by the ATS. The information received from the ATS may or may not be added to databases maintained by the recruitment enhancement system.
[0490] As another example, a recruiter can execute a search locally on their ATS and the ATS can return a set of information. Prior to outputting the information, the ATS can establish communications with the recruitment enhancement system and invoke one or more filters available at system 12. The information to be filtered can be sent to system 12. After the information is filtered by system 12, it can be returned to the ATS, such as 16 or 20. Then, the filtered information can be output via local interfaces, such as 14 or 18.
[0491] The search logging 44 can be used to keep records of searches performed at the system 12. For instance, when a search is performed, the system can store a record of when the search was performed, who performed the search, information related to search parameters and filters applied to the information and some details related to information returned in the search. For example, names and/or pictures of individuals identified by a search can be stored. In some instances, the search logs can be stored for years.
[0492] The recruiter interface 26 can allow recruiters to directly access some portion of the information available at system 12. In another embodiment, as described in more detail with respect to Figure 35, a recruiter can access the system 12 via a plug- in module coupled to their ATS, such as 16 or 20. As described above, the information can be filtered before it is output via the recruiter interface 26. For example, via the interface, recruiters may be able to access scores and ranks about various individuals. Further, via the interface a recruiter may be able to search for individuals with particular skills and assess their job suitability parameters, such as scores and ranking. In addition, via the interface, the recruiter may be able to view secondary data that is available for individuals identified in the search. The secondary data can be filtered so that only certain types of information are output, such as education details. After initial filtering, a second filtering operation can be instantiated to identify and remove certain types of information, such as protected class information (e.g., age related information).
[0493] The company 6 and the recruiter 8 can each maintain applicant tracking systems, such as 16 and 20. If a recruiter 8 works for the company 6, then the recruiter 8 may not maintain a separate applicant tracking system. The recruiter 8 may have access to an interface or interfaces that allows them to interact with the recruitment enhancement system 12, an individual 2 via their device 4 or some other mechanism, their own application tracking system 20 and a company application tracking system 16.
[0494] The company 6 can sponsor a job-site interface 16. Via the job site interface 14, an individual, such as 2, may be able to learn about different jobs and optional apply for jobs. The job application process can include uploading a resume.
Subsequently, the information in the resume can be filtered via system 12.
Information from the job application process can be stored to the company application tracking system 16. In particular embodiments, the company 6 may allow recruiters, such as 8, and/or the recruitment enhancement system to access the application tracking system 16.
[0495] In one embodiment, one or more direct recruitment data gathering activities can be triggered from the job-site interface 14. For example, when a user applies for a particular job, such as after submitting their information or a part of submitting their information, a link can be established from the job-site interface 14 to the recruitment enhancement system 12. The recruitment enhance system 12 can then generate a recruitment enhancement activity that is used to directly gathered recruitment data. For example, as part of the recruitment enhancement activity an individual may be asked to take a knowledge-based test, play a game, solve a puzzle, solve a problem or combinations thereof.
[0496] Information captured from the recruitment enhancement activity can be scored or ranked and made available to recruiters via the interface generated by the recruiter interface generation component 26. An individual can participate in a number of different recruitment activities. Thus, the interface may allow a recruiter to view descriptions of the recruitment enhancement activities in which an individual has participated and jobs suitability parameters derived from their performance.
[0497] In one embodiment, via the interface, a recruiter may be able to send a request to an individual to participate in a particular recruitment enhancement activity. For instance, the recruiter may be interested in an individual for a particular job position. Via the interface, the recruiter can send a message to the individual indicating their interest in the person and requesting them to participate in a recruitment enhancement activity, such as taking a knowledge-based test associated with the job position. The message received by the individual can include information, such as a link to a website, which allows the individual to engage in the recruitment enhancement activity.
[0498] In one embodiment, the recruitment activity that is selected can be related to information associated with an available job. For example, a knowledge-based test can be implemented that is related to a skill needed for the job position. Their answers can be scored and the person can be ranked relative to other job applicants that have taken the test.
[0499] In other embodiment, the recruitment enhancement activity that is selected can be related to information supplied by the individual applying for the job. Via interface 14, an individual can submit information indicating that they possess a particular skill at a particular skill level, such as a number of years of experience in the skill. For instance, the interface can ask the person to indicate how many years of experience the person has with the skill or the person can submit a resume including this information.
[0500] Based upon information supplied by the individual, the system 12 can be configured to select a recruitment enhancement activity that in some way measures or is predictive of the person having the claimed skill. For example, an individual can be asked to take a knowledge-based test associated with their claimed skill. Harder or easier test can be implemented depending upon the person's claimed skill level. The test answers can be received by the system 12, scored and analyzed. The scores and analyses can be used to assess individuals in regards to their proficiency in the claimed skill. In addition, the individual can be ranked relative to other individuals with the same claimed skill level or according to a scale constructed for individuals of the claimed skill level. The system 12 can be configured to make information about an individual's score or rank in the recruitment enhancement activity available to recruiters.
Recruitment Enhancement System
[0501] Figure 34 illustrates in more detail a functional block diagram of a recruitment enhancement system 12 in accordance with an embodiment of the present invention. The recruitment enhancement system may reside on one or more servers with associated processors and memory, wherein the computer code is stored on a computer readable memory. A database memory may be provided to store information for the recruiting service, including candidate profile information.
[0502] In the example below, the generation of a database, which may be a proprietary database, is described. The database is focused on individuals that are software developers. The use of software developers is provided for the purposes of illustration only and databases associated with individuals in other skilled professions can also be generated and maintained at the recruitment enhancement system.
[0503] For programmers, a crawler 105 is provided to crawl code repositories. For example, the crawler may use an API for code hosting sites such as GitHub. A new candidate profile generation module 110 determines whether the crawler has identified a new developer. If so, a profile ID is generated to build a new profile. A code file type analysis module 1 15 determined the file type of files being crawled.
After the file type has been determined, the language-specific code analysis module is selected by module 120. Scoring and cheating detection is then performed by module 125.
[0504] Profiles are stored in a profile information database 130. A social media access module 135 provides access to social media information sites and a social media aggregation module 140 correlates aggregated social media information for individual profiles. In one embodiment, the social media information can include posts generated by an individual. For example, posts to their social media account, posts to a blog and/or tweets to their twitter account. The system can be configured to gather information using API's associated with various sites or via scraping information from the sites. Scraping involves extracting information from displayed web-pages. The scraping could involve optical character recognition and/or parsing of underlying mark-up language for information.
[0505] In one embodiment, the system can attempt to obtain a similar amount of information for each individual. For example, one hundred tweets or one hundred blog or status updates can be obtained for each person. As another example, the system can attempt to obtain up to one some number of words, such as one thousand words of content for each individual.
[0506] The system can be configured to regularly search for new content generated by an individual and update the content stored at the system 12. In one embodiment, when a content limit is reached, the system can be configured to remove old content when new content is added to maintain a content limit. For example, the system can be configured to store ten tweets for each individual. When the limit is reached and the system wants to add an additional tweet (e.g., the eleventh tweet), the system can be configured to remove add the most recent tweet and then remove the oldest tweet from the database. This updating method where older content is removed to make room for newer content can be applied to other types of content, such as blog posts.
[0507] The information filtering 152 can be used to identify and optionally block from output certain types of information. For example, as described above, the information filtering 152 can be used to identify protected class information previously gathered by the system 12. Then, the system can block this information from being output. In one embodiment, the information filtering 152 can be applied before information is even uploaded to the system. For example, the web crawlers or social media access modules may locate a unit of information associated with individual, such as a blog post or tweet. Next, the unit of information can be processed by the information filtering module. If the unit of information includes some content deemed unsuitable for viewing, such as protected class related information, then the system may not even upload the information. In other embodiments, the system can be configured to upload but tag the information so that it is only viewable in certain circumstances. In yet other embodiments, the information filtering 152 may not be applied during the upload process but may be applied when the information is retrieved by the system and prior to output by the system.
[0508] A messaging interface 145 is included in one embodiment as a mechanism for recruiters to contact individual developers. However it will be understood the messaging interface 145 may be omitted in some implementations. The messaging may, for example, be brokered in the sense of cloaking the user information and email address of the recruiter during initial attempts to contact a developer. A recruiting search engine and graphical user interface (GUI) module 150 is responsible for generating the graphical user interface that is provided for display on a user's computer. In one embodiment, the recruiter's search engine and GUI module can be configured to interact with a plug-in module that is provided to the recruiter's ATS (e.g., see Figure 35).
[0509] The new candidate profile generation module 1 10 utilizes author information from crawled sites to detect that there is a new developer to be added to the system. Code repository sites include author information for each project. This author information is searched by the crawler. Each individual person with a profile has a unique ID. The unique ID is created the first time an individual programmer's name is discovered in crawling author information in code hosting sites. For example, when the crawler finds the names of people that have contributed code to a code hosting site, the system compares the unique ID from the network that the person is found on to the unique IDs in the database of the recruiting service for that network. If an ID doesn't exist, a new user ID is created.
[0510] The social media access module 135 and the social media aggregation module 140 provide a comprehensive set of social media links for each profile. The author information obtained from public repository sites such as GitHub and Stack Overflow may be incomplete or contain inaccuracies. However typically the author information will include at least an email address and perhaps also a name. This information can then be used to obtain additional social media information using commercial services such as Full Contact, Inc. of Denver, Colorado, Fliptop, Inc. of San Francisco, California and Rap Leaf of San Francisco, California. Many commercial services check by unique information, like email address, or a hash of the email address (a hash is a unique number generated by an email address. That way, companies can match users by email addresses, but protect their privacy by looking at hash numbers). In one embodiment a search of social media sites is performed of all of the sites listed under Full Contact's set of Social Network Types. From this information profile information identifying the names of developers may be generated along with associated information. For example, work history may also be scraped from social networking sites.
[0511] Direct scanning of social media sites is also an option, such as the option of scanning sites such as Linkedln and Google Plus. However, there's usually not a one- to-one results process. For example, if a developer has a common name, such as "John Smith," a scan based on their name may turn up more than one hit. To find additional social media links for a particular profile it is thus desirable to look for multiple matching factors (location, title, company, name, etc.), and then calculate the probability that it's a match. If the probability is higher than a certain number, the system automatically merges the profiles. If the probability is less than that threshold, the system sends a notification that there needs to be a manual review process.
[0512] In one embodiment, an individual may have granted the recruitment enhancement system access to one of their profiles at a social media site. For example, to participate in contest, which is an example of a recruitment enhancement activity described with respect to Figure 33, the individual may have given the recruitment enhancement system permission to access their profile at Linkedln™. When permission is granted in this way, the recruitment enhancement system may be able to access more information about the individual than when social media sites are directly scanned. Once links to social media are identified for a developer they can be refreshed at a rate slower than other information in the public code repositories. Individuals typically add new social networks infrequently and the URLs of social media sites are generally static.
Recruiting Services Using a Recruitment Enhancement System
[0513] In this section, recruitment services that leverage information gathered and analyzed at the recruitment enhancement system are described. One service that can be provided is identifying and optionally filtering certain types of information. For example, the system can be configured to identify protected class information and filter it out before it is viewed by a recruiter or another person involved in a hiring decision. Interfaces that allow outside entities to access these services are also described. For instance, an interface can be provided for an applicant tracking system that allows a recruiter to access recruitment services, such as candidate scoring, from a recruitment enhancement system. As another example, an interface can be provided that allows a recruiter to upload data that can be filtered by the system, such as filtering out protected class information.
[0514] Figure 35 is a block diagram of recruitment ecosystem 200. The recruitment ecosystem 200 includes two applicant tracking systems (ATSs), 202 and 208 and a recruitment enhancement system 12 including a developer database 206. The developer database 206 includes information about software developers. This example is provided for the purposes of illustration only as the recruitment ecosystem 200 can include a plurality of different applicant tracking systems. In addition, as is described in more detail below, the recruitment enhancement system 12 can be configured to provide access interfaces that don't require an ATS. Further, database 206 can include information about individuals from other professions other than software developers and/or separate databases (not shown) can be maintained for individuals in other professions.
[0515] The applicant tracking systems, 202 and 208, can include data associated with a plurality of individuals that have applied for jobs over some period of time. In applying for a job, each individual may have supplied information that is stored to an ATS. For example, an individual may have electronic uploaded a resume in some format. After the resume is uploaded, the resume may be processed. For example, optical character recognition can be applied to a scanned in image of the resume.
[0516] After processing, information from the resume can be parsed and stored to an ATS database. The format of the database including the record structure and the information stored in the database can vary from ATS system to ATS system. For example, ATS 202 can store a first set of applicant information in a first order and ATS 208 can store a second set of applicant information in a second order.
[0517] As another example, the ATSs may also include information about individuals that have expressed interest in the company through some mechanism, such as a social media application. For example, via a social media application a user may have requested interest in a company and supplied a name and e-mail to receive some type of information. As another example, via a social media application, a user may have provided a social media profile to participate in a company related activity. Information obtained via a mechanism, such as a social media application, can be stored to the ATS. Thus, in general, there are many different scenarios in which information can be gathered about an individual and then entered into an ATS and information obtained via the job application process is only one example in which information about an individual can be provided to an ATS.
[0518] The ATSs can be associated with different entities. For example, ATS 202 can be associated with a first company where it includes information for applicants that have applied for jobs at the first company and ATS 202 can be associated with a second company where it includes information for applicants that have applied for jobs at the second company. As another example, ATS 202 can be associated with a first recruiter that has performed recruitment searches for a first group of companies and ATS 208 can be associated with a second recruiter that has performed recruitment searches for a second group of companies different from the first group. Thus, individuals in one ATS may have applied for jobs at different companies serviced by a recruiter or group of recruiters.
[0519] In some instances, information stored in each of the ATSs can overlap. For instance, when ATS 202 and ATS 208 are associated with companies in the same field, it is possible that the same individual has applied for jobs at each of the companies. Thus, ATS 202 and ATS 208 can include information, such as resume information associated with the same individual. However, because companies and recruiters usually closely guard the information stored in their ATS and don't generally share the information, neither of the entities controlling ATS 202 or ATS 208 is likely to know about the overlaps between their ATS databases.
[0520] Using an ATS database, such as 202 and 208, a recruiter can perform searches over the records in the ATS database. A search can allow a recruiter to identify a group of records that satisfy particular search criteria. For example, if the database stores information related to an individual's experience level with a particular programming language, then a recruiter can perform a search for candidates satisfying the experience level requirements for a particular job. The outcome of the search can be a list of candidates meeting the search criteria and information about each candidate.
[0521] After searching, a recruiter can choose to contact various individuals for additional scrutiny, such as interviews. The recruiter may continue to contact individuals represented in the ATS until a job position is filled. After the job position is filled, the information about individuals that were not hired may still remain in the ATS. Thus, search results can include information about job applicants that have applied for a particular job opening, information about individuals that have previously applied for other jobs and individuals that have been entered into the ATS through other mechanisms, such as via a social media application.
[0522] In particular embodiments, information about all or a portion of the candidates in an ATS can be sent to the recruitment enhancement system 12. As an example, in 210, information about one or more individuals stored in ATS 202 can be sent to recruitment enhancement system 12. As another example, in 214, information about one or more individuals stored in ATS 208 can be sent to the recruitment enhancement system 12.
[0523] The information sent from the ATS to the recruitment enhancement system 12 can be used to provide a recruitment enhancement service. In various embodiments, as will be described in more detail below, one recruitment enhancement service can involve scoring or ranking individuals, such as software developers, which are identified from the information received from ATS in a recruitment service request. Another recruitment enhancement service can be to provide secondary recruitment data about an individual, such as information retrieved from publically available sources. Some examples of information that can be used to identify an individual that may be stored in an ATS and sent to system 12 are described with respect to Figure 36A. Some examples of information that can be returned about an individual from system 12, such as but not limited to a score or a rank are described with respect to Figure 36B.
[0524] Next, some of the information that can be sent to system 12 as part of a recruitment service request is described with respect to Figure 36A. Then, additional details about Figure 35 are described followed by a description of Figure 36B. The individual in Figure 36A can be one among group of individuals whose information is sent in 210 or 214. In a communication between an ATS and system 12, identification information about each individual, such as a first name and a last name 302, can be sent. In addition, one or more e-mail addresses, 304 and 306 can be sent that are believed to be associated with the identified individual. In one embodiment, the recruitment enhancement system requires a first name, last name and at least one e- mail address to provide a recruitment service involving assessing a skill level of the individual.
[0525] For competitive purposes and for privacy reasons, a company may not wish to release too much information about an individual. An individual name and e-mail address or addresses are likely to be publically available. Thus, an advantage of sending only a name and email addresses is it allows the company to maintain that it is protecting the privacy of individuals because only a limited amount of information is sent and what is sent is likely to be publically available.
[0526] Besides the name and the e-mail address or addresses, one or more additional types of information can be sent. The recruitment service may involve determining whether the recruitment enhancement system 12 has additional information about the named individual. Individuals can share names. Thus, the additional information may allow the system to confirm with a higher level of confidence that a named individual from an outside entity is properly identified by system 12.
[0527] Some example of additional information include but are not limited to location information 308, blog information 310, social profile information 312, nicknames 314, user names 316 and profile names. The location information 308 may be an address where the person lives or works. In one embodiment, the location information 308 may be a partial address, such as a city where the individual lives or works. The blog information 310 is example of information associated with an individual's online activities. For example, information 310 can be associated with a user's online blog, such as link (e.g., a URL) to the blog.
[0528] In another example, information 312 may be associated with an individual's on-line profile. For example, a link to a Linkedln™ profile or information obtained from the on-line profile can be sent. Multiple instances of information 312 can be sent. For instance, a first link to a first profile and a second link to a second profile can be sent. The system 12 can be configured to parse the received information when multiple instances of particular type of information or no instances of a particular type of information, such as 312, are received. In other examples, information, such as nicknames 314, usernames 316 or profile names 318, associated with a user's on-line activities can be provided to system 12. As will be described in more detail as follows, the received information can be used to determine whether the named individual also has a record in the developer database 206 maintained by the recruitment enhancement system 12. In general, any information available at the ATS that can help to locate information about the individual in the developer database 206 or other database maintained at the recruitment enhancement system can be sent.
[0529] The information that is sent can vary from individual to individual. For example, for a first individual, a name and one e-mail address can be sent. For a second individual, a name and two e-mail addresses can be sent. For a third individual, a name, one e-mail address and a link to a social profile. For a fourth individual, a name, two e-mail addresses and a number of different user names can be sent. Thus, the recruitment enhancement system 12 can be configured parse and utilize different combinations of information that vary from individual to individual.
[0530] In a particular embodiment, the recruitment system can be configured to receive information about an unnamed individual. For example, the first name and last name 302 may not be known. Based upon the received information, the system 12 can be configured to determine a possible name of the individual and return the information to the entity that sent the information about the unnamed individual.
[0531] The information described above doesn't necessarily have be stored in or sent from an ATS. In one embodiment, the received information is used to locate information about individuals in a database maintained by the recruitment enhancement system 12. For this purpose, it doesn't matter where the information resides before it sent as long as it is suitable for locating records of individuals maintained in the database or databases of system 12. Thus, the example of an ATS as a source of the information sent to the recruitment enhancement system is for the purposes of illustration only and is not meant to be limiting.
[0532] In one embodiment, besides information about the individuals, information about a job position for which candidates are desired can be sent. For example, a portion of a job description can be sent in a recruitment service request. In one embodiment, when the system 12 receives a job description and a list of a number of individuals, the system 12 can be configured to store information about the individual and information about the job position for which they are being considered to a database maintained at system 12.
[0533] Returning to Figure 35, in one embodiment, a plug-in module, such as 204 and 206, can be provided for an ATS. The plug-in module can be configured to be compatible with the recruitment enhancement system 12. The plug-in modules can be configured to extract information from an associated ATS and send it to the recruitment enhancement system in format that is recognizable by system 12.
[0534] In one embodiment, the plug-in module may provide an export feature. For instance, an export button can be located within an interface provided with the search functions of the ATS. In response to a search, a selection of the export feature can cause an electronic data file to be generated that is compatible with the recruitment enhancement system 12. The electronic data file can include information consistent with the search, such as the names and e-mail addresses of the individuals identified in the search. As described above with respect to Figure 36A, additional information related to a particular individual can also be included.
[0535] An API (Application Program Interface) can be provided with system 12. The API can describe formats and particular data types that system 12 can accept. The API can be used by third-party developers to construct a plug-in module for an ATS.
[0536] The extracted information can be sent to the recruitment enhancement system in 210 or 214. The information can be sent as part of a recruitment service request. The plug-in module may require a user to provide an additional input before the data is sent, such as a selection of a send button. In one embodiment, the plug- in module, such as 204 or 206, can be configured to establish electronic communications with the recruitment enhancement system 12 and then send data to the recruitment enhancement system 12 that has been extracted from an ATS. The data can be sent electronically via a secure connection between the ATS and the recruitment enhancement system 12. The information that allows the secure connection to be established may have already been exchanged by the systems.
[0537] In one embodiment, entities can be charged according to their utilization of the recruitment enhancement system 12. Thus, the data sent from the ATS, such as 202 or 208, can include identification and authorization information for an entity requesting a service. The authorization information can be used to determine whether the entity sending the data is authorized to obtain a requested service from the recruitment enhancement system 12. The identification information may allow the recruitment enhancement system to attribute its utilization to the entity and bill it for the services.
[0538] Although not shown, the recruitment enhancement system 12 can be configured to provide an interface, such as web-based interface, that allows an entity to upload information that is compatible for receiving a service from system 12, such as a file including a list of names and e-mail addresses to the system. The system 12 can be configured to process the file and generate a report. Upon payment, entity can receive the report. The report can be emailed to an address specified by the entity or a link can be provided that allows the entity to download the report. Thus, a plug- in module that provides an interface to an ATS may not be required.
[0539] After receiving the individual information in 210 and 214, the system 12 is configured to determine whether it has information about the received individual's in its developer database 206. Possible results of this determination are shown graphically in Figure 35. The largest circle represents individuals in the developer database 206. Group 218 can include one or more individuals from ATS 202. It can be seen that a circle associated with group 218 partially lies within the large circle associated with 206 and partially outside of it. The portion inside the circle represents individuals in group 218 that can be matched to individuals in the developer database 216. The portion outside the circle 206 represents the portion of individuals that can't be matched to individuals within the developer database 206 with an acceptable degree of confidence.
[0540] In this example, most of the individuals in group 218 were matched to individuals described in database 206. In other embodiments, it is possible no matches may be found. In yet other embodiments, it is possible that all of the individuals in a group, such as 218, can be matched to individuals in the database 206. Thus, the fractions shown in Figure 35 are for the purposes of illustration only and are not meant to be limiting.
[0541] The system 12 can maintain a number of databases that identify individuals. The individuals can have different job skills and may be members of different professions. At some point, the system 12 may have received information that allows a score or a rank to be developed for the individual. In various embodiments, the scores or ranks can be an indicator of such factors, as the quality of their one or more job skills, their influence within their profession and their effectiveness at their profession.
[0542] When reporting the results of a search, the system 12 can report which individuals received in a group, such as 218, were matched to an individual in at least one database maintained at 12 and which individuals were not matched. For named individuals in a group that are matched to someone in the developer database 206 a confidence measure can be determined. The confidence measure may be based upon the amount of information that is matched. For example, a number of points can be given for a last name match, a number of points can be given for a first and last name match and a number of points can be given for each e-mail address match. The more information that is matched the higher score and hence the higher level of confidence that the named individual is a match for an individual in the developer database 206.
[0543] In one embodiment, when an individual is not matched or the confidence measure is determined to below a certain threshold, the system 12 can be configured to initiate a search and attempt to locate information about a named individual in a group that allows them to be scored or ranked or raises the confidence measure of the match to an individual in the database 206. A named individual newly scored or ranked can be added to the database 206 and their score/rank can be reported in reply to the service request, such as in 212 or 216. When a confidence measure of a match has been increased as a result of search such that it exceeds threshold value for reporting the result, then the result can be reported in a reply to the recruitment service request.
[0544] In one embodiment, when a named individual can't be located in the database 206 and information about the named individual can't be found which allows the individual to be scored or ranked, the system 12 can still be configured to keep a record of the individual. The system may keep the information so that future searches can be carried out in which attempts are made to gather information on the named individual where the gathered information can be used to develop a score or rank for the individual. Further, if the named individual shows up again as a result of a request from another entity to score the individual, then the system 12 can report that an individual may be actively looking for a job based upon the fact that the system 12 has received information about the individual from multiple entities. This information can be provided even if a score or rank is not available for the named individual.
[0545] In one embodiment, when a match is not found for a named individual in a service request and a record of the unmatched named individual is stored, the system can be configured to attempt to match the named individual at a later time. It is possible a match may be subsequently found and a score or ranking may become available for the named individual that was not previously matched. The system 12 can be configured to report the score or ranking to one or more entities that requested a score or ranking for the named individual that originally were notified that a score or ranking was not available for the individual or the confidence level of the match was below a threshold value when a score or ranking becomes subsequently available. This feature may be a system option in that a service requester can specify whether they wish to be updated or not if additional information is subsequently discovered about a named individual, such as whether a score or ranking has been developed for the name individual that was previously not available.
[0546] In general, the system can be configured to allow the recruitment service requester to specify whether or not they want updates regarding named individuals that have been previously submitted in a service request, such as 210 or 214. For example, the system can be configured to maintain a watch list for an entity including a number of named individuals in a group. In response to specified triggers, the system 12 can be configured to notify the entity. For instance, the system can notify an entity, such as recruiter, when it is determined a named individual on the watch list appears to be searching for a job. In another example, the system can be configured to notify an entity, when a score or ranking of an individual on the watch list has changed in some manner, such as exceeding some threshold value or changing by some percent amount.
[0547] The notifications from system 12 can occur through one or more different communications channels. For example, an e-mail can be sent to a recruiter. In another example, a text message can be sent to a recruiter's portable electronic device. In yet another example, an e-mail and text message can both be sent.
[0548] The system 12 can be configured to receive in a request for a recruitment service subsets of named individuals grouped in some manner. The named groups may be used for comparison purposes. These different subsets are represented by the circles in Figure 35 that are smaller than the largest circle. As described above, the largest circle can represent individuals in the developer database 206 and portions of the smaller circles lying outside of the largest circle represent named individuals in a group that have not been matched to an individual in the developer database 206.
[0549] In one example, the recruitment service request 210 for ranking/scoring of individuals can include groups 218, 220, 230 and 232. The named individuals in group 218 are possible job candidates. These candidates may have applied for a job over a first time period, such as in the last year. Group 220 may be individuals that are distinct from other members of group 218 in some manner, such as having applied for a job in the last month or having applied for a job multiple times. Group 230 can include individuals that have been offered a job. Group 232 can include individuals that have been hired.
[0550] In another example, the recruiting service request 214 can include three specified groups, 224, 226 and 228. Group 224 can be a group of individuals that have applied for jobs in between 6 months to a year ago. Group 226 can be individuals that have applied more recently for a job or jobs, such as within the last month. Group 228 can represent a current employees of a company that perform a job or jobs that are similar to the job or jobs for which the named individuals in group 228 are applying.
[0551] In one embodiment, scores/ranks can be provided for each individual in a group as well as for the group as whole (e.g., an average score for the named individuals in a group can be provided). The scores/ranks can be used for various comparison purposes. For example, the score of a named individual in the recent 220 group can be compared to the average score or rank of the hired individuals in group 232 or the average score or rank of the individuals that were offered jobs in group 230. Similar comparisons can also be made between individuals in groups 224, 226 and 228. For example, scores/ranks of a named individual in group 226 that has recently applied for a job can be compared to scores/ranks of individuals in group 228 or an average of scores/ranks for individuals in group 228. In another example, the average rank/scores of the individuals in group 224 that have applied 6 to 12 months ago can be compared to another group, such as individuals in group 220 that have applied for a job in the last month.
[0552] In one embodiment, this comparison feature can be used as a filter. For example, the system 12 can be configured to only return information on named individuals in a group whose score or rank is above the average of the named individuals of a comparison group. As another example, the system 12 can be configured to only return information on named individuals in a group whose score/rank is above some threshold value, such as the average score/rank of the named individuals in a group.
[0553] In the examples in Figure 35, some of the named individuals that are submitted in different recruitment service requests can overlap. For instance, some of the named individuals in group 218 can also be named individuals in group 224. In Figure 35, the overlap of members for groups 218 and 224 is represented by the intersections of the circle 222. In example in Figure 35, the entity associated 202 and the entity associated with 208 may not be aware of their individual recruitment service requests and the fact one or more individuals are in each request as different entities, such as companies don't share this information. However, when a named individual is identified from multiple recruitment service requests, the system 12 can be configured to keep track of and leverage this information.
[0554] As an example, the system can be configured to record each time an individual appears in a recruitment service request. If a named individual appears multiple times over some period, it can indicate the candidate is looking for a job. If the named individual is highly ranked, this factor can be used as a trigger to notify a recruiter. For instance, as described above, a recruiter can place an individual on a watch list and request to be notified under some conditions. One of the conditions can be if the individual appears to be looking for a job which may be determined from their appearance in multiple recruitment service requests. The system 12 can be configured to allow a user to specify parameters that determine under what conditions they are to be notified.
[0555] In a recruitment service request, such as 210 or 214, a large number of names can be submitted. For example, tens of thousands of names can be submitted. The processing of a large group may take a significant amount of time. In one
embodiment, the system 12 can be configured to provide results of a recruitment service request all at once. For example, the system 12 can be configured to process all of the named individuals in a recruitment service request and then return the results in a batch mode when request is completed. In another embodiment, the system 12 can be configured to send a portion of the results in real-time before the request is completed. For example, when a match is made for a named individual in a recruitment service request, the results can be returned as soon as the match has been made. As another example, when a match is made for a named individual meeting certain criteria (e.g., a ranking or score about some specified value), the results can be returned as soon as the match is made. The remaining results including or not including individual results previously sent can be returned in a batch mode when the matching process for the recruitment service request is completed. [0556] The format of the returned results can vary. In one embodiment, a selectable link can be provided that allows additional information to be learned about an individual listed in a result. This information can include information that has been gathered by system 12. In one embodiment, a recruiter can provide filtering requirements that affect the additional information that is returned from the system. For instance, the recruiter can specify in accordance with a company policy that protected class information or images associated with identified individuals are not to be returned. In response, the system 12 can filter out this information from the additional information it has on particular individuals before it is returned to the recruiter. Additional details of this process are described below with respect to Figures 37-40B.
[0557] In one embodiment, the plug-in, such as 204 or 206, can be configured to automatically submit recruitment service requests to the system 12. For example, each time an individual applies for a job and their information is entered into the ATS 202, plug-in 204 can be configured to submit a recruitment service request to system 12. When a match is found, the system 12 can return information, such as a score or ranking, about the individual and additional information that is filtered according to parameters specified by the recruiter. The returned information may be stored to the ATS. In another example, the recruitment service request can be triggered after some number of applicants have applied, such as after receiving 10 applications. Then, the plug-in module can send a recruitment service request including ten names to system 12. In another example, the recruitment service request from the ATS can be triggered at some time interval, such as once a day.
[0558] In the example above, one or more named individuals are described as being sent in a recruitment service request to system 12, the system 12 tries to match the named individual to an individual stored in a database, such as database 206, and then information, such as scores for matched individuals can be returned. The use of the service is described with respect to recruiting individual candidates for jobs.
However, the use of the system is not limited to recruiting individual candidates for jobs.
[0559] For example, in one embodiment, the system 12 can be used to rank or score the strength of a team, such as a software development team. An employer may have a number of software development teams for different products. The individuals on each team can be submitted to system 12 and the development team can be scored. Based upon the scores, a manager may decide to reformulate the groups. For instance, some development teams have much higher scores than other development teams and a manager may reshuffle the team members such that the scores for the teams are more balanced.
[0560] In another example, a company may wish to hire a team of developers for a project. The company may receive bids from different contractors. The company can request the names of individuals that are going to be working the project for each contractor and submit their names to the recruitment enhancement system 12. The system can return scores for each of the development teams for each contractor. Based upon the scores/rankings provided for each of the team members as well as the scores as a group, the company can decide which contractor to hire for the project.
[0561] In yet another example, a company, such as a software contractor, may have their software developers ranked/scored by system 12. The system 12 can generate individual scores. The individual scores can be used to develop company scores.
Then, the company may use individual scores and/or the company scores to promote the technical expertise of their employees.
[0562] As described above in response to a recruitment service request including a number of named individuals, such as 210 or 214, results which may include scores and/or ranks can be returned to the requester in 212 and or 216. An example of what information can be returned is described as follows with respect to Figure 36B. Other types of information can be returned and the example provided in Figure 36B is for the purposes of illustration only.
[0563] In one embodiment, the individual output data 350 can include some set of baseline information and optional information. As an example, the baseline information can include a link to a profile 352 or the information contained in a profile. The profile 352 can be a profile collated from various information sources to which system 12 has access, such as various on-line sources. The baseline information can include one or more scores, such as 354 and 356, skills possessed by the individual and an indicator 360 of the confidence measure of the match. The confidence measure can be indicated as a score, such as a numerical score, or according to some scale, such as high, medium or low. [0564] Returning to Figure 36B, some examples of optional information include one or more social profile links 362, additional contact information 364 and a current job status 366. The additional candidate information can be information that the system has obtained from publically available source. For example, the additional candidate information 368 may consist of blog posts, tweets and status updates that the candidate has posted to the web over time and the system has gathered via crawling or scraping of various web-sites. When the additional candidate information is selected, it can be presented to the user. As described below, the information can be filtered before it is selected. The formulation of the baseline information is provided for illustrative purposes only and different combinations of baseline information and optional information can be provided
[0565] In one embodiment, the system 12 can be configured to receive parameters that allow a user to customize the information and the format of the information that is received in 350. The customization may be implemented on a request by request basis, implemented globally as part of the plug-in module or implemented on a user by user basis For example, a first user can specify one set of preferences and a second user may specify a second set of preferences for receiving information 350 where the preferences can affect the type of information that is received and its format. One example of customization can be only returning information that can be used to make legal hiring decisions. Information that can be used to make an illegal decision, such as gender, health status or age related information, may be blocked such that the information is not output via the system.
[0566] In one embodiment, for a large job including information about many individuals, the system 12 can electronically send a report or a link to location that allows the report to be downloaded when the report is ready. In addition, the plug-in modules, such as 204 and 206, can be configured to add all or a portion of the information in the report to an ATS. For example, scoring results can be integrated into the ATS such that future searches involving the ATS can return scoring information without having to contact the recruitment enhancement system 12. The plug-in module can be configured to periodically communicate with the recruitment enhancement system to update the scores or ranks stored to the ATS. Additional details of a recruitment enhancement system are described in U.S. applications nos. 13/557,812, titled, "Method and Apparatus for Enhancing Job Recruiting," and 13/568,493, titled, "Method and Apparatus for Electronic Job Recruiting," each of which are incorporated by reference and for all purposes.
Gathering and Filtering Aggregated Individual Information
[0567] As described above, information from a number of publically available sources, such as social media sites or professional sites, can be gathered, aggregated and analyzed for various individuals. The system can provide filters for the data that can applied at different stages in the recruitment process, such as before storing data to the system or prior to viewing previously stored data. The filters can be used to identify and return certain types of information and/or prevent certain types of information from being returned. The filters that are applied can depend on the context in which the information is to be used, such as according to a stage in the recruitment process. Some examples of how information filters may vary depending on the stage in the recruitment process or post recruitment process are described in the following paragraph.
[0568] In many recruitment scenarios, a candidate is a prospect in that the person has not formally applied for a job. The recruiter in such a scenario is thus searching for individuals to contact. Under the labor laws of many countries, the regulations for the type of information used to identify a prospect are generally less stringent than using acquired information after an individual has applied for a job. Thus, no filtering or minimal filtering may be required in this context. However, stricter standards may apply in some countries regarding the types of information that may be used to evaluate current employees. Thus, to comply with these regulations more stringent filtering may be used. Further, during the hiring a process, a different set of regulations may apply which may require different filtering. Moreover, individual countries have different laws regarding how self-authored writings of an individual may be used in different employment scenarios. Additionally, there have historically been certain professions in which a condition of employment included the expectation that a very wide range of personal information could be collected and evaluated (e.g., psychological testing for firefighters). Thus, depending on the context, it may be appropriate to present more or less aggregated individual information to a user of the recruitment enhance system where the amount of information that is provided depends on the amount of filtering that is used. [0569] As will be discussed in more detail with respect to Figures 37, 38, 39, 40A and 40B, apparatus and methods are described for gathering information on individuals and then filtering the information prior to output to account for the context in which the information is being used. In a particular embodiment, when the aggregated individual information is being used for making decisions that may affect whether an individual is hired or not, the information can be filtered for "protected class information." The filtering of the protected class information can prevent inferences from being made that the information is being used in a discriminatory manner, i.e., someone is not being hired as a result of being a member of a protected class. For example, if the individual making the hiring doesn't know anything about an individual's protected classes, then the individual can't be accused of making an illegal hiring decision based upon this information.
[0570] Referring to Figure 37, in one embodiment a portion of the recruitment enhancement system 12 (or other computer system including a processor and a non- transitory computer readable memory storing instructions) can use a spider to crawl links to content authored by the candidate (e.g., social media) in step 405. The Internet is crawled with a spider and content is downloaded to a database (DB) and associated with an individual user in step 410. As examples, the content may include text extracted from social media, such as blogs and Twitter® posts. Text equivalents of video or audio posts may also be generated. Additionally, the content may include articles available on the Internet written by the individual. Examples of sites that can be crawled include but are not limited to Facebook™, Myspace™, Google+™, Twitter™, Pinterest™, Linkedln™, Github™, etc
[0571] The collection of content may be performed in view of other contextual information, such as crawling the Internet searching for individuals having certain educational qualifications, job-related qualifications, or business opportunity qualifications to form an initial pool of potential candidates from which additional information for specific individuals is desired. Alternately, the collection of data may be performed for a specific individual whose identity is already known. For example, in response to request from a recruiter that wishes to learn additional information about an individual. In one embodiment, the crawling can be performed on a regular basis and used to update a database that is maintained by the system, such as daily, weekly monthly, etc. In another embodiment, all or portion of the information can be gathered on demand to ensure that the information that is being gathered is up to date.
[0572] The analysis of content preferably uses filtering techniques to verify authorship, such as filtering out re-posts on social media (e.g., filtering out "retweets" of Twitter® postings). Additionally, the filtering may include verification tests to verify that the content is associated with a single individual and not misattributed from other individuals having similar names. The analysis of the content may be performed over a selected time range and trend analysis may also be performed if there is sufficient historical data.
[0573] In one embodiment, a natural language module may be used in step 415 to process the text. The text can be manipulated using natural language processing, such as determining a frequency of detected keywords and correlations and semantic associations between words. The semantic association between words can be used to filter certain types of information, such as protected class information. For instance, a descriptions related to a person's medical status may be removed based upon identifying key words and semantic associations. An exemplary tool for performing natural language processing of text is the Natural Language Toolkit (NLTK) from the NLTK Group for use with Python® programming. NLTK provides a suite of processing libraries for performing computational linguistics for linguistic classification, tokenization, stemming, tagging, parsing, and semantic reasoning.
[0574] In another embodiment, a Machine Learning (ML) module can be used in step 420 to classify the text associated with a particular individual and obtain additional information to assess the potential interest and potential fit of an individual for a job. ML is branch of artificial intelligence which includes algorithms enabling software to learn. ML is used to classify content associated with a particular individual. ML programs perform classification based on training to find a pattern based on the training. ML typically generates results in which there is certain probability of a particular classification being correct. If the result is over a specific threshold, a particular classification is assigned. As an example, a classification can be provided as to whether the information is protected class information or not.
[0575] After the information is gathered and aggregated for one or more individuals, the aggregated individual information can be filtered before it is output. For example, the information can be filtered to remove protected class information. In one embodiment, the filtering can be performed before the data is uploaded to the system so that certain types of information may not be uploaded to the system. An example of outputting the information is described above with respect to Figure 36B. A method 500 of performing the filtering is described as follows with respect to Figures 38 and 39.
[0576] In 502, the system can attempt to make a determination of the context in which the aggregated individual information is to be utilized. For example, the system can output to a display a question to the user, such as "How is the information to be used," followed by a number of selectable options. The selectable options can be indicative of the stage of the recruiting process in which the information is to be used. Examples of the selectable options can be "Outside of the hiring process for a particular job-general networking," "Within the hiring process for a particular job- initial candidate search and screening," "Within the hiring process for a particular job- pre-candidate interview," or "Within the hiring process for a particular job-post interview."
[0577] A selection of these options can affect filtering settings that are to be applied to the aggregated individual information that has been gathered. For example, if the selection is outside the hiring process for a particular job, the default filtering setting may be "no filtering." For "within the hiring process for a particular job-initial candidate search," the default filtering setting may be federal protected class information for the United States. Federal protected class information is one of information associated with race, color, national origin, religion sex (including pregnancy, childbirth, and related medical conditions), disability, age (40 and older), citizenship status and genetic information.
[0578] After filtering is applied, an interface can be provided that allows a user to view and adjust the filtered information. The interface can display the results of the filtering process performed by the system. If the filter doesn't completely remove all of the protected class information, using the interface, a user may able to remove and then save the manually filtered information associated with a particular individual for later viewing, such as by a hiring manager.
[0579] When the "within the hiring process for a particular job-pre-candidate interview" option is selected, the system can be configured to return the filtered aggregate individual information generated during the pre-screening process described in the previous paragraph. Some companies may require that a person different from the individual with hiring power perform the initial candidate search and screening for a particular job. In one embodiment, when the initial candidate search and screening has not been performed and this option is selected, the system can be configured to not return any data and notify the person that pre-screening needs to be performed. This configuration may prevent a hiring manager from seeing information that has not been approved for viewing by the hiring manager at particular stage in the recruiting process, such as protected class information prior to determining whether to interview a candidate or not.
[0580] Finally, when the "within the hiring process-post interview" option is selected, in one embodiment, less filtering may be used allowing some or a portion of the protected class information to be returned. This option may be allowed because during the interview the interviewer may have learned protected class information, such as gender and race, just by seeing the person. Thus, it may no longer necessary to filter out this data. Therefore, the system can be configured to return less filtered information on an individual at the post-interview stage. In one embodiment, this option can be company specific, i.e., one company may allow more data to be seen after the interview whereas some other companies may still only permit a user to see information filtered for the protected class information after the interview.
[0581] In 504, the system can receive location information. The location information can be used to adjust filter settings. For example, the location information can be country, such as the USA or a region, such as Europe, where the employment is to take place. Different countries can have different requirements in regards to what information can be considered in the hiring process as well as privacy protections for their citizens. Thus, based upon the location setting some default filtering settings pertinent to the location can be retrieved.
[0582] In one embodiment, when the USA is specified, the system may allow a user to select a state. Many states have protected classes that are beyond the federally mandated protected classes. For example, California in addition to the federal protected classes includes protections based upon marital status, sexual orientation and identity, AIDS/HIV status, medical conditions and political activities or affiliations. In Minnesota, marital status, sexual orientation, creed, status with regard to public assistance, membership or activity in a local human rights commission, are additional protected classes. In addition in Minnesota, it is generally unlawful to refuse to hire an individual because of his or her use of lawful consumable products during nonworking hours off the employer's premises. Thus, as an example, for a potential Minnesota hire, the system can be configured to filter out any information related to drinking or smoking.
[0583] In Wisconsin, marital status, ancestry, sexual orientation, arrest record, conviction record, military service, genetic testing, the use or nonuse of lawful products off the employer's premises, or declining to attend a meeting or participate in any communication about religious or political matters are protected. Also, arrest and conviction records may be considered only in certain limited circumstances. For North Dakota, presence of any mental or physical disability, status with regard to marriage or public assistance, or participation in lawful activity off the employer's premises during nonworking hours that is not in direct conflict with the essential business-related interests of the employer may be protected. In South Dakota, marriage or public assistance, or participation in lawful activity off the employer's premises during nonworking hours is protected. Finally, in Iowa, sexual orientation and gender identity. These examples are provided for illustrative and are not meant to be limiting to only the states listed above as other states not listed also can have protected classes beyond what is federally mandated.
[0584] As mentioned above, different countries can have different rules. For example, in Canada, human rights laws across prohibit employers from discriminating against individuals in hiring, firing, or the terms and conditions of employment because of certain personal characteristics (unless it is for a valid job requirement). With some exceptions, workers in Canada are protected from discrimination based on: 1) national or ethnic origin, race, ancestry, place of origin, color; 2) disability (physical and/or mental), 3) religion, creed, political belief, association, 4) sex, sexual orientation, pregnancy, 5) age (with exceptions for minors and seniors in some cases) and 6) marital or family status. Canadian labor laws prohibit discrimination against any person for union activity or because of union membership. You can't be treated unfairly or differently because of your association with a union. The provinces, the territories, and the federal government all have slightly different laws. Some jurisdictions protect workers from discrimination on additional grounds, such as language, social status, or previous convictions for which a pardon has been granted. Thus, the system can be configured to apply certain filtering settings when Canada and/or a specific province of Canada are entered into the system.
[0585] Another factor (not shown in Figure 38) which can affect the filtering of the individual data is the number of employees in a company. Certain laws only apply to companies of a certain size. For example, under federal law, companies with 15 or more employees are covered by Title VII, the primary law prohibiting employment discrimination, the Americans with Disabilities Act, which prohibits discrimination on the basis of disability, and the Genetic Information Nondiscrimination Act, which prohibits discrimination based on genetic information. Companies with 20 or more employees are subject to the Age Discrimination in Employment Act (ADEA), the federal law that prohibits discrimination against employees 40 years or older.
Companies with four or more employees must comply with the employment discrimination provisions of the Immigration Reform and Control Act, which prohibits discrimination on the basis of citizenship status. Thus, in one embodiment, the number of employees of a company can be an input.
[0586] Based upon the input of the number of employees, the system may set default filtering settings and/or indicate which laws are applicable. For example, if the input is 10 employees. The system may set filters that remove only citizenship status information. As another example, for 17 employees, the system set filters that remove information associated with Title VII and citizenship status but not age related information that comes into effect at twenty employees.
[0587] Besides protected class information, there are certain rules related to financial status that may be enforced. For example, when a person has been involved in a bankruptcy, it may be desirable not to use this information in a hiring decision. Thus, in particular embodiments, the system can be configured to remove financial related information, such as whether they have gone bankrupt before or not.
[0588] In 506, the system can receive user information. In one embodiment, the user information can be used to determine one or more filtering settings. For example, in one embodiment, the user may have saved some combination of filtering settings and the saved filtering settings can be recalled when the user information is provided. In another example, the user may have saved a number of different filtering settings. When the user information is provided, the system can display a list of their saved filtering settings. Then, a user can select one of the saved filtering settings to utilize. [0589] In yet another example, the user information may link the person to a particular company. The particular company may have a default filtering setting associated with a company policy for filtering. When the user information is received and it the company is determined a default company policy for filtering can be located and applied. As an example, a multistate company may have a filtering policy that is consistent with a state in which they are located that has the most protected class requirements. As another example, a company, as a matter of policy, may choose to recognize more protected classes than are required United States federal regulations.
[0590] In another example, via the user information, a user can be granted more less privileges with the system. For example, a first person can be granted the privilege to view unfiltered information about candidates, such as a person in charge or pre- screening but not hiring. However, a second person, such as a person expected to make hiring decisions may be allowed to view only filtered information according to some privilege setting that is determined from the user information that they have provided to the system.
[0591] In 508, based upon the information received in 502, 504 and 506, the system can determine a number of default filter settings. In 510, the system can be configured to display the default filter settings that are to be utilized. In 512, the system can be configured to receive custom setting. For example, a user may decide to remove one of the default settings determined in 508. In another example, a user may decide to add additional setting beyond the default settings determined in 510. In one embodiment, the interface may allow a user to select filtering setting to add to the default settings or remove from the default settings that have been determined by the system. For example, a default setting may be to filter image data and the user can remove this filter setting to allow image data to be viewed or a default setting may be to allow viewing of image data and the setting can be changed to no image data.
[0592] Filtering image data can involve determining whether an image is safe or unsafe for use in a hiring decision. For example, an image of a person applying for a job may be considered unsafe to use in a hiring decision and the system can be configured to identify images including the person applying or people and prevent such images from being output. However, an image of project a person completed may be considered safe to use in a hiring decision. Thus, the system may allow such an image to be output. Thus, in general, the system may be configured to discern whether certain images are safe or unsafe to use in a hiring decision and hence whether the images are to be output or not output.
[0593] In one embodiment, the system may include a number of examples of individual profiles that a user can view unfiltered and then again with one or more combinations of filtering settings applied. The interface can be configured to display the individual profiles and the associated filtering settings that have been used to generate the profiles in a side by side manner to allow a user to see the effects when different filtering settings are applied. For example, the system can be display an unfiltered profile in a side by side manner with a profile generated from the default filtering settings determined by the system in 510.
[0594] In 514, the system can receive a confirmation of the setting and then store of record of the settings if desired. As described above, the user may store a combination of filtering setting which they may later wish to re-use. Next, the system can apply the filtering settings that have been selected. For example, a user can provide a list of names of a number of different individuals and other types of identification information, such as e-mail addresses, associated with each individual. Next, the system can attempt to retrieve from a database or search the web for individuals satisfying the received identification information as described above with respect to Figure 37.
[0595] After gathering the information on the individual, the system can filter the information according to the specified filtering settings confirmed in 514. For example, all information associated with age can be removed, such as a birth date, a date when they graduated high school or an image of the person. As another example, all references to marital status or references to health conditions and overall health can be removed. In yet another example, all references that might indicate gender can be removed. For instance, if the person indicated they were a member of women's hockey team, the system might return that the individual was a member of a hockey team.
[0596] In 516, the system can present aggregated individual information that has been filtered. The information can be presented for one or more individuals. In one embodiment, the system may have an interface that allows a user to remove additional information about a particular individual. For example, if the system returned information about a person drinking in a legal setting and it was not caught by the filtering, the system might allow the user to remove this information for a displayed profile. A person prescreening profiles for a hiring manager can perform this task before the hiring manager is allowed to see the filtered profiles.
[0597] In one embodiment, when a person manually removes information, the system can be configured to determine whether other profiles of interest to the user have similar information. If other profiles have this type of information, the system can query a user to ask whether they would like the system to remove the similar information in the other profiles. When the system receives an approval, it can then apply the manual changes performed by the person for one profile to the other profiles.
[0598] In 518, the system can be configured to store the filtered information. For example, a user performing pre-screening for a hiring manager may request the system to store the filtered profiles for later review by the hiring manager. Optionally, the system can also store the unfiltered data to allow someone in a later stage in the hiring process, such as after a candidate has interviewed.
[0599] In one embodiment, the types of unfiltered data that are saved can be specified. For instance, a user can specify that the system save an image of the user that can be later viewed after interview. However, the user may specify that no information related to citizenship status of the individual be saved. Thus, a person later reviewing the saved profiles can be kept from seeing this additional data about the individual in their profile.
[0600] With respect to Figure 39, additional details of a method 600 for gathering content associated with different individual and identifying when all or a portion of the gathered content provides information undesirable for use in a hiring decision. As described above, whether the information is undesirable for use in a hiring decision may depend on the stage of the hiring process in which it is to be utilized. Thus, in some embodiments, different filters can be applied at different stages in the hiring process. In other embodiments, the same level of filtering can be applied throughout the hiring process.
[0601] In step 602, content can be gathered that can be used to a train a filter for identifying certain types of content. In one embodiment, the filter can be Bayesian type filter. A Bayesian type filter uses a statistical technique for identifying one or more particular types of content. It can make use of a naive Bayes classifier to identify particular types of content. Bayesian classifiers work by correlating the use of tokens (typically words, or sometimes other things), with "a particular type of content" and "not the type of content" and then using Bayesian inference to calculate a probability that a unit of content includes or does not include the type of content. As an example, the Bayesian classifier can be developed that identifies whether a unit of content includes or doesn't include information associated with one or more protected classes. This type of analysis can be applied to text, audio files or video files where filters can be applied to the words in the text, audio or video files.
[0602] As an example, the identification of protected class information using a Bayesian type filter is described. However, other types of information can be identified using a Bayesian type filter and this example is provided for the purposes of illustration and is not meant to be limiting. For example, as described above, a filter can be developed to identify financial information, such as bankruptcy information.
[0603] In a Bayesian filter, particular words may have particular probabilities of occurring in unit of content including protected class information as opposed to a unit content not including protected class information. For instance, near an individual's birthday, an individual may post content to a social media site including words, such as "my", "birthday," "50," "fifty," "year," "age," and "old." These words may be encountered more frequently in units of content where a person is talking about their age (e.g., fifty) as opposed to a unit of content where a person is not talking about their age. When an individual is forty or over, the individual is part of a protected class in the United States. Thus, probabilities can be assigned to these words and used to determine whether the unit of content that is being analyzed includes protected class information, such as whether the person is over forty.
[0604] A unit of content that is analyzed can vary. For example, a unit of content may be a tweet which has an upper limit on the amount characters. A comment on a social media site, such as a Facebook™ wall may include one or more sentences. A blog post may include multiple sentences and multiple paragraphs. In one embodiment, a unit of content can be analyzed as a whole such that a probability of the unit of content including a particular type of content is determined. In another embodiment, a unit of content can be divided into a number of portions, such as sentences or paragraphs, and each portion can be analyzed to determine whether the portion includes a particular type of content. [0605] In the instance where portions are analyzed, the system can be configured to perform operations based on the portions. For instance, in the instance where any portion of a unit of content is determined to have a probability of above a threshold value of including a particular type of content, the system can be configured to operate on the entire unit of content as a whole, such as blocking from output the entire unit of content. As another example, the system can be configured to operate on the portions of a unit of content on a portion by portion basis. For example, in a blog post including multiple paragraphs, where each sentence is analyzed for a particular type of content, the system can be configured to output only paragraphs not identified as having protected class content.
[0606] The filter doesn't know these probabilities in advance, and must first be trained so it can build them up. Thus, in 602, a number of content units that contain and don't particular content types such as protected class information can be provided. To train the filter, in 602, the system can receive indications in 604 of whether the content unit includes a particular type of content or not, such as protected class information. In one embodiment, these indications can be generated manually, i.e., the determination can be made based upon one or more humans reviewing the content and providing some indication to the system of whether it contains the particular type of content or not.
[0607] In 606, for all words in each training content unit, the filter can adjust the probabilities that each word will appear in a content unit including a particular type of content or a content unit not including the content. For instance, a Bayesian filter may typically have learned a very high probability for the words "Christian" and "Bible", to indicate the content unit includes religious content but a very low probability for words seen only in content unit including non-religious content, such as the names of types of food. When a word first appears that has not been previously seen, the system can define it a default probability. Then, as additional indicators are received, such as its appearance in a content unit identified as including a particular content type or not including a particular type of content, a probability associated with the word relative to the particular content type can be adjusted.
[0608] Bayesian filters can be constructed by assigning probabilities on a word-byword basis. Other types of filters can be constructed that assign probabilities to groups of phrases appearing or not appearing in content units that include a certain type of content. For example, a probability can be assigned to the phrase "my birthday" as being in a content unit including information about someone's age as opposed to assigning probabilities to the words, "my" and "birthday" separately. Thus, in the embodiments described herein, probabilities can be assigned to words individually and/or word phrases for the purposes of determining whether a particular content unit includes a particular content type or not.
[0609] After training, the word probabilities (also known as likelihood functions) are used to compute the probability that a content unit with a particular set of words in it belongs to either category associated with the filter. Each word in the content unit can contribute to determining the content unit's probability of being in a category, or only the most interesting words may be used to contribute to the probability. This contribution is called the posterior probability and is computed using Bayes' theorem. When, the content unit's probability is computed over all or a portion of the words in the content unit, and if the total exceeds a certain threshold (say 95%), the filter will can mark the content unit as including or not including the particular type of content.
[0610] In one embodiment, the calculated probability or probabilities can be saved so that they don't have to be calculated each time. As described below, the system can be configured to allow an entity to specify a different threshold values. Thus, depending on the threshold values specified by an entity, a content unit may or may not be tagged as including a particular content type. Once the probability is determined for a content unit a first time for a particular filter, the probability can be reused without its recalculation to check against different threshold values specified by different entities.
[0611] In 606, one or more filters can be trained. For example, in the case of protected class information, a filter can be trained to determine a probability that a content unit includes information related to age, nation of origin, health condition, gender and genetic conditions (i.e., protected class information). The filter can be trained to determine a probability that the content unit includes any of these content types. In another example, a first filter can be trained to determine a probability of whether a content unit includes age information about an individual, a second filter can be trained to estimate a probability of whether a content unit includes nation of origin information about an individual, a third filter can be trained to estimate a probability that the content unit includes health condition information about an individual, a fourth filter can be trained to estimate a probability that the content unit includes information about the individual's gender and a fifth filter can be trained to determine to estimate a probability that the content unit includes genetic information about an individual. Each of these filters can be applied to a content unit to estimate whether the content unit includes or doesn't include the aforementioned information associated with each filter.
[0612] As described above, for the United States, a default set of protected classes is specified at the federal level and then individual states may specify additional protected classes that can't be used to affect hiring decision. Thus, in one
embodiment, a first filter can be trained to recognize the federal protected classes and then additional filters can be trained for each of the additional state requirements. In another embodiment, a filter can be trained for each state's requirements, respectively, where the filter is configured to determine a probability of whether a content unit includes information related to any of the protected classes associated with a particular state. As described above, in general different countries and localities within countries may have different rules regarding what information may or may not be used in a hiring decision and different filters can be trained to identify this information at a country level and in addition at a local level.
[0613] As described above, different entities (e.g., companies) may have different rules for viewing content in a content unit. In one embodiment, different filters can be trained for different entities depending on these rules. In another embodiments, a set of filters can be trained and then an entity can specify whether all or a portion of filters are to be applied and under what circumstances the filters are to be applied, such as what combination of filters are to be applied in the candidate identification phase as opposed to what combinations of filters are to be applied in post interview phase. In one embodiment, as described above, the same level of filtering may be applied throughout the hiring process.
[0614] In 608, the system can gather and/or update content units attributed to various individuals, such as blog posts. The content can be stored to a database in the recruitment enhancement system. In one embodiment, descriptive metadata, such as a title can be associated with a content unit. For example, a blog post can have a title. In one embodiment, filtering, such as Bayesian filtering, can be applied separately to the metadata (e.g., title) as well as the body of content associated with the metadata where the probabilities assigned to words in the metadata are different or the same as the probabilities assigned to words in the body of content. [0615] In 610, the system can receive filtering parameters to be utilized. For example, filtering parameters can be one or more different filters to apply and a filter threshold associated with the filter. In particular embodiment, the filter thresholds can be different for each filter. In one embodiment, the system can be configured to receive a number on a scale (e.g., 0-4) or a qualitative description of a scale, such low, medium or high, where a threshold value can be assigned to the number or qualitative descriptor. For instance, a value of "1" may correspond to a threshold value of 20% whereas the descriptor, "high," may correspond to a value of "80%."
[0616] In one embodiment, an entity, such as a company, can have specific filter parameters that are to be used when someone is carrying out hiring related tasks.
Users of the system can provide information that allows them to be identified with a particular entity. When the user is identified, the system can determine whether they are associated with a particular entity and look up appropriate filter parameters for the entity and apply them to the user's activities in the recruitment enhancement system that require content filtering.
[0617] In 612, the system can receive search parameters or some other metrics (e.g., a list of names) that causes the system to locate content for a number of different individuals, such as individuals maintained in a database at the recruitment enhancement system. In 614, the system can retrieve content for the one or more individuals. As described above, the content may be content generated by the individuals, such as blog posts or tweets, which can be retrieved by the system.
[0618] In another embodiment, as is described in more detail below, the system can tabulate a frequency of certain words or combinations of words appearing in an individual's social media content and/or resume. Filters can be developed to remove words that may influence a hiring decision in an unadvisable way (e.g., one that could lead to a lawsuit). As an example, words or combination of words related to or which can be associated with a protected class can be removed. For example, the words "pregnant" or "cancer" can be removed as it could lead to a hiring decision considered discriminatory.
[0619] In 616, the system can determine a probability that content in a content unit falls into a particular category, such as including or not including protected class information. In one embodiment, this probability may have been already determined for the content unit and stored by the system. Thus, the system may simply retrieve the value associated with the content unit. As described above, the content unit can be a word, a combination of words, a sentence, a paragraph, multiple paragraphs, a portion of speech analyzed from an audio track, etc. For an individual, the system can be configured to gather and analyze different content unit types generated by the individual.
[0620] In 618, the system can tag content as being in a particular category based upon the determined probabilities and threshold values. For example, when a threshold value is 80% and a probability determined from a filter for identifying protected class information for a content unit is greater than 80%, the system can tag the content unit as including protected class information. If the user has requested that content units including information tagged by the filter not to be displayed, then the system does not output the content unit. In other examples, the user may be looking for content units having a particular content type. In this example, the system may only output information the content units tagged as containing the particular type of content.
[0621] In 620, the system can assemble filtered content for one or more individuals. The assembled content may be required to be tagged or not tagged in a certain manner. The assembled filtered information can be written to a file and optionally made available via a graphical user interface (e.g., see Figures 40A and 40B). For instance, a list of names can be presented and when a user selects the name or some other indicator in the interface, the assembled filtered information for each individual can be output via the interface.
[0622] A mechanism can be provided that allows a user to indicate and/or remove improperly tagged information. For example, content units can be filtered for protected class information where only content units tagged as not including protected class information are output. If a content unit is output that a user identifies as having protected class information, the system can receive an indication of this determination and remove the content if desired from the interface and optionally a file if the content has been written to a file.
[0623] In one embodiment, the system can provide an interface that allows a user to view content that has been tagged or not tagged in a certain manner. For example, the system may allow the user to view content that has been tagged as including protected class information from a particular search. The viewer may wish to view this information to determine how well the system is filtering based upon the specified threshold parameters used for the tagging.
[0624] As described above, the filtering may be performed to prevent a user from viewing particular categories of information for certain individuals. When the system displays tagged information that a user doesn't generally doesn't want to see for a particular individual but still wants to assess the filtering performed by the system, it can be anonymized in some manner. For example, the order of the tagged content can be randomized so that it doesn't correspond to the order of the individuals identified in the search. Further, if desired, such as when the number of individuals in the search is small, the tagged content can be mixed with tagged content from individuals not identified in a search so that the person reviewing the content that was tagged as being or not being in a particular category can't associate it with a particular individual.
[0625] As with the content that is output to the interface according to a tagging criteria, a mechanism can be provided that allows a user to identify information that has been tagged improperly. The system can accept input that assigns a new tag to the identified information. The newly tagged information can be used to retrain one or more filters used in the system.
[0626] In 622, the system can receive indicators of content improperly tagged. In 624, the system can use the indicators and the content associated with the filter to retrain one or more filters. In some instance, the retraining can be performed each time the indicators are received. In other embodiments, the retraining can be performed on a time schedule, such as daily or weekly where all of the indicators and content received since the last training are used to retrain the filters. In yet other
embodiments, retraining may be triggered when a certain number of instances have occurred. For example, each time twenty indicators of improperly tagged content are received for a particular filter, the system may retrain the particular filter.
[0627] In 626, the system can log search results, such as one or more of the person that performed the search, search parameters associated with the search, filters and filter parameters applied during the search, a list of individuals identified in the search, when the search was performed and whether the filtered data for any individuals was output. The names of the individuals whose filtered data was output can also be saved. For example, the system may provide a list of names from a search but may not display information for one of the individuals on the list until it receives a selection of the one or more of the individuals.
[0628] In 626, results such as filtered data for various individuals can be output. For example, a recruiter can perform a search based upon a specified search criterion and receive search results that include a list of names. The recruiter can then select one of the names to receive data that has been filtered in some manner, such as for protected class information. An example of filtered data can be output is described as follows with respect to Figures 40A and 40B.
[0629] Figures 40A and 40B shows an example of an interface 700 configured to display information that can be gathered and aggregated by the system for an individual and then filtered according to one or more of the methods described herein. The interface 700 includes at the top links, "My searches," 702, "How to" 704, "Notification" 706 and "Questions or Feedback," 708.
[0630] When selected, the link "My searches," can cause the interface to generate interface states that allow an individual to access a number of different searches that have been carried out. When selected, the link, "How to," 704 can cause the interface generate interface states including details of how to utilize interface 700. When selected, the link "Notifications," 706 may cause the interface to generate a state showing a user's list of notifications. An example of notification may be messages received from someone else, such as potential candidate or a notification regarding a status of a particular candidate from the system. When selected, the link "Questions or feedback" 708 can cause the interface to generate a state that allows a user to provide feedback or ask questions to a system administrator or some other individual associated with the interface.
[0631] In one embodiment, the interface 700 can include a number of selectable tabs. A different set of information can be displayed in the interface depending on which tab is selected. In the embodiment of Figures 40A and 40B, five tabs are shown, "Summary" 710, "Skills" 712, "Projects" 714, "Social media" 716 and "Safe Media" 718. When selected, the "Summary" tab 710 can cause the interface to display a summary about an individual. When selected, the skills tabs 712 can cause the interface to display skills associated with individual. When selected, the "Projects" tab 714 can cause the interface to display information about projects for an individual, such as projects associated with one of their skills. For example, if one of the skills of an individual is programming, the project information may be related to a programming project. When select, the "social media" tab may include social media related information, such as links to an individual's social media profiles, such as to Linkedln™ or Facebook™. Further, profile information obtained from the social media sites can be displayed.
[0632] In Figures 40A and 40B, the safe media tab 718 has been selected and the safe media information is displayed. The safe media information can refer to information that is considered safe for use in a hiring decision. For example, the information may have been filtered according to the one or more filters previously described, such as a filter for protected class information. In one embodiment, based upon, information received from or gathered for an individual, a psychological profile 720. The psychological profile 720 can include a profile type 722 and a description 724 of the type. In one embodiment, a Myers-Briggs psychological profile can be generated. Indicators 726 associated with a Meyer-Briggs analysis are displayed. The indicators are supposed to indicate a relative tendency towards each of the word pairs, such as whether the person is an introvert or extrovert or whether the person is a feeling or thinking type person.
[0633] In another embodiment, a person's writing style can be analyzed. A writing style 728 is shown. In this example, a tendency towards a formal or more personal writing style is indicated. Additional details of psychological analysis and other information that can be included in a Safe Media tab 718 are described in U.S. patent application no. 13/652,749, entitled, "METHOD, APPARATUS AND COMPUTER PROGRAM PRODUCT TO GENERATE PSYCHOLOGICAL, EMOTIONAL, AND PERSONALITY INFORMATION FOR ELECTRONIC JOB RECRUITING," filed October 16, 2012, which is incorporated by reference in its entirety and for all purposes.
[0634] In one embodiment, the interface can be configured to tabulate a recurrence of particular words or combinations of words that appear in the social media data that has been gathered for a user. In the interface 700, a recurrence of single words 730 and word pairs (i.e., bigrams) 734 is displayed. The relative frequency of the occurrence of single words 732 or bigrams 736 is displayed such that more frequently appearing words appear larger than less frequently appearing words. In other embodiments, a numerical value can be displayed next to each word to indicate there appearance frequency. In particular embodiments, the particular words or word pairs can be filtered for compliance with legal hiring practices. For example, words or word pairs related to protected classes, such as words associated with religion, race or medical conditions may be removed before they are displayed on the interface.
[0635] In another embodiment, recurrent topics 738 can be tabulated that appear in the aggregated data can be displayed. In 700, the topics computers, business, home, society, games, recreations, arts, health, sports, science, cat, cars and cooking are displayed. The topics are sized to indicate a relative frequency of their occurrence. The topics can be selected to be safe for use in a hiring a decision. Thus, unsafe topics, such as topics that may indicate a protected class status, may not be used.
[0636] In yet other embodiments, tweets 742 and blog entries 744 generated by a user and gathered by the system can be displayed. The tweets and blog posts are referred to as safe tweets because one or more filters may have been applied so that only tweets and blog posts including information safe for use in a hiring decision are utilized. However, as described above, the filters may not be perfect. Hence, information that may be considered unsafe by a particular company or individual can be flagged and removed. The tagged information can be used to retrain filters to catch such information in the future.
[0637] While the invention has been described in conjunction with specific embodiments, it will be understood that it is not intended to limit the invention to the described embodiments. On the contrary, it is intended to cover alternatives, modifications, and equivalents as may be included within the spirit and scope of the invention as defined by the appended claims. The present invention may be practiced without some or all of these specific details. In addition, well known features may not have been described in detail to avoid unnecessarily obscuring the invention.
[0638] In accordance with the present invention, the components, process steps, and/or data structures may be implemented using various types of operating systems, programming languages, computing platforms, computer programs, and/or general purpose machines. Methods and graphical user interfaces of the present invention may also be tangibly embodied as a set of computer instructions stored on a computer readable medium, such as a memory device.
[0639] Further, the various aspects, embodiments, implementations or features of the described embodiments can be used separately or in any combination. Various aspects of the described embodiments can be implemented by software, hardware or a combination of hardware and software. The computer readable medium is any data storage device that can store data which can thereafter be read by a computer system. Examples of the computer readable medium include read-only memory, random- access memory, CD-ROMs, DVDs, magnetic tape and optical data storage devices. The computer readable medium can also be distributed over network-coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Claims

CLAIMS What is claimed is:
1. A computer implemented method of providing information for recruiting software developers, comprising:
receiving a query specifying at least one skill of interest to a user; and generating a graphical user interface that is displayed to the user in response to the query, the graphical user interface including profiles of individual software developers and a ranking of software developers fulfilling the query based on an evaluation of software projects stored in public code repositories.
2. The method of claim 1, wherein a profile of each individual software developer includes at least one of: social media links, a listing of software projects, an influence score, a knowledge score, estimated years of experience, a messaging link, and a skill ranking.
3. The method of any of claims 1 or 2, wherein the graphical user interface includes an initial listing of developers satisfying the query ranked based on overall experience.
4. The method of any of claims 1, 2 or 3, wherein the graphical user interface includes an initial listing of developers satisfying the user query along with the ranking of each developer with respect to the at least one skill of the user query.
5. The method of claim 4, wherein the initial listing further includes a ranking for skills commonly associated with the at least one skill set forth in the query.
6. The method of any of claims 1, 2, 3 or 4 further comprising generating a user interface display of a profile of an individual developer selected by the user.
7. A computer implemented method of providing information for recruiting software developers, comprising:
generating profile information for software developers, the profile information including social media information for each developer and a scoring of skills of each developer based on projects individual developers store in code repositories;
generating a graphical user interface that is displayed to a user in response to the user inputting a programming skill query, the user interface including profiles of individual software developers and a ranking of software developers fulfilling the query based on information from software projects stored in public code repositories.
8. The method of claim 7, wherein a profile of each individual software developer includes at least one of: social media links, a listing of software projects, an influence score, a knowledge score, estimated years of experience, a messaging link, and a skill ranking.
9. The method of any of claims 7 or 8, wherein the user interface includes an initial listing of developers satisfying the query ranked based on overall experience.
10. The method of any of claims 7, 8 or 9, wherein the user interface includes an initial listing of developers satisfying the user query along with the ranking of each developer with respect to the at least one skill of the user query.
1 1. The method of claim 10, wherein the initial listing further includes a ranking for skills commonly associated with the at least one skill set forth in the query.
12. The method of any of claims 7, 8, 9 or 10 further comprising generating a user interface display of a profile of an individual developer selected by the user.
13. A computer readable storage medium having computer program code including:
computer program code for generating a graphical user interface in response to a query including at least one programming skill, the graphical user including: an initial listing of profiles of software developers having the at least one programming skill;
selectable individual profiles, wherein an individual profile for an individual software developer includes:
a ranking of the individual developer with respect to other developers based on projects stored in public code repositories.
14. The computer readable medium of claim 13, wherein the graphical user interface further includes: social media information for the individual developer.
15. The computer readable medium of any of claims 13 or 14, wherein an individual profile further comprises: an influence score; a knowledge score; and years of experience.
16. The computer readable medium of any of claims 13, 14 or 15 wherein an individual profile further comprises:
a messaging interface to contact the individual developer.
17. A method, comprising including:
displaying a graphical user interface in response to a query including at least one programming skill, the graphical user including:
an initial listing of profiles of software developers having the at least one programming skill;
selectable individual profiles, wherein an individual profile for an individual software developer includes:
a ranking of the individual developer with respect to other developers based on projects stored in public code repositories.
18. The method of claim 17, wherein the graphical user interface further includes: social media information for the individual developer.
19. The method of any of claims 17 or 18, wherein an individual profile further comprises: an influence score; a knowledge score; and years of experience.
20. The method of any of claims 17, 18 or 19, wherein an individual profile further comprises:
a messaging interface to contact the individual developer.
21. A computer-implemented method of performing recruiting, comprising: acquiring social media content authored by an individual; and
classifying the social media content using Machine Learning (ML) according to at least one factor indicative of a psychological, emotional, or personality characteristic of the individual relevant to recruiting the individual for a job.
22. The method of claim 21, further comprising generating a user interface displaying the at least one factor in response to a request from a user. 23. The method of any claims 21 or 22 wherein the factor indicative of potential interest in seeking new employment includes mood or sentiment.
24. The method of any of claims 21, 22 or 23 wherein the factor of potential interest in seeking new employment includes looking for new employment.
25. The method of any of claims 21, 22, 23 or 24, wherein the factor of potential interest in seeking new employment includes openness to new opportunities.
26. The method of any of claims 21, 22, 23, 24 or 25, wherein the factor indicative of a psychological characteristic of the individual relevant to employment include at least one of personality type, topics of interest, professional formality, masculine or feminine traits, and psychological age.
27. The method of any of claims 21, 22, 23, 24, 25 or 26 further comprising analyzing trends in the classified data.
28. The method of claim 28, further comprising generating an alarm based on a comparison of the trend to a threshold.
29. The method of claim 28, where the alarm is an alarm indicative of a level of unhappiness and a level of mood. 30. The method of any of claims 21, 22, 23, 24, 25, 26 or 27, further comprising correlating the classifications with information regarding the individual's current employer.
31. The method of any of claims 21, 22, 23, 24, 25, 26, 27 or 30, further comprising correlating the classification with information regarding the individual's skills or work experience.
32. A computer-implemented method of performing recruiting, comprising: acquiring social media content authored by an individual; and
generating a user interface including at least one factor, determined from the acquired social media, indicative of a psychological, emotional, or personality characteristics of the individual relevant to recruiting the individual for a job.
33. The method of claim 32, further comprising generating a user interface displaying the at least one factor in response to a request from a user.
34. The method of any of claims 32 or 33 wherein the factor indicative of potential interest in seeking new employment includes mood or sentiment. 35. The method of any of claims 32, 33 or 34 wherein the factor of potential interest in seeking new employment includes looking for new employment.
36. The method of any of claims 32, 33, 34 or 35 wherein the factor of potential interest in seeking new employment includes openness to new opportunities.
37. The method of any of claims 32, 33, 34, 35 or 36 wherein the factor indicative of psychological characteristics of the individual relevant to employment include at least one of personality type, topics of interest, professional formality, masculine or feminine traits, and psychological age.
38. The method of any of claims 32, 33, 34, 35, 36 or 37, further comprising the user interface analyzing trends in the classified data.
39. The method of any of claims 32, 33, 34, 35, 36, 37 or 38, further comprising the user interface generating an alarm based on a comparison of the trend to a threshold.
40. The method of claim 39, where the alarm is an alarm indicative of a level of unhappiness.
41. The method of any of claims 32, 33, 34, 35, 36, 37, 38 or 39, further comprising classifying the acquired social media content from Machine Learning.
42. The method of any of claims 32, 33, 34, 35, 36, 37, 38, 39 or 41, further comprising correlating the classification from Machine Learning with psychological, emotional, or personality characteristics.
43. A system, comprising:
a processor and a memory;
a spider to acquire social media content authored by an individual; and a Machine Learning (ML) module to classify the social media content according to at least one factor indicative of a psychological, emotional, or personality characteristics of the individual relevant to recruiting the individual for a job.
44. A computer-implemented method of performing recruiting, comprising: acquiring social media content authored by an individual; and
classifying the social media content using Machine Learning (ML) according to at least one factor indicative of a psychological, emotional, or personality characteristic of the individual relevant to perform at least operation from the group consisting of: identifying a prospect for a job opening, evaluating a job applicant, and evaluating a current employee.
45. A method of electronic job recruiting in a server including a processor and memory, the method comprising:
determining in the processor a recruitment enhancement activity useful for filtering job candidates in a recruiting process is to be generated;
generating in the processor an interface state that allows a participant to grant access to social media information from a social media site;
retrieving in the processor the social media information from the social media site;
generating in the processor a plurality of interface states that allow the participant to participate in the recruitment enhancement activity;
scoring in the processor the participant's participation in the recruitment enhancement activity;
based upon the participant's score, ranking in the processor the participant's performance; and
generating in the processor an interface state that allows a job recruiter to view the participant's score, ranking and the social media information retrieved about the participant from the social media site.
46. The method of claim 45, wherein the recruitment enhancement activity is a knowledge-based test. 47. The method of claim 46, wherein the recruitment enhancement activity is generated in response to the participant applying for a job position and the knowledge-based test is related to a skill associated with the job position.
48. The method of any of claims 45 or 46, wherein the recruitment enhancement activity includes or more of a knowledge-based questions for the participant to answer, a problem for the participant to solve, a game for the participant to play or combinations thereof.
49. The method of any of claims 45, 46 or 48, wherein the recruitment enhancement activity is generated as part of contest.
50. The method of claim 49, further comprising: receiving a description of one or more prizes associated with the contest.
51. The method of claim 49, further comprising: notifying the participant based upon their score and/or ranking that they have won a prize in the contest. 52. The method of claim 49, further comprising: generating a leader board for the contest.
53. The method of claim 49, further comprising: receiving a time period during which the contest is to be implemented.
54. The method of claim 49, further comprising: receiving a location for the contest wherein the participant's eligibility for the contest is based upon the participant's proximity to the location. 55. The method of claim 49, further comprising: receiving a format for a message that can be posted to a social media site after the participant has participated in the contest.
56. The method of any of claims 45, 46, 48 or 49, further comprising: after generating the recruitment enhancement activity is completed, generating an interface state including a pre-formatted message and a selectable button for posting the pre- formatted message to a social media site wherein the pre-formatted message includes information about the recruitment enhancement activity and the participant's performance in the recruitment enhancement activity.
57. The method of claim 56, further comprising, posting the pre-formatted message in response to detecting a selection of the selectable button.
58. The method of any of claims 45, 46, 48, 49 or 56, further comprising:
determining whether the participant is eligible to participate in the recruitment enhancement activity. 59. The method of claim 58, wherein the determination of the eligibility is based upon whether the participant has previously participated in the recruitment enhancement activity.
60. The method of claim 58, wherein the determination of the eligibility is based upon whether the participant is located with a particular geographic location.
61. The method of any of claims 45, 46, 48, 49, 56 or 58, further comprising: generating a list of recruitment enhancement activities in which the participant is eligible to participate.
62. The method of any of claims 45, 46, 48, 49, 56, 58 or 61, further comprising: receiving information indicating a claim of a skill that the participant possesses; and selecting a recruitment enhancement activity for the participant wherein the participant's performance in the recruitment enhancement activity is predictive of whether the participant possesses the claimed skill.
63. The method of claim 62, further comprising: receiving information indicating a skill level for the claimed skill and
based upon the claimed skill level, selecting the recruitment enhancement activity and/or scoring the recruitment enhancement activity.
64. The method of any of claims 45, 46, 48, 49, 56, 58, 61 or 62, wherein the participant is ranked by comparing their performance to a performance a group of other participants that have participated in the recruitment enhancement activity.
65. The method of any of claims 45, 46, 48, 49, 56, 58, 61, 62 or 64, wherein the participant is ranked according to a scale developed for the recruitment enhancement activity.
66. The method of any of claims 45, 46, 48, 49, 56, 58, 61, 62, 64 or 65, further comprising: generating in the processor an interface state that allows a job recruiter to view the participant's score and ranking from a plurality of different recruitment enhancement activities.
67. The method of any of claims 45, 46, 48, 49, 56, 58, 61, 62, 64, 65 or 66, further comprising: receiving a request from a recruiter for the participant to participate in the recruitment enhancement activity and notifying the participant of the recruiter's request to participate in the recruitment enhancement activity.
68. A method of electronic job recruiting in a server including a processor and memory, the method comprising:
receiving in the processor information associated with professional skills of a first plurality of individuals;
for each the first plurality of individuals, generating in the processor from the received information a score wherein the score provides a qualitative indicator of an aptitude at a profession associated with the professional skills;
storing by the processor at least the score and identification information for each of the first plurality of individuals;
receiving in the processor a recruitment service request including information associated with a second plurality of individuals;
based upon the information received in the service request, determining in the processor whether each of the second plurality of individuals matches one of the first plurality of individuals; and
generating in the processor a response to the recruitment service requests that includes each of the second of plurality of individuals that are matched to one of the first plurality of individuals and the score for the matched individuals. 69. The method of claim 69, wherein the information associated with the professional skills is received from a web crawler.
70. The method of any of claim 68 or 69, wherein the profession is a software developer and the professional skills include software development skills.
71. The method of claim 70, wherein the information associated with the professional skills includes computer code generated by one of the first plurality of individuals and the score is at least partially based upon computer code.
72. The method of any of claim 68, 69 or 70, further comprising: identifying a new individual, generating a score for the new individual and storing the new individual's identification information and score to the database.
73. The method of claim 72, further comprising: receiving new information and based upon the new information updating the score of one of the first plurality of individuals.
74. The method of any of claim 68, 69, 70 or 72, wherein the second plurality of individuals are job applicants that have recently applied for a particular job opening.
75. The method of any of claim 68, 69, 70, 72 or 74, wherein one or more of the second plurality of individuals has previously applied for a job opening in the past that is no longer open.
76. The method of any of claim 68, 69, 70, 72, 74 or 75, wherein the recruitment service request is received from an applicant tracking system in electronic communication with the server.
77. The method of claim 76, wherein information associated with the second plurality of individuals is extracted from a database maintained by the applicant tracking system.
78. The method of any of claim 68, 69, 70, 72, 74, 75 or 76, wherein the information associated with the second plurality of individuals includes a first name, a last name and at least one e-mail address for each individual.
79. The method of claim 78, wherein for one or more of the second plurality of individuals a plurality of e-mail addresses is included. 80. The method of any of claim 68, 69, 70, 72, 74, 75, 76 or 78, further comprising: receiving a plurality of different recruitment service requests from different entities.
81. The method of claim 80, wherein the different entities are different applicant tracking systems.
82. The method of claim 80 wherein at least one individual is common to each of the plurality of different recruitment service requests and based upon the plurality of different recruitment service requests, determining that the at least one individual is looking for a job.
83. The method of claim 82, further comprising: generating and sending a notification message to a recruiter indicating that the at least one individual is looking for a job.
84. The method of any of claim 68, 69, 70, 72, 74, 75, 76, 78 or 80, further comprising: storing to the database information indicating an individual was included in a recruitment service request. 85. The method of any of claim 68, 69, 70, 72, 74, 75, 76, 78, 80 or 84, further comprising: for each of the matches, generating a confidence measure, wherein the confidence measure provides a qualitative indication of a reliability of the match.
86. The method of any of claim 68, 69, 70, 72, 74, 75, 76, 78, 80, 84 or 85, further comprising: generating a group score based upon scores of the second plurality of individuals.
87. The method of any of claim 68, 69, 70, 72, 74, 75, 76, 78, 80, 84, 85 or 86, further comprising: establishing electronic communications with an applicant tracking system and sending the generated response to the applicant tracking system. 88. The method of any of claim 68, 69, 70, 72, 74, 75, 76, 78, 80, 84, 85, 86 or 87, wherein the recruitment service request is generated in response to a search query performed using an applicant tracking system.
89. The method of any of claim 68, 69, 70, 72, 74, 75, 76, 78, 80, 84, 85, 86, 87 or 88, further comprising receiving information related to a third plurality of individuals in the recruitment service request and determining scores to the third plurality of individuals wherein the third plurality of individuals provide a baseline of comparison for the second plurality of individuals. 90. The method of any of claim 68, 69, 70, 72, 74, 75, 76, 78, 80, 84, 85, 86, 87, 88 or 89, further comprising receiving one or more parameters in recruitment service request for generating a custom score.
91. A method of electronic job recruiting in a system including a processor and memory, the method comprising:
training by the processor one or more filters to identify safe data, unsafe data or combinations thereof wherein the safe data is selected to be safe to use in a hiring decision;
receiving by the processor hiring data from a plurality of data sources for a plurality of individuals wherein the hiring data includes resume data and social media data;
aggregating by the processor the received hiring data for each individual; storing by the processor the aggregated hiring data for each individual to a database , wherein the individual's aggregated hiring data includes a plurality of different portions;
receiving by the processor search parameters associated with a hiring decision; based upon the search parameters, locating by the processor at least one individual in the database that satisfy the search parameters; applying by the processor the one or more filters to each portion of the at least one individual's aggregated hiring data to identify safe or unsafe portions of the data; and
outputting by the processor only the safe portions.
92. The method of claim 91 wherein the social media data includes content generated by an individual and posted to a social media site wherein the content is not generated as part of applying for a particular job in a job application process. 93. The method of any of claims 91 or 92 wherein the resume data includes content generated by an individual and submitted as part of applying for a particular job in a job application process.
94. The method of any of claims 91, 92 or 93 wherein at least one of the filters is configured to identify data associated with a federally defined protected class in the
United States of America wherein the data identified as associated with the federally defined protected is considered unsafe data and is not output by the system.
95. The method of any of claims 91, 92, 93 or 94 further comprising receiving an input of a location associated with the hiring decision;
based upon the input of the location, determining a plurality of different types of data considered to be unsafe to use in the hiring decision at the location;
selecting one or more filters to identify the plurality of different types of data; applying the selected one or more filters to the individual's aggregated hiring data.
96. The method of claim 95, wherein the location is a state in the United States of America. 97. The method of any of claims 91, 92, 93, 94 or 95, further comprising receiving information that identifies a company;
determining a plurality of different types of data considered to be unsafe to use in the hiring decision for the company; selecting one or more filters to identify the plurality of different types of data; and
applying the selected one or more filters to the individual's aggregated hiring data.
98. The method of any of claims 91, 92, 93, 94, 95 or 97, wherein the one or more filters are applied to each portion to determine a probability of whether each portion is safe or unsafe to output. 99. The method of claim 98, further comprising receiving a probability threshold and
based upon the received probability threshold, determining whether each portion is safe or unsafe to output. 100. The method of any of claims 91-95, 97 or 98, further comprising receiving an indication that a first portion of the aggregated individual's data identified as safe by the one or more filters is unsafe.
101. The method of claim 100, based upon the indication removing the first portion from the output and preventing the first portion from subsequently being output.
102. The method of claim 100, based upon the indication retraining the one or more filters. 103. The method of any of claims 91-95, 97, 98 or 100, further comprising retrieving the social media data from one or more different social media sites.
104. The method of claim 103, further comprising periodically checking the social media sites to determine whether new social media data is available and updating the aggregated hiring data in the database when new social media data is available including deleting old social media data.
105. The method of any of claims 91-95, 97, 98, 100 or 103, wherein the individual's aggregated hiring data includes images and further comprising applying the one or more filters to determine whether each of the images is safe or unsafe to use in the hiring decision and outputting only the safe images.
106. The method of any of claims 91-95, 97, 98, 100, 103 or 105, further comprising determining a frequency at which particular words or combinations of words appear in the individual's aggregated hiring data, applying the one or more filters to the particular words or combinations of words to determine whether each one is safe or unsafe to use in a hiring decision and outputting only the particular words or combinations of words determined to be safe.
107. The method of any of claims 91-95, 97, 98, 100, 103, 105 or 106, wherein the social media data in the at least one individual's aggregated hiring data includes tweets, applying the one or more filters to each tweet to determine whether each tweet is safe or unsafe to use in the hiring decision and outputting only the safe tweets.
108. The method of any of claims 91-95, 97, 98, 100, 103, 105, 106 or 107, wherein the social media data in the at least one individual's aggregated hiring data includes blog posts, applying the one or more filters to each blog post to determine whether each blog post is safe or unsafe to use in the hiring decision and outputting one the safe blog posts.
109. The method of any of claims 91-95, 97, 98, 100, 103, 105, 106, 107 or 108, further comprising receiving a request to output one or more portions determined to be unsafe for use in the hiring decision and prior to outputting the one or more portions, anonymizing the one or more portions to prevent the one or more portions from being associated with a particular individual. 1 10. The method of any of claims 91-95, 97, 98, 100, 103, 105, 106, 107, 108 or
109, further comprising receiving an input indicating a stage in the hiring process and based upon the input stage, selecting one or more filters to apply to the individual's aggregated hiring data wherein types of data considered to be safe or unsafe to use in the hiring decision varies depending on the stage of the hiring process.
1 1 1. The method of any of claims 91-95, 97, 98, 100, 103, 105-109 or 110, further comprising storing a log of the search including names of individuals identified in the search, an identifier of the person performing the search and information regarding the one or more filters used to identify data that is safe or unsafe to use in a hiring decision. 1 12. The method of claim 1 1 1, further comprising storing an indication of whether the safe portions were viewed or not viewed for each of the individuals identified in the search.
23. A tangible computer readable medium for storing computer code executed by a processor to generate a method of job recruiting, the computer readable medium comprising:
computer code for training one or more filters to identify safe data, unsafe data or combinations thereof wherein the safe data is selected to be safe to use in a hiring decision;
computer code for receiving hiring data from a plurality of data sources for a plurality of individuals wherein the hiring data includes resume data and social media data;
computer code for aggregating the received hiring data for each individual; computer code for storing the aggregated hiring data for each individual to a database , wherein the individual's aggregated hiring data includes a plurality of different portions;
computer code for receiving search parameters associated with a hiring decision;
computer code, based upon the search parameters, for locating at least one individual in the database that satisfy the search parameters;
computer code for applying the one or more filters to each portion of the at least one individual's aggregated hiring data to identify safe or unsafe portions of the data; and computer code for outputting only the safe portions.
PCT/US2013/038702 2012-04-30 2013-04-29 Recruitment enhancement system WO2013165923A1 (en)

Applications Claiming Priority (12)

Application Number Priority Date Filing Date Title
US201261640656P 2012-04-30 2012-04-30
US61/640,656 2012-04-30
US13/493,791 2012-06-11
US13/493,791 US8719179B2 (en) 2012-04-30 2012-06-11 Recruiting service graphical user interface
US13/557,812 2012-07-25
US13/557,812 US20140032435A1 (en) 2012-07-25 2012-07-25 Method and apparatus for enhancing job recruiting
US13/568,493 US20130290206A1 (en) 2012-04-30 2012-08-07 Method and apparatus for electronic job recruiting
US13/568,493 2012-08-07
US13/652,749 2012-10-16
US13/652,749 US20130290207A1 (en) 2012-04-30 2012-10-16 Method, apparatus and computer program product to generate psychological, emotional, and personality information for electronic job recruiting
US13/739,381 US20130290208A1 (en) 2012-04-30 2013-01-11 Social media data filtering for electronic job recruiting
US13/739,381 2013-01-11

Publications (1)

Publication Number Publication Date
WO2013165923A1 true WO2013165923A1 (en) 2013-11-07

Family

ID=49514782

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2013/038702 WO2013165923A1 (en) 2012-04-30 2013-04-29 Recruitment enhancement system

Country Status (1)

Country Link
WO (1) WO2013165923A1 (en)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016118432A1 (en) * 2015-01-21 2016-07-28 44Lab5, Llc Method and system for auto -responding on social media posting
US9734207B2 (en) 2015-12-28 2017-08-15 Entelo, Inc. Entity resolution techniques and systems
US10332137B2 (en) * 2016-11-11 2019-06-25 Qwalify Inc. Proficiency-based profiling systems and methods
CN110442862A (en) * 2019-07-11 2019-11-12 新华三大数据技术有限公司 Data processing method and device based on recruitment information
US20190347355A1 (en) * 2018-05-11 2019-11-14 Facebook, Inc. Systems and methods for classifying content items based on social signals
CN111798217A (en) * 2020-07-10 2020-10-20 河北冀联人力资源服务集团有限公司 Data analysis system and method
US11127232B2 (en) 2019-11-26 2021-09-21 On Time Staffing Inc. Multi-camera, multi-sensor panel data extraction system and method
US11144880B2 (en) 2018-12-06 2021-10-12 At&T Intellectual Property I, L.P. Document analysis using machine learning and neural networks
US11144882B1 (en) 2020-09-18 2021-10-12 On Time Staffing Inc. Systems and methods for evaluating actions over a computer network and establishing live network connections
US11184578B2 (en) 2020-04-02 2021-11-23 On Time Staffing, Inc. Audio and video recording and streaming in a three-computer booth
US20220020074A1 (en) * 2020-07-17 2022-01-20 SupportFinity Inc. System and Method for Automatically Generating Online Quote for Team or Service
US11423071B1 (en) 2021-08-31 2022-08-23 On Time Staffing, Inc. Candidate data ranking method using previously selected candidate data
US11457140B2 (en) 2019-03-27 2022-09-27 On Time Staffing Inc. Automatic camera angle switching in response to low noise audio to create combined audiovisual file
US20230038755A1 (en) * 2021-08-03 2023-02-09 Hiredscore Inc. System and method for improving fairness among job candidates
US11727040B2 (en) 2021-08-06 2023-08-15 On Time Staffing, Inc. Monitoring third-party forum contributions to improve searching through time-to-live data assignments
US11763263B1 (en) * 2013-12-30 2023-09-19 Massachusetts Mutual Life Insurance Company Systems and methods for identifying and ranking successful agents based on data analytics

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090187473A1 (en) * 2008-01-18 2009-07-23 Blaze Jerry M System and method for recruiting online
US20100153290A1 (en) * 2008-12-12 2010-06-17 Paul Duggan Methods of matching job profiles and candidate profiles
US20110077989A1 (en) * 2009-09-24 2011-03-31 Accenture Global Services Limited System for valuating employees
US20110196802A1 (en) * 2010-02-05 2011-08-11 Nicholas Jeremy Ellis Method and apparatus for hiring using social networks
US20110313963A1 (en) * 2010-01-22 2011-12-22 AusGrads Pty Ltd Recruiting system
US20120005113A1 (en) * 2009-01-29 2012-01-05 Lifehistory Pty Ltd System and method for assessing employee work performance

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090187473A1 (en) * 2008-01-18 2009-07-23 Blaze Jerry M System and method for recruiting online
US20100153290A1 (en) * 2008-12-12 2010-06-17 Paul Duggan Methods of matching job profiles and candidate profiles
US20120005113A1 (en) * 2009-01-29 2012-01-05 Lifehistory Pty Ltd System and method for assessing employee work performance
US20110077989A1 (en) * 2009-09-24 2011-03-31 Accenture Global Services Limited System for valuating employees
US20110313963A1 (en) * 2010-01-22 2011-12-22 AusGrads Pty Ltd Recruiting system
US20110196802A1 (en) * 2010-02-05 2011-08-11 Nicholas Jeremy Ellis Method and apparatus for hiring using social networks

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11763263B1 (en) * 2013-12-30 2023-09-19 Massachusetts Mutual Life Insurance Company Systems and methods for identifying and ranking successful agents based on data analytics
WO2016118432A1 (en) * 2015-01-21 2016-07-28 44Lab5, Llc Method and system for auto -responding on social media posting
US9734207B2 (en) 2015-12-28 2017-08-15 Entelo, Inc. Entity resolution techniques and systems
US10332137B2 (en) * 2016-11-11 2019-06-25 Qwalify Inc. Proficiency-based profiling systems and methods
US20190347355A1 (en) * 2018-05-11 2019-11-14 Facebook, Inc. Systems and methods for classifying content items based on social signals
US11144880B2 (en) 2018-12-06 2021-10-12 At&T Intellectual Property I, L.P. Document analysis using machine learning and neural networks
US11457140B2 (en) 2019-03-27 2022-09-27 On Time Staffing Inc. Automatic camera angle switching in response to low noise audio to create combined audiovisual file
CN110442862A (en) * 2019-07-11 2019-11-12 新华三大数据技术有限公司 Data processing method and device based on recruitment information
CN110442862B (en) * 2019-07-11 2022-08-09 新华三大数据技术有限公司 Data processing method and device based on recruitment information
US11127232B2 (en) 2019-11-26 2021-09-21 On Time Staffing Inc. Multi-camera, multi-sensor panel data extraction system and method
US11783645B2 (en) 2019-11-26 2023-10-10 On Time Staffing Inc. Multi-camera, multi-sensor panel data extraction system and method
US11184578B2 (en) 2020-04-02 2021-11-23 On Time Staffing, Inc. Audio and video recording and streaming in a three-computer booth
US11636678B2 (en) 2020-04-02 2023-04-25 On Time Staffing Inc. Audio and video recording and streaming in a three-computer booth
CN111798217A (en) * 2020-07-10 2020-10-20 河北冀联人力资源服务集团有限公司 Data analysis system and method
US20220020074A1 (en) * 2020-07-17 2022-01-20 SupportFinity Inc. System and Method for Automatically Generating Online Quote for Team or Service
US11720859B2 (en) 2020-09-18 2023-08-08 On Time Staffing Inc. Systems and methods for evaluating actions over a computer network and establishing live network connections
US11144882B1 (en) 2020-09-18 2021-10-12 On Time Staffing Inc. Systems and methods for evaluating actions over a computer network and establishing live network connections
US20230038755A1 (en) * 2021-08-03 2023-02-09 Hiredscore Inc. System and method for improving fairness among job candidates
US11727040B2 (en) 2021-08-06 2023-08-15 On Time Staffing, Inc. Monitoring third-party forum contributions to improve searching through time-to-live data assignments
US11423071B1 (en) 2021-08-31 2022-08-23 On Time Staffing, Inc. Candidate data ranking method using previously selected candidate data

Similar Documents

Publication Publication Date Title
US11748572B2 (en) Enabling chatbots by validating argumentation
US10839154B2 (en) Enabling chatbots by detecting and supporting affective argumentation
US10679011B2 (en) Enabling chatbots by detecting and supporting argumentation
US20130290208A1 (en) Social media data filtering for electronic job recruiting
WO2013165923A1 (en) Recruitment enhancement system
US10839161B2 (en) Tree kernel learning for text classification into classes of intent
US20130290207A1 (en) Method, apparatus and computer program product to generate psychological, emotional, and personality information for electronic job recruiting
US11386274B2 (en) Using communicative discourse trees to detect distributed incompetence
Canter et al. Profiling in policy and practice
Wolfgang Taming the ‘trolls’: How journalists negotiate the boundaries of journalism and online comments
Haque et al. Combating Misinformation in Bangladesh: roles and responsibilities as perceived by journalists, fact-checkers, and users
US20130290206A1 (en) Method and apparatus for electronic job recruiting
US20220318513A9 (en) Discourse parsing using semantic and syntactic relations
Ha et al. Political efficacy and the use of local and national news media among undecided voters in a swing state: A study of general population voters and first-time college student voters
Wang Debunking misinformation about genetically modified food safety on social media: Can heuristic cues mitigate biased assimilation?
Kwon et al. Dark knowledge and platform governance: A case of an illicit e-commerce community in reddit
Judge et al. Voter privacy and big-data elections
Chan Technology-facilitated gender-based violence, hate speech, and terrorism: a risk assessment on the rise of the incel rebellion in Canada
Doss Cyber Privacy: Who Has Your Data and Why You Should Care
Bohigues et al. Drug legalization, elite support, and government effectiveness in Latin America
Suarez We are cyborgs: Developing a theoretical model for understanding criminal behaviour on the internet
McNeill Oceania’s ‘crimmigration creep’: Are deportation and reintegration norms being diffused?
US20220284194A1 (en) Using communicative discourse trees to detect distributed incompetence
US20220377035A1 (en) Kiwi chat
Bansal et al. A scoping review of technology-facilitated gender-based violence in low-and middle-income countries across Asia

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13784356

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205N DATED 12/01/2015)

122 Ep: pct application non-entry in european phase

Ref document number: 13784356

Country of ref document: EP

Kind code of ref document: A1