US20230068717A1 - Methods and systems for determining relevance of documents - Google Patents
Methods and systems for determining relevance of documents Download PDFInfo
- Publication number
- US20230068717A1 US20230068717A1 US18/045,308 US202218045308A US2023068717A1 US 20230068717 A1 US20230068717 A1 US 20230068717A1 US 202218045308 A US202218045308 A US 202218045308A US 2023068717 A1 US2023068717 A1 US 2023068717A1
- Authority
- US
- United States
- Prior art keywords
- resumes
- resume
- documents
- text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 239000013598 vector Substances 0.000 claims abstract description 33
- 238000012216 screening Methods 0.000 claims abstract description 16
- 230000008569 process Effects 0.000 description 22
- 238000013528 artificial neural network Methods 0.000 description 20
- 238000012549 training Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 12
- 230000008676 import Effects 0.000 description 9
- 230000009466 transformation Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 230000004913 activation Effects 0.000 description 4
- 238000013434 data augmentation Methods 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 238000000844 transformation Methods 0.000 description 3
- 230000003416 augmentation Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011982 device technology Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000003278 mimic effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/131—Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/151—Transformation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/216—Parsing using statistical methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/237—Lexical tools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/284—Lexical analysis, e.g. tokenisation or collocates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
Definitions
- FIGS. 1 A and 1 B illustrate example processes for screening resumes, according to embodiments of the invention.
- FIGS. 1 C and 1 D illustrate example systems for screening resumes, according to an embodiment of the invention.
- FIG. 2 provides an example of word-to-vector encoding, according to an embodiment of the invention.
- FIG. 3 provides examples of how vector algebra can be used with word vectors, according to an embodiment of the invention.
- FIG. 4 illustrates how the system can be flexible and trained to learn different goals and vocabularies, according to an embodiment of the invention.
- FIG. 5 illustrates an example of PDF to text transformation, according to an embodiment of the invention.
- FIGS. 6 A, 6 B and 6 C illustrates an example of tokenization, according to an embodiment of the invention.
- FIGS. 7 A, 7 B and 7 C illustrate an example of resume input and process, according to an embodiment of the invention.
- FIG. 8 illustrates an example of data augmentation, according to an embodiment of the invention.
- FIGS. 9 A, 9 B and 9 C illustrate an example use of a deep neural network, according to an embodiment of the invention.
- FIG. 10 illustrates an example of predicting candidate goodness, according to an embodiment of the invention.
- FIG. 1 A illustrates a process for screening resumes, according to an embodiment.
- resumes can be entered.
- the resumes can be entered in pdf format and converted to text.
- augmentation can be done on the resumes to create additional resumes. (These can be used, for example, in training the system.)
- a vocabulary list can be built for the position of interest.
- each word in the vocabulary list can be coded as a vector using word embedding.
- vector relationships can be defined.
- a classification e.g., based on a relevance probability score
- FIG. 1 B illustrates an overview process for screening resumes, according to an embodiment.
- resumes can be classified in a learning process based on content learned from relevant samples.
- words from resumes are input.
- each word of the resumes can be transformed into a series of numbers (e.g., vectors), which can be called embeddings.
- the deep neural network can learn the vector values using the relationships that adjacent words have in classifying good and bad resumes.
- each resume can be transformed into a set of previously learned word vectors which can be used to determine a classification score which reflect the possibility that a candidate is worth pursuing.
- FIG. 1 B resumes can be classified in a learning process based on content learned from relevant samples.
- words from resumes are input.
- each word of the resumes can be transformed into a series of numbers (e.g., vectors), which can be called embeddings.
- the deep neural network can learn the vector values using the relationships that adjacent words have in classifying good and bad resumes.
- each resume can be transformed
- resumes can be classified in a learning process based on content learned from relevant samples.
- words from resumes are input.
- each word of the resumes can be transformed into a series of numbers (e.g., vectors), which can be called embeddings.
- the deep neural network can learn the vector values using the relationships that adjacent words have in classifying good and bad resumes.
- each resume can be transformed into a set of previously learned word vectors which can be used to determine a classification score which reflect the possibility that a candidate is worth pursuing.
- FIG. 1 C illustrates a block diagram of an example system architecture 100 implementing the features and processes described herein.
- the architecture 100 may be implemented on any electronic device that runs software applications derived from compiled instructions, including without limitation personal computers, servers, smart phones, media players, electronic tablets, game consoles, email devices, etc.
- the architecture 100 may include one or more processors 102 , one or more input devices 104 , one or more display devices 106 , one or more network interfaces 108 , and one or more computer-readable mediums 110 . Each of these components may be coupled by bus 112 .
- Display device 106 may be any known display technology, including but not limited to display devices using Liquid Crystal Display (LCD) or Light Emitting Diode (LED) technology.
- Processor(s) 102 may use any known processor technology, including but not limited to graphics processors and multi-core processors.
- Input device 104 may be any known input device technology, including but not limited to a keyboard (including a virtual keyboard), mouse, track ball, and touch-sensitive pad or display.
- Bus 112 may be any known internal or external bus technology, including but not limited to ISA, EISA, PCI, PCI Express, NuBus, USB, Serial ATA or FireWire.
- Computer-readable medium 110 may be any medium that participates in providing instructions to processor(s) 102 for execution, including without limitation, non-volatile storage media (e.g., optical disks, magnetic disks, flash drives, etc.), or volatile media (e.g., SDRAM, ROM, etc.).
- non-volatile storage media e.g., optical disks, magnetic disks, flash drives, etc.
- volatile media e.g., SDRAM, ROM, etc.
- Computer-readable medium 110 may include various instructions 114 for implementing an operating system (e.g., Mac OS®, Windows®, Linux).
- the operating system may be multi-user, multiprocessing, multitasking, multithreading, real-time, and the like.
- the operating system may perform basic tasks, including but not limited to: recognizing input from input device 104 ; sending output to display device 106 ; keeping track of files and directories on computer-readable medium 110 ; controlling peripheral devices (e.g., disk drives, printers, etc.) which can be controlled directly or through an I/O controller; and managing traffic on bus 112 .
- Network communications instructions 116 may establish and maintain network connections (e.g., software for implementing communication protocols, such as TCP/IP, HTTP, Ethernet, etc.).
- FIG. 1 D illustrates a system 199 for screening documents, according to an embodiment.
- This system comprises: a PDF to text module 195 , an augmentation module 188 , a build vocabulary module 190 ; a coding words module 185 , a deep neural network module 180 , and a classify resumes module 175 .
- FIG. 2 provides an example of how embeddings can encode and summarize relevant features.
- the word HOUSE can be turned into a vector using word-to-vector encoding.
- the vectors can comprise latent or unknown vectors that the system determines are relevant.
- embeddings can be mappings. Size can be a part of designing the problem solution.
- Each word e.g., 172,000 in the English vocabulary
- the features can be about a word's syntax and semantic context captured by learning the embeddings.
- FIG. 3 provides examples of how vector algebra can be used with the word vectors.
- the algebra of vectors can mimic the correct interpretation of English words, using latent features vectors (i.e. embeddings).
- latent features vectors i.e. embeddings.
- the system recognizes that a “good” resume will have a sum of 12, then any combination of vector values that reaches 12 can indicate that that resume is a good resume. So if two vectors have a value of 1 and 5, then the sum is 6, and this would not be a “good” resume. However, if two word vectors have a sum of 13, then this would be considered a “good” resume.
- the screening process can be utilized to determine the probability that a resume is of interest. For example, if 1000 resumes are submitted for a particular position, the screening process can provide a probability of relevance for each resume and rank the order of the resumes according to the probability of relevance rankings. In this way, a hiring person can determine almost immediately which resumes are the most important. This may be valuable, for example, when it is important to make an early contact for a prime candidate for a position, rather than waiting several weeks or months to get to that particular resume in a list or pile of resumes. This process can also, for example, cut down significantly on the amount of time it takes to review resumes.
- a hiring person can start reviewing the resumes with the highest probabilities first in order to maximize the chances that the most relevant or desirable hires are reviewed and interviewed first.
- the screening process can also, for example, cut down on the time it takes to interview a candidate. For example, if a resume is determined to be a very good candidate, the first pre-screening interview can focus on language skills and personality, instead of qualifications, which can result in a significant reduction of the time spent in the first interview.
- resumes that are determined to have a low probability of relevance can be reviewed, and if it is determined that they should have a higher probability of relevance, these can be fed into the system in order to better train the system.
- resumes that are determined to have a high probability of relevance can be reviewed, and if it is determined that they should have a lower probability of relevance, these can be fed into the system in order to better train the system.
- FIG. 4 illustrates how the system can be flexible and trained to learn different goals and vocabularies.
- sample documents e.g., resumes
- the PDF documents can be imported into the system, and the words can be tokenized.
- a dictionary of vocabulary words can be created.
- the vocabulary words can be used to embed an index.
- a model code and performance metrics can be output with accuracy, recall (e.g., true positives divided by the sum of true positives and false negatives) and precision (e.g., true positives divided by the sum of the true positives and the false positives) information.
- three set of data may be input: resumes (in PDF format) to be scored, the relevant vocabulary list, and the model.
- Each PDF document can be run through the model and a probability of being a good candidate can be assigned.
- the output can be a prioritized list of candidates to interview.
- Vocabulary Building In some embodiments, the following process can be used to build the vocabulary.
- PDF to Text Transformation In some embodiments, the following process can be used to transform the pdf document to text.
- the training procedure can assume there is a set of resume PDF documents that have been stored in two different folders.
- Folder “yes” can include candidates that have been deemed to be successful in the past after a pre-screening hiring round by a hiring team of person(s).
- Folder “no” can include candidates who did not pass the pre-screening hiring round by the hiring team of person(s).
- the following process can be used, For example, using Python as the coding language, with a Keras library and a TensorFlow backend.
- FIG. 7 C Once we have transformed the resume into a list of word indexes based on the predetermined vocabulary, as shown in FIG. 7 C , we can proceed with a typical machine learning step of splitting our data into: 1) a training dataset, which can be used to determine the model parameters; and 2) a validation dataset, which can be used to test the generalization of the model. Once we have determined the training set, we can proceed with the data augmentation technique.
- the shuffled sentences can then be joined together to create new example resumes that can help teach the computer. For example, if we found 7 sentences in a particular resume, we could order those S1 to S7, and then shuffle them randomly so that we create several more example resumes. Thus, the original resume could be ordered S1, S2, S3, S4, S5, S6, and S7. After shuffling and joining together the shuffled sentences, we could have several other example resumes made of the same sentences. An example could be the resume including sentences in the following order: S2, S4, S7, S5, S1, S3, and S6. Once we have the new resumes, we can turn them into tokens and encode them. This process can be helpful when treating resumes as these documents usually include sentences and/or paragraphs and/or other portions that are independent from each other. The resulting ‘shuffled’ resumes are still resumes.
- a deep neural network is an ANN (Artificial Neural Network) with multiple hidden layers between the input and output layers.
- FIG. 9 A is an example of pre-processing that comprises: eliminating non-frequent words, padding the text to make all resume the same length, removing resumes that are too short, etc.
- embeddings can be used so that word indexes can be mapped to random vectors of a fixed size (e.g. 100, 200, 300). Those vectors can get their numbers fixed during the training procedure.
- FIG. 9 B illustrates an example of a Deep Neural Network using convolutional layers and word embeddings written in Keras. The embeddings value can create a matrix of a number of words in the resume ⁇ size of the embedding vector.
- Each row is a layer of the network gets input from the previous layer, applies the layer specific transformations and provides its output as the following layer input.
- FIG. 9 C illustrates an example transformation of a resume sentence into a set of embeddings using a word index vocabulary.
- the embeddings values can be initialized randomly.
- the Deep Neural Network algorithm can find the ones that best fit our proposed purpose during the training procedure using an optimization algorithm such as, for example, the Stochastic Gradient Descent.
- Stochastic gradient descent (often shortened to SGD), also known as incremental gradient descent, is a stochastic approximation of the gradient descent optimization and iterative method for minimizing an objective function that is written as a sum of differentiable functions.
- the scoring procedure comes from multiple transformations of the word vectors using model parameters found during training for the neural network functions.
- the output can then reshaped to a nice .cvs format as illustrated in FIG. 10 .
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Medical Informatics (AREA)
- Probability & Statistics with Applications (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Machine Translation (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
- This application is a continuation application to U.S. application Ser. No. 16/891,396 filed Jun. 3, 2020, which is a continuation application to U.S. application Ser. No. 16/203,102 filed Nov. 28, 2018, which claims the benefit of U.S. Provisional Application No. 62/621,404 filed Jan. 24, 2018, which are incorporated by reference in their entireties.
-
FIGS. 1A and 1B illustrate example processes for screening resumes, according to embodiments of the invention. -
FIGS. 1C and 1D illustrate example systems for screening resumes, according to an embodiment of the invention. -
FIG. 2 provides an example of word-to-vector encoding, according to an embodiment of the invention. -
FIG. 3 provides examples of how vector algebra can be used with word vectors, according to an embodiment of the invention. -
FIG. 4 illustrates how the system can be flexible and trained to learn different goals and vocabularies, according to an embodiment of the invention. -
FIG. 5 illustrates an example of PDF to text transformation, according to an embodiment of the invention. -
FIGS. 6A, 6B and 6C illustrates an example of tokenization, according to an embodiment of the invention. -
FIGS. 7A, 7B and 7C illustrate an example of resume input and process, according to an embodiment of the invention. -
FIG. 8 illustrates an example of data augmentation, according to an embodiment of the invention. -
FIGS. 9A, 9B and 9C illustrate an example use of a deep neural network, according to an embodiment of the invention. -
FIG. 10 illustrates an example of predicting candidate goodness, according to an embodiment of the invention. - The following description illustrates example embodiments for screening documents. Although resumes are used throughout this application as example embodiments, those of ordinary skill in the art will see that many other types of documents may also be utilized.
FIG. 1A illustrates a process for screening resumes, according to an embodiment. In 105, resumes can be entered. In some embodiments, the resumes can be entered in pdf format and converted to text. In 107, augmentation can be done on the resumes to create additional resumes. (These can be used, for example, in training the system.) In 110, a vocabulary list can be built for the position of interest. In 112, each word in the vocabulary list can be coded as a vector using word embedding. In 115, vector relationships can be defined. In 120, a classification (e.g., based on a relevance probability score) for each resume can be output. -
FIG. 1B illustrates an overview process for screening resumes, according to an embodiment. InFIG. 1B , resumes can be classified in a learning process based on content learned from relevant samples. In 195, words from resumes are input. In 196, each word of the resumes can be transformed into a series of numbers (e.g., vectors), which can be called embeddings. In 197, the deep neural network can learn the vector values using the relationships that adjacent words have in classifying good and bad resumes. In 198, each resume can be transformed into a set of previously learned word vectors which can be used to determine a classification score which reflect the possibility that a candidate is worth pursuing. InFIG. 1B , resumes can be classified in a learning process based on content learned from relevant samples. In 195, words from resumes are input. In 196, each word of the resumes can be transformed into a series of numbers (e.g., vectors), which can be called embeddings. In 197, the deep neural network can learn the vector values using the relationships that adjacent words have in classifying good and bad resumes. In 198, each resume can be transformed into a set of previously learned word vectors which can be used to determine a classification score which reflect the possibility that a candidate is worth pursuing. -
FIG. 1C illustrates a block diagram of anexample system architecture 100 implementing the features and processes described herein. Thearchitecture 100 may be implemented on any electronic device that runs software applications derived from compiled instructions, including without limitation personal computers, servers, smart phones, media players, electronic tablets, game consoles, email devices, etc. In some implementations, thearchitecture 100 may include one ormore processors 102, one ormore input devices 104, one ormore display devices 106, one ormore network interfaces 108, and one or more computer-readable mediums 110. Each of these components may be coupled bybus 112. -
Display device 106 may be any known display technology, including but not limited to display devices using Liquid Crystal Display (LCD) or Light Emitting Diode (LED) technology. Processor(s) 102 may use any known processor technology, including but not limited to graphics processors and multi-core processors.Input device 104 may be any known input device technology, including but not limited to a keyboard (including a virtual keyboard), mouse, track ball, and touch-sensitive pad or display.Bus 112 may be any known internal or external bus technology, including but not limited to ISA, EISA, PCI, PCI Express, NuBus, USB, Serial ATA or FireWire. Computer-readable medium 110 may be any medium that participates in providing instructions to processor(s) 102 for execution, including without limitation, non-volatile storage media (e.g., optical disks, magnetic disks, flash drives, etc.), or volatile media (e.g., SDRAM, ROM, etc.). - Computer-
readable medium 110 may includevarious instructions 114 for implementing an operating system (e.g., Mac OS®, Windows®, Linux). The operating system may be multi-user, multiprocessing, multitasking, multithreading, real-time, and the like. The operating system may perform basic tasks, including but not limited to: recognizing input frominput device 104; sending output to displaydevice 106; keeping track of files and directories on computer-readable medium 110; controlling peripheral devices (e.g., disk drives, printers, etc.) which can be controlled directly or through an I/O controller; and managing traffic onbus 112.Network communications instructions 116 may establish and maintain network connections (e.g., software for implementing communication protocols, such as TCP/IP, HTTP, Ethernet, etc.). -
FIG. 1D illustrates asystem 199 for screening documents, according to an embodiment. This system comprises: a PDF totext module 195, anaugmentation module 188, abuild vocabulary module 190; acoding words module 185, a deepneural network module 180, and a classifyresumes module 175. -
FIG. 2 provides an example of how embeddings can encode and summarize relevant features. For example, the word HOUSE can be turned into a vector using word-to-vector encoding. The vectors can comprise latent or unknown vectors that the system determines are relevant. Mathmatically speaking, embeddings can be mappings. Size can be a part of designing the problem solution. Each word (e.g., 172,000 in the English vocabulary) can be a dimension. The features can be about a word's syntax and semantic context captured by learning the embeddings. -
FIG. 3 provides examples of how vector algebra can be used with the word vectors. InFIG. 3 , the algebra of vectors can mimic the correct interpretation of English words, using latent features vectors (i.e. embeddings). As a very simply example, if the system recognizes that a “good” resume will have a sum of 12, then any combination of vector values that reaches 12 can indicate that that resume is a good resume. So if two vectors have a value of 1 and 5, then the sum is 6, and this would not be a “good” resume. However, if two word vectors have a sum of 13, then this would be considered a “good” resume. - The screening process can be utilized to determine the probability that a resume is of interest. For example, if 1000 resumes are submitted for a particular position, the screening process can provide a probability of relevance for each resume and rank the order of the resumes according to the probability of relevance rankings. In this way, a hiring person can determine almost immediately which resumes are the most important. This may be valuable, for example, when it is important to make an early contact for a prime candidate for a position, rather than waiting several weeks or months to get to that particular resume in a list or pile of resumes. This process can also, for example, cut down significantly on the amount of time it takes to review resumes. For example, when a probability is given for each resume, a hiring person can start reviewing the resumes with the highest probabilities first in order to maximize the chances that the most relevant or desirable hires are reviewed and interviewed first. The screening process can also, for example, cut down on the time it takes to interview a candidate. For example, if a resume is determined to be a very good candidate, the first pre-screening interview can focus on language skills and personality, instead of qualifications, which can result in a significant reduction of the time spent in the first interview.
- In some embodiments, resumes that are determined to have a low probability of relevance can be reviewed, and if it is determined that they should have a higher probability of relevance, these can be fed into the system in order to better train the system. In a similar way, resumes that are determined to have a high probability of relevance can be reviewed, and if it is determined that they should have a lower probability of relevance, these can be fed into the system in order to better train the system.
-
FIG. 4 illustrates how the system can be flexible and trained to learn different goals and vocabularies. For example, with respect to the vocabulary setting, sample documents (e.g., resumes) in PDF format can be input into the system. The PDF documents can be imported into the system, and the words can be tokenized. A dictionary of vocabulary words can be created. The vocabulary words can be used to embed an index. - With respect to the model training, two folders of PDF documents can be created, one with examples of good candidates or resumes, and one with examples of bad candidates or resumes. The PDF documents can be converted to text. Sentences in the resumes can be pre-processed. The words can be mapped to embeddings. The data can then be augmented. A model code and performance metrics can be output with accuracy, recall (e.g., true positives divided by the sum of true positives and false negatives) and precision (e.g., true positives divided by the sum of the true positives and the false positives) information.
- With respect to candidate predictions, three set of data may be input: resumes (in PDF format) to be scored, the relevant vocabulary list, and the model. Each PDF document can be run through the model and a probability of being a good candidate can be assigned. The output can be a prioritized list of candidates to interview.
- Vocabulary Building. In some embodiments, the following process can be used to build the vocabulary.
-
- 1. Import resume file names into a dataframe.
- 2. For each line of the dataframe.
- a. Read the resume file name.
- b. Import the PDF file.
- c. Transform the PDF into text and append it as a column of the dataframe.
- d. Process the text and append the list of words as a column of the dataframe.
- e. Calculate the number of words in each resume and append it as a column to the dataframe.
- 3. Create a set including all the words in the resumes.
- 4. Count how many times each word appears.
- 5. Sort words by count in descending order.
- 6. Create a dictionary of vocabulary words using number to word matching based on previous sorting.
- PDF to Text Transformation. In some embodiments, the following process can be used to transform the pdf document to text.
-
- 1. Create a list of words (e.g., English, French, any language) present in all the resumes we are going to examine. Each word can be called a “token”.
- 2. The resumes used as input can be stored in PDF, which can be the format candidates use to submit the resume in order to apply to a job. Because the PDF format may not allow immediate token identification and manipulation, we can transform the PDF documents into text documents using the example function set forth in
FIG. 5 which takes a PDF document as input and returns a text version of the document as output. The function inFIG. 5 can be re-used at any stage of the process to transform a PDF into text. For example, this process can be used on a batch of resumes being used to train the model. This function can also be used when a new pdf resume is received and checked to see if it is good (e.g., a desirable candidate). In both these cases pdf files need to be transformed into text. Of course, if a resume is received in a format other than PDF, it can be easily converted into PDF using standard PDF technology and then the PDF document can be transformed into text.
- Tokenization.
-
- 1. Once the PDF documents have been transformed into text, the tokens (e.g., words) can be extracted.
FIG. 6A illustrates an example function where a text file is input, and a set of tokens are output. The text can be split into words, sentences, and/or paragraphs using a set of regular expressions and character based rules (e.g., using Regex). This function can be re-used at any stage of the process to tokenize text. For example, this function can be used on a batch of resumes being used to train the model, but can also be used when a new pdf resume is received and is being checked for the probability of the resume being a good one. In both cases, the text can be tokenized. - 2. Once we have the tokens, we can count how many times each word appears in the corpus (e.g., the set of documents considered) as shown in
FIG. 6B . - 3. When we have the count of how many times each word appears in the corpus (e.g., the body of resumes), we can sort those words in decreasing order (e.g., the words with the highest count appear first) as shown in
FIG. 6C and we can assign each word their ranking number (e.g., index) in this special sorted list. If you have two words appearing the exact same number of times, in some embodiments, they can be given subsequent numbers in alphabetical order.
- 1. Once the PDF documents have been transformed into text, the tokens (e.g., words) can be extracted.
- Training. The training procedure can assume there is a set of resume PDF documents that have been stored in two different folders. Folder “yes” can include candidates that have been deemed to be successful in the past after a pre-screening hiring round by a hiring team of person(s). Folder “no” can include candidates who did not pass the pre-screening hiring round by the hiring team of person(s). In some embodiment, the following process can be used, For example, using Python as the coding language, with a Keras library and a TensorFlow backend.
-
- 1. Import resume file names from the “yes'” folder into a dataframe.
- 2. Create a column “proceed”.
- 3. For each resume append the
value 1 in the column “proceed”. - 4. Import resume file names from the “no” folder into a dataframe.
- 5. For each resume append the
value 0 in the column “proceed”. - 6. For each line of the dataframe:
- a. Read the resume file name.
- b. Import the PDF file.
- c. Transform the PDF into text and append it as a column of the dataframe.
- d. Process the text and append the list of words as a column of the dataframe.
- e. Transform list of word into a list of indexes using the pre-built vocabulary and append it to the dataframe.
- 7. Split dataset into train and validation randomly (e.g., using known machine learning techniques).
- 8. Augment data by using a sentence and/or paragraph shuffling procedure. This can be done only for the train dataset in some embodiments.
- 9. Append the additional resumes created in the previous step to the training dataset.
- 10. Pre-process data to feed Deep Neural networks.
- 11. Train Deep Neural networks.
- Resume Import and Processing. The example function of
FIG. 7A can be used to import resumes and mark them as “good” (=1) or “bad” (=0). We can then proceed to transform the resume into a list of indexes using the vocabulary. This can be done using an example procedure join_cv shown inFIG. 7B , which can use a supporting function named encode_cv. Once we have transformed the resume into a list of word indexes based on the predetermined vocabulary, as shown inFIG. 7C , we can proceed with a typical machine learning step of splitting our data into: 1) a training dataset, which can be used to determine the model parameters; and 2) a validation dataset, which can be used to test the generalization of the model. Once we have determined the training set, we can proceed with the data augmentation technique. - Data Augmentation. Because machines learn from examples, it is quite helpful to have more examples (e.g., hundreds or thousands or more). By using the fact that sentences resumes are quite independent, we can create “new” example resumes by shuffling sentences belonging to the resume around. This technique can improve the prediction power of the model. For example, in
FIG. D1 , the text document for the resume can be split into segments by finding the end of the paragraph and/or sentence (e.g., in the code inFIG. 8 , we refer to these segments as “sentences”), which can be designated in the text by a return, which can be designated in the text as “\n”. We can then use the Python instruction ‘random.shuffle( )’ to shuffle the sentences around. The shuffled sentences can then be joined together to create new example resumes that can help teach the computer. For example, if we found 7 sentences in a particular resume, we could order those S1 to S7, and then shuffle them randomly so that we create several more example resumes. Thus, the original resume could be ordered S1, S2, S3, S4, S5, S6, and S7. After shuffling and joining together the shuffled sentences, we could have several other example resumes made of the same sentences. An example could be the resume including sentences in the following order: S2, S4, S7, S5, S1, S3, and S6. Once we have the new resumes, we can turn them into tokens and encode them. This process can be helpful when treating resumes as these documents usually include sentences and/or paragraphs and/or other portions that are independent from each other. The resulting ‘shuffled’ resumes are still resumes. - Deep Neural Network. We are now ready to do some pre-processing of the encoded resumes before feeding them to our Deep Neural Network.
- A deep neural network (DNN) is an ANN (Artificial Neural Network) with multiple hidden layers between the input and output layers.
- https://en.wikipedia.org/wiki/Deep_learning#Deep_neural_networks.
FIG. 9A is an example of pre-processing that comprises: eliminating non-frequent words, padding the text to make all resume the same length, removing resumes that are too short, etc. At this point, we can create the architecture of our deep neural network and start training it. For example, embeddings can be used so that word indexes can be mapped to random vectors of a fixed size (e.g. 100, 200, 300). Those vectors can get their numbers fixed during the training procedure.FIG. 9B illustrates an example of a Deep Neural Network using convolutional layers and word embeddings written in Keras. The embeddings value can create a matrix of a number of words in the resume×size of the embedding vector. - The following code represents a Deep Neural Network written in Keras
- Each row is a layer of the network gets input from the previous layer, applies the layer specific transformations and provides its output as the following layer input.
-
Specific trasformations are: Embedding(vocab_size+1, 100, input_length=seq_len), Transforms the resumes into a matrix with 100 columns Dropout(0.2), Randomly deletes 20% of the output of the embedding layer Convolution1D(12, 3, padding=‘same’, activation=‘relu’), Applies a convolution operation to the output of the dropout layer and then applies a relu (rectified linear unit) function to it Dropout(dof), See above MaxPooling1D( ), Takes the maximum values of the output provided by the dropout layer Convolution1D(12, 3, padding=‘same’, activation=‘relu’), See above Dropout(dof), See above MaxPooling1D( ), See above Flatten( ), Transforms the output of the maxpooling layer into a long flat vector to be fed into the dense layer Dense(10, activation=‘relu’), Transforms the output of flatten layer and condenses it into 10 positive numbers Dropout(0.7), See above Dense(1, activation=‘sigmoid’)]) - Transforms the output of dropout into 1 positive between 0 and 1 which represents the probability of the resume to be a ‘good’ one.
FIG. 9C illustrates an example transformation of a resume sentence into a set of embeddings using a word index vocabulary. The embeddings values can be initialized randomly. The Deep Neural Network algorithm can find the ones that best fit our proposed purpose during the training procedure using an optimization algorithm such as, for example, the Stochastic Gradient Descent. - Stochastic gradient descent (often shortened to SGD), also known as incremental gradient descent, is a stochastic approximation of the gradient descent optimization and iterative method for minimizing an objective function that is written as a sum of differentiable functions.
- https://en.wikipedia.org/wiki/Stochastic_gradient_descent. When we are finished training the model, we can save both the embeddings value and the model multipliers into a file.
- Predicting Candidate “Goodness”. To predict how a new candidate will perform during pre-screening steps, we can load the model file and pass to it the PDF to get a prediction. The prediction process set forth below can be called scoring and for each resume it can return the predicted probability of the candidate passing an HR department's pre-screening tests.
-
- 1. Import resume into a dataframe
- 2. For each line of the dataframe.
- a. Read the resume file name.
- b. Import the PDF file.
- c. Transform the PDF into text and append it as a column of the dataframe.
- d. Process the text and append the list of words as a column of the dataframe.
- e. Transform list of word into a list of indexes using the pre-built vocabulary and append it to the dataframe.
- f. Apply same pre-processing steps as was done in the Deep Neural Network section.
- 3. Load model.
- 4. Feed resume to model.
- 5. Get prediction.
- 6. Output prediction to .csv file
- Once the PDF has been transformed into a sequence of indexes and pre-processed as described above, the scoring procedure comes from multiple transformations of the word vectors using model parameters found during training for the neural network functions. In very simplified terms, this transformation can be represented as y(x)=f(x), where: y can be a predicted probability of a candidate being “good”; f can be the set of parametrized transformations happening as result of optimized neural network parameters determined by the Deep Neural Network algorithm during the model training procedure; and x can be the encoded resume (which can be translated into embeddings during the scoring procedure).
-
predictions=NN.predict(X_cand, verbose=1) - The output can then reshaped to a nice .cvs format as illustrated in
FIG. 10 . - While various embodiments have been described above, it should be understood that they have been presented by way of example and not limitation. It will be apparent to persons skilled in the relevant art(s) that various changes in form and detail can be made therein without departing from the spirit and scope. In fact, after reading the above description, it will be apparent to one skilled in the relevant art(s) how to implement alternative embodiments. For example, other steps may be provided, or steps may be eliminated, from the described flows, and other components may be added to, or removed from, the described systems. Accordingly, other implementations are within the scope of the following claims.
- In addition, it should be understood that any figures which highlight the functionality and advantages are presented for example purposes only. The disclosed methodology and system are each sufficiently flexible and configurable such that they may be utilized in ways other than that shown.
- Although the term “at least one” may often be used in the specification, claims and drawings, the terms “a”, “an”, “the”, “said”, etc. also signify “at least one” or “the at least one” in the specification, claims and drawings.
- Finally, it is the applicant's intent that only claims that include the express language “means for” or “step for” be interpreted under 35 U.S.C. 112(f). Claims that do not expressly include the phrase “means for” or “step for” are not to be interpreted under 35 U.S.C. 112(f).
Claims (1)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18/045,308 US20230068717A1 (en) | 2018-01-24 | 2022-10-10 | Methods and systems for determining relevance of documents |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201862621404P | 2018-01-24 | 2018-01-24 | |
US16/203,102 US10789460B1 (en) | 2018-01-24 | 2018-11-28 | Methods and systems for screening documents |
US16/891,396 US11521402B2 (en) | 2018-01-24 | 2020-06-03 | Methods and systems for determining relevance of documents |
US18/045,308 US20230068717A1 (en) | 2018-01-24 | 2022-10-10 | Methods and systems for determining relevance of documents |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/891,396 Continuation US11521402B2 (en) | 2018-01-24 | 2020-06-03 | Methods and systems for determining relevance of documents |
Publications (1)
Publication Number | Publication Date |
---|---|
US20230068717A1 true US20230068717A1 (en) | 2023-03-02 |
Family
ID=72423726
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/203,102 Active US10789460B1 (en) | 2018-01-24 | 2018-11-28 | Methods and systems for screening documents |
US16/891,396 Active 2039-02-22 US11521402B2 (en) | 2018-01-24 | 2020-06-03 | Methods and systems for determining relevance of documents |
US18/045,308 Pending US20230068717A1 (en) | 2018-01-24 | 2022-10-10 | Methods and systems for determining relevance of documents |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/203,102 Active US10789460B1 (en) | 2018-01-24 | 2018-11-28 | Methods and systems for screening documents |
US16/891,396 Active 2039-02-22 US11521402B2 (en) | 2018-01-24 | 2020-06-03 | Methods and systems for determining relevance of documents |
Country Status (1)
Country | Link |
---|---|
US (3) | US10789460B1 (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11748571B1 (en) * | 2019-05-21 | 2023-09-05 | Educational Testing Service | Text segmentation with two-level transformer and auxiliary coherence modeling |
US11636411B2 (en) * | 2019-08-12 | 2023-04-25 | Wynden Stark Llc | Apparatus for determining role fitness while eliminating unwanted bias |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8180633B2 (en) * | 2007-03-08 | 2012-05-15 | Nec Laboratories America, Inc. | Fast semantic extraction using a neural network architecture |
US20150019204A1 (en) * | 2013-07-12 | 2015-01-15 | Microsoft Corporation | Feature completion in computer-human interactive learning |
US20170357896A1 (en) * | 2016-06-09 | 2017-12-14 | Sentient Technologies (Barbados) Limited | Content embedding using deep metric learning algorithms |
US20180373696A1 (en) * | 2015-01-23 | 2018-12-27 | Conversica, Inc. | Systems and methods for natural language processing and classification |
US10803380B2 (en) * | 2015-09-11 | 2020-10-13 | Google Llc | Generating vector representations of documents |
US20210049536A1 (en) * | 2019-08-12 | 2021-02-18 | Wynden Stark LLC dba GQR Global Markets | Apparatus for Determining Role Fitness While Eliminating Unwanted Bias |
US20230129874A1 (en) * | 2019-11-15 | 2023-04-27 | Intuit Inc. | Pre-trained contextual embedding models for named entity recognition and confidence prediction |
Family Cites Families (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6137911A (en) * | 1997-06-16 | 2000-10-24 | The Dialog Corporation Plc | Test classification system and method |
US7028250B2 (en) * | 2000-05-25 | 2006-04-11 | Kanisa, Inc. | System and method for automatically classifying text |
AU2003201799A1 (en) * | 2002-01-16 | 2003-07-30 | Elucidon Ab | Information data retrieval, where the data is organized in terms, documents and document corpora |
US20070033064A1 (en) * | 2004-02-27 | 2007-02-08 | Abrahamsohn Daniel A A | Method of and system for capturing data |
US20070203776A1 (en) * | 2005-12-28 | 2007-08-30 | Austin David J | Method of displaying resume over the internet in a secure manner |
US20110302100A1 (en) * | 2010-06-07 | 2011-12-08 | Forte Hcm Inc. | Method and system for identification of talent on the basis of a target candidate profile |
US20120166922A1 (en) * | 2010-12-28 | 2012-06-28 | Franklin Keith Rolles | Content Management System for Resume and Portfolio Data for Producing Multiple Interactive Websites |
US8892488B2 (en) * | 2011-06-01 | 2014-11-18 | Nec Laboratories America, Inc. | Document classification with weighted supervised n-gram embedding |
US9430563B2 (en) * | 2012-02-02 | 2016-08-30 | Xerox Corporation | Document processing employing probabilistic topic modeling of documents represented as text words transformed to a continuous space |
US20140108308A1 (en) * | 2012-07-13 | 2014-04-17 | Social Data Technologies, LLC | System and method for combining data for identifying compatibility |
EP2877956B1 (en) * | 2012-07-24 | 2019-07-17 | Webroot Inc. | System and method to provide automatic classification of phishing sites |
US20140122355A1 (en) * | 2012-10-26 | 2014-05-01 | Bright Media Corporation | Identifying candidates for job openings using a scoring function based on features in resumes and job descriptions |
US20140250024A1 (en) * | 2013-03-04 | 2014-09-04 | Murali G Takalpati | Formatting system and method for producing a standardized resume |
US20180053114A1 (en) * | 2014-10-23 | 2018-02-22 | Brighterion, Inc. | Artificial intelligence for context classifier |
WO2015198317A1 (en) * | 2014-06-23 | 2015-12-30 | Intervyo R&D Ltd. | Method and system for analysing subjects |
US20170287356A1 (en) * | 2014-09-26 | 2017-10-05 | Accessible Publishing Systems Pty Ltd | Teaching systems and methods |
US20190179903A1 (en) * | 2015-01-23 | 2019-06-13 | Conversica, Inc. | Systems and methods for multi language automated action response |
US9898458B2 (en) * | 2015-05-08 | 2018-02-20 | International Business Machines Corporation | Generating distributed word embeddings using structured information |
US20170154314A1 (en) * | 2015-11-30 | 2017-06-01 | FAMA Technologies, Inc. | System for searching and correlating online activity with individual classification factors |
US10643182B2 (en) * | 2016-03-16 | 2020-05-05 | Oracle International Corporation | Resume extraction based on a resume type |
US20180032611A1 (en) * | 2016-07-29 | 2018-02-01 | Paul Charles Cameron | Systems and methods for automatic-generation of soundtracks for live speech audio |
US9715495B1 (en) * | 2016-12-15 | 2017-07-25 | Quid, Inc. | Topic-influenced document relationship graphs |
US10607273B2 (en) * | 2016-12-28 | 2020-03-31 | Google Llc | System for determining and displaying relevant explanations for recommended content |
US10380259B2 (en) * | 2017-05-22 | 2019-08-13 | International Business Machines Corporation | Deep embedding for natural language content based on semantic dependencies |
US11048879B2 (en) * | 2017-06-19 | 2021-06-29 | Vettd, Inc. | Systems and methods to determine and utilize semantic relatedness between multiple natural language sources to determine strengths and weaknesses |
WO2019075406A1 (en) * | 2017-10-13 | 2019-04-18 | Choositoi Inc. | Reading level based text simplification |
US10699321B2 (en) * | 2017-10-17 | 2020-06-30 | Adobe Inc. | Global vector recommendations based on implicit interaction and profile data |
WO2019108793A1 (en) * | 2017-11-29 | 2019-06-06 | John Maclaren Walsh | Recommender methods and systems for patent processing |
-
2018
- 2018-11-28 US US16/203,102 patent/US10789460B1/en active Active
-
2020
- 2020-06-03 US US16/891,396 patent/US11521402B2/en active Active
-
2022
- 2022-10-10 US US18/045,308 patent/US20230068717A1/en active Pending
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8180633B2 (en) * | 2007-03-08 | 2012-05-15 | Nec Laboratories America, Inc. | Fast semantic extraction using a neural network architecture |
US20150019204A1 (en) * | 2013-07-12 | 2015-01-15 | Microsoft Corporation | Feature completion in computer-human interactive learning |
US20180373696A1 (en) * | 2015-01-23 | 2018-12-27 | Conversica, Inc. | Systems and methods for natural language processing and classification |
US10803380B2 (en) * | 2015-09-11 | 2020-10-13 | Google Llc | Generating vector representations of documents |
US20170357896A1 (en) * | 2016-06-09 | 2017-12-14 | Sentient Technologies (Barbados) Limited | Content embedding using deep metric learning algorithms |
US20210049536A1 (en) * | 2019-08-12 | 2021-02-18 | Wynden Stark LLC dba GQR Global Markets | Apparatus for Determining Role Fitness While Eliminating Unwanted Bias |
US20230129874A1 (en) * | 2019-11-15 | 2023-04-27 | Intuit Inc. | Pre-trained contextual embedding models for named entity recognition and confidence prediction |
Also Published As
Publication number | Publication date |
---|---|
US10789460B1 (en) | 2020-09-29 |
US20200293771A1 (en) | 2020-09-17 |
US11521402B2 (en) | 2022-12-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Dasigi et al. | A dataset of information-seeking questions and answers anchored in research papers | |
US20230068717A1 (en) | Methods and systems for determining relevance of documents | |
CA3129721C (en) | Pre-trained contextual embedding models for named entity recognition and confidence prediction | |
JP7282940B2 (en) | System and method for contextual retrieval of electronic records | |
Bhatia et al. | End-to-end resume parsing and finding candidates for a job description using bert | |
Long et al. | World knowledge for reading comprehension: Rare entity prediction with hierarchical lstms using external descriptions | |
US11461415B2 (en) | Assessing semantic similarity using a dual-encoder neural network | |
JP2023553121A (en) | Document classification using domain-specific natural language processing models | |
Huang et al. | Character-level convolutional network for text classification applied to chinese corpus | |
CN110955766A (en) | Method and system for automatically expanding intelligent customer service standard problem pairs | |
WO2021190662A1 (en) | Medical text sorting method and apparatus, electronic device, and storage medium | |
US11983506B2 (en) | Hybrid translation system using a general-purpose neural network machine translator | |
CN114218392A (en) | Futures question-answer oriented user intention identification method and system | |
Badawi | Using multilingual bidirectional encoder representations from transformers on medical corpus for Kurdish text classification | |
Andriyanov | Combining Text and Image Analysis Methods for Solving Multimodal Classification Problems | |
Siddique et al. | Bilingual word embeddings for cross-lingual personality recognition using convolutional neural nets | |
Tian et al. | Chinese short text multi-classification based on word and part-of-speech tagging embedding | |
Luz de Araujo et al. | Sequence-aware multimodal page classification of Brazilian legal documents | |
Fonseca Cacho et al. | OCR post processing using support vector machines | |
Cacho | Improving OCR post processing with machine learning tools | |
Reusch et al. | Transformer-encoder-based mathematical information retrieval | |
Fonseca Cacho | Improving OCR post processing with machine learning tools | |
WO2021075998A1 (en) | System for classifying data in order to detect confidential information in a text | |
Mathew et al. | ATS BREAKER-A system for comparing candidate resume and company requirements | |
KR102601932B1 (en) | System and method for extracting data from document for each company using fingerprints and machine learning |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: THE BOSTON CONSULTING GROUP, INC., MASSACHUSETTS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LAPIELLO, EMILIO ALFREDO;REEL/FRAME:062314/0640 Effective date: 20190510 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |