JP2020113218A - Information processing device, display method, and program - Google Patents

Abstract

To provide an information processing device, a display method and a program for presenting information that enables what particulars affect the evaluation of a person to be grasped.SOLUTION: An information processing device (evaluation device 20) comprises: a learning model storage unit 220 for storing at least nonstandard data in text format and standard data in other than text format pertaining to a person; an evaluation information storage unit 232 for storing evaluation on a person; a learning processing unit 211 for creating a first prediction model by machine learning in which word data relating to a word included in the nonstandard data is input data and evaluation is teaching data, and creating a second prediction model by machine learning in which a calculated value calculated by applying the nonstandard data to the first prediction model and the standard data are input data; and a contribution degree output unit (important word output unit 213) for extracting the degree of contribution to evaluation per nonstandard data and standard data from the second prediction model and outputting the extracted degree of contribution.SELECTED DRAWING: Figure 4

Description

The present invention relates to an information processing device, a display method and a program.

People are evaluated in recruitment, employee management, and health management. Patent Document 1 discloses that a person image is evaluated based on words in an entry sheet that are similar to evaluation factors (enthusiasm, leadership, etc.).

JP, 2018-116710, A

However, in the technique described in Patent Document 1, the evaluation of the applicant for employment is to be determined depending on whether or not a word related to a predetermined factor is described, and the factor actually affects the adoption. Not necessarily.

The present invention has been made in view of such a background, and an object of the present invention is to provide a technique capable of grasping what kind of item influences a person's evaluation.

A main invention of the present invention for solving the above-mentioned problems is an information processing apparatus, and a learning data storage unit that stores at least non-standard data in text format and standard data other than text format related to a person, and the person. A first prediction model by machine learning using word data relating to words included in the atypical data as input data and machine learning using the evaluation as teacher data, and the first prediction. A learning processing unit that creates a second prediction model by machine learning using a calculation value calculated by giving the atypical data to a model and the atypical data as input data; and the atypical data from the second prediction model. And a contribution degree output unit that extracts the contribution degree to the evaluation for each of the standard data and outputs the extracted contribution degree.

Other problems disclosed by the present application and a method for solving the problems will be clarified by the section of the embodiments of the invention and the drawings.

According to the present invention, it is possible to understand what kind of matter affects the evaluation of a person.

It is a figure showing an example of whole composition of an evaluation system which is one embodiment of the present invention. It is a figure explaining the flow about the example of the learning process by the evaluation device 20. It is a figure explaining the flow about the example of the learning process by the evaluation device 20. 3 is a diagram showing a software configuration example of the evaluation device 20. FIG. It is a figure which shows the structural example of the application data memorize|stored in the application data storage part 231. FIG. 6 is a diagram showing a configuration example of evaluation information stored in an evaluation information storage unit 232. It is a figure which shows the flow of the learning process which concerns on CNN221. It is a figure which shows the output example of an important word. It is a figure which shows the flow of the learning process which concerns on the machine learning model 222. It is a figure which shows the example of an output of the contribution degree of each item. It is a figure which shows the flow of the prediction process about the employability of an evaluation object person. It is a figure which shows the software structural example of the evaluation apparatus 20 provided with the contribution storage part 233 which memorize|stores the contribution of each item and each word. It is a figure which shows the structural example of the contribution storage part 233. It is a figure which shows the structural example of the survey data used instead of application data when learning the possibility of employee's retirement. It is a figure which shows the structural example of the evaluation information in the case of learning the possibility of employee's retirement.

The contents of the embodiments of the present invention will be listed and described. The device according to the embodiment of the present invention has the following configuration.

[Item 1]
A learning data storage unit that stores at least non-standard data in text format and standard data in non-text format related to a person,
An evaluation information storage unit that stores the evaluation of the person,
A prediction result in which a first prediction model is created by machine learning using word data relating to words included in the atypical data as input data and the evaluation as teacher data, and the atypical data is given to the first prediction model. And a learning processing unit that creates a second prediction model by machine learning using the standard data as input data,
A contribution degree output unit that extracts a contribution degree to the evaluation for each of the atypical data and the boilerplate data from the second prediction model, and outputs the extracted contribution degree;
An information processing apparatus comprising:
[Item 2]
The information processing apparatus according to item 1,
Further comprising a word contribution output unit that extracts the contribution of each word to the evaluation from the first prediction model and outputs the extracted contribution of each word.
An information processing device characterized by:
[Item 3]
The information processing apparatus according to item 1,
The learning processing unit creates at least the first prediction model using a convolutional neural network;
An information processing device characterized by:
[Item 4]
The information processing apparatus according to item 3,
The method further comprises a word contribution output unit that extracts the contribution of each word to the evaluation from the first prediction model by class activation mapping for the convolutional neural network and outputs the extracted contribution of each word. thing,
An information processing device characterized by:
[Item 5]
The information processing apparatus according to any one of Items 1 to 4,
An evaluation data acquisition unit that acquires the atypical data and the atypical data created by an evaluation subject,
An evaluation prediction unit that predicts the evaluation of the evaluation target person based on the atypical data, the fixed data, and the first and second prediction models,
An evaluation output unit that outputs the predicted evaluation,
An information processing apparatus, further comprising:
[Item 6]
A learning data storage unit that stores at least non-standard data in text format and standard data in non-text format related to a person,
A prediction model storage unit that stores a prediction model for predicting an evaluation for the person based on the atypical data and the atypical data,
A contribution degree output unit that extracts the contribution degree for each of the atypical data and the fixed form data from the prediction model, and outputs the extracted contribution degree,
An information processing apparatus comprising:
[Item 7]
The information processing apparatus according to item 6,
For a word included in the atypical data, further comprising a word contribution output unit that extracts the contribution of each word to the evaluation from the prediction model, and outputs the extracted contribution of each word,
An information processing device characterized by:
[Item 8]
The information processing apparatus according to item 7,
The word contribution output unit classifies and outputs the word that contributes to increase the evaluation and the word that contributes to decrease the evaluation,
An information processing device characterized by:
[Item 9]
Computer
A step of storing at least text-type atypical data relating to a person and non-text-type atypical data;
Storing a rating for the person,
Creating a first prediction model by machine learning using word data relating to words included in the atypical data as input data and the evaluation as teacher data;
Creating a second predictive model by machine learning using the calculated value obtained by giving the atypical data to the first predictive model and the standard data as input data, and using the evaluation as teacher data,
Extracting the contribution to the evaluation for each of the atypical data and the atypical data from the second prediction model,
Displaying the extracted contribution,
How to display.
[Item 10]
The display method according to item 9,
The computer is further
A method of extracting the contribution of each word to the evaluation from the first prediction model, and displaying the extracted contribution of each word.
[Item 11]
The display method according to item 9,
The computer creates at least the first predictive model using a convolutional neural network;
Display method characterized by.
[Item 12]
The display method according to claim 11,
The computer is further
A step of extracting the contribution degree of each word to the evaluation from the first prediction model by class activation mapping for the convolutional neural network, and displaying the extracted contribution degree of each word. Information processing device.
[Item 13]
The display method according to any one of Items 9 to 12,
The computer is further
Acquiring the atypical data and the atypical data created by an evaluation subject,
Predicting the evaluation of the evaluation subject based on the atypical data, the typical data, and the first and second prediction models;
Displaying the predicted assessment,
Display method characterized by executing.
[Item 14]
Computer
A step of storing at least text-type atypical data relating to a person and non-text-type atypical data;
Storing a prediction model for predicting an evaluation for the person based on the atypical data and the atypical data;
Extracting the contribution of each of the atypical data and the boilerplate data from the prediction model, and displaying the extracted contribution.
Display method characterized by executing.
[Item 15]
The display method according to item 14,
The computer is further
A display method comprising: performing a step of extracting a contribution degree of each word to the evaluation from the prediction model for a word included in the atypical data and outputting the extracted contribution degree of each word.
[Item 16]
The display method according to item 15,
The computer classifies and outputs the word that contributes to increase the evaluation and the word that contributes to decrease the evaluation,
Display method characterized by.
[Item 17]
On the computer,
A step of storing at least text-type atypical data relating to a person and non-text-type atypical data;
Storing a rating for the person,
Creating a first prediction model by machine learning using word data relating to words included in the atypical data as input data and the evaluation as teacher data;
Creating a second predictive model by machine learning using the calculated value obtained by giving the atypical data to the first predictive model and the standard data as input data, and using the evaluation as teacher data,
Extracting the contribution to the evaluation for each of the atypical data and the atypical data from the second prediction model,
Displaying the extracted contribution,
A program to execute.
[Item 18]
The program according to item 17,
Further on the computer,
A program for executing a step of extracting a contribution degree of each word to the evaluation from the first prediction model and displaying the extracted contribution degree of each word.
[Item 19]
The program according to item 17,
Causing the computer to create at least the first predictive model using a convolutional neural network;
A program characterized by.
[Item 20]
The program according to item 19,
Further on the computer,
A program for executing a step of extracting the contribution degree of each word to the evaluation from the first prediction model by class activation mapping for the convolutional neural network, and displaying the extracted contribution degree of each word.
[Item 21]
The program according to any one of items 18 to 20,
Further on the computer,
Acquiring the atypical data and the atypical data created by an evaluation subject,
Predicting the evaluation of the evaluation subject based on the atypical data, the typical data, and the first and second prediction models;
Displaying the predicted assessment,
A program to execute.
[Item 22]
On the computer,
A step of storing at least text-type atypical data relating to a person and non-text-type atypical data;
Storing a prediction model for predicting an evaluation for the person based on the atypical data and the atypical data;
Extracting the contribution of each of the atypical data and the boilerplate data from the prediction model, and displaying the extracted contribution.
A program to execute.
[Item 23]
The program according to item 22,
Further on the computer,
A program for executing a step of extracting a contribution degree of each word to the evaluation from the prediction model for a word included in the atypical data and outputting the extracted contribution degree of each word.
[Item 24]
The program according to item 23,
Causing the computer to classify and output the words that contribute to increase the evaluation and the words that contribute to decrease the evaluation,
A program characterized by.
[Item 25]
A display method for outputting the degree of contribution to an evaluation of a person of an item including a plurality of words, which is composed of non-standard data in a text format,
Display the contribution to evaluation for a person for each item of multiple items,
A display method of displaying, for each of one or more words included in each item, the degree of contribution to the evaluation.
[Item 26]
The display method according to item 25,
Accepting a selection request from the user for one of the plurality of items,
A display method of displaying, for each of one or more words included in the one item, the degree of contribution to the evaluation.
[Item 27]
The display method according to item 25,
The display method, wherein the contribution degree displayed for each of the plurality of words included in the one item is a breakdown of the contribution degree displayed for each item of the plurality of items.
[Item 28]
The display method according to item 25,
Among the words included in the one item, a word having a predetermined contribution is displayed.
[Item 29]
The display method according to item 25,
A method of displaying, wherein the degree of contribution to an evaluation of a person is displayed together with the degree of contribution to an evaluation of a person for each item of the plurality of items, the item being composed of fixed-form data other than the text.

Hereinafter, an evaluation support system including the evaluation device 20 (information processing device) according to the embodiment of the present invention will be described. FIG. 1 is a diagram showing an example of the overall configuration of the evaluation support system of this embodiment. As shown in the figure, the evaluation device 20 is communicatively connected to the user terminal 10 via the communication network 30. The communication network 30 is constructed by, for example, a public telephone line network, a mobile telephone line network, a dedicated line network, Ethernet (registered trademark), a wireless communication path, or the like. In the present embodiment, the communication network 30 is assumed to be the Internet, but a LAN (Local Area Network) or various WANs (Wide Area Networks) can also be adopted as the communication network 30.

The user terminal 10 is a computer operated by a user who uses the person evaluation service provided by the evaluation device 20. The user terminal 10 is, for example, a personal computer, a smartphone, a tablet computer, or the like. As shown in FIG. 1, the user terminal 10 includes a CPU 101, a memory 102, a storage device 103, a communication interface 104, an input device 105, and an output device 106. The storage device 103 is, for example, a hard disk drive, a solid state drive, a flash memory, or the like that stores various data and programs. The communication interface 104 is an interface for connecting to the communication network 30. For example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone line network, and a wireless communication device for performing wireless communication. , A USB (Universal Serial Bus) connector for serial communication, an RS232C connector, and the like. The input device 105 is, for example, a keyboard, a mouse, a touch panel, a button, a microphone or the like for inputting data. The output device 106 is, for example, a display, a printer, a speaker, or the like that outputs data. The user terminal 10 can execute a Web browser and access the evaluation device 20 by HTTP (HyperText Transfer Protocol) or the like to use the person evaluation service.

The evaluation device 20 is a computer that provides a person evaluation service. The evaluation device 20 may be a physical computer such as a personal computer, a workstation, a smartphone, or a tablet computer, or a virtual computer provided by cloud computing. As shown in FIG. 1, the evaluation device 20 includes a CPU 201, a memory 202, a storage device 203, a communication interface 204, an input device 205, and an output device 206. The storage device 203 is, for example, a hard disk drive, a solid state drive, a flash memory, or the like that stores various data and programs. The communication interface 204 is an interface for connecting to the communication network 30. For example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone line network, and a wireless communication device for performing wireless communication. , A USB (Universal Serial Bus) connector for serial communication, an RS232C connector, and the like. The input device 205 is, for example, a keyboard, a mouse, a touch panel, a button, a microphone or the like for inputting data. The output device 206 is, for example, a display, a printer, a speaker, or the like that outputs data.

The evaluation device 20 evaluates a person. In the present embodiment, as a specific example, it is assumed that the hiring suitability of a new graduate hiring candidate is determined. The evaluation device 20 of the present embodiment uses not only numerical information (standard data) such as the result of the SPI test but also text data (non-standard data) such as an entry sheet as input data, and the actual acceptance/rejection as teacher data. It is intended to create a prediction model by learning and use this prediction model to determine whether or not a person who wants to be adopted will be adopted or rejected. In this embodiment, convolutional neural networks (CNN) and gradient boost tree algorithms are used for machine learning, but the invention is not limited to these.

2 and 3 are diagrams illustrating the flow of an example of the learning process performed by the evaluation device 20.

First, the evaluation device 20 divides text data such as an entry sheet created by an applicant for employment into words by processing such as morphological analysis, and vectorizes the divided words. A known method such as Word2Vec can be used for vectorization of words. The evaluation device 20 learns vectorized text data by CNN as shown in FIG. It should be noted that, as the teacher data, a binary value indicating whether it has been adopted (passed) or not (failed) can be given to predict the possibility of passing. When predicting an evaluation that is not pass or fail, it is also possible to give an evaluation value of a numerical value or a category value as teacher data for learning.

Next, the evaluation device 20 uses a method of class activation map (CAM) to focus on the word focused on in the convolution processing by CNN, that is, the word that is determined to be important for the pass determination (hereinafter, important A word). CAM is generally used in image recognition processing to obtain a heat map of a spot of interest on an image. However, by applying it to vectorized text data, CAM is judged to be important in learning processing in CNN. It is possible to specify the word that has been written. By providing this important word to the user, the evaluation device 20 of the present embodiment finds out what kind of word the text data used contributes to the evaluation (for example, pass or adoption) of the applicant. Can be obtained. The evaluation device 20 may extract words that are determined to be important by a method other than CAM. For example, when using a recurrent neural network (RNN), words may be extracted according to the attention weight of the RNN, or a multiple regression equation using each word (vector) as an explanatory variable is estimated. In this case, important words may be extracted according to the coefficient (weight) of each word.

FIG. 4 is a diagram illustrating a software configuration example of the evaluation device 20. As shown in the figure, the evaluation device 20 includes a learning processing unit 211, an important item output unit 212, an important word output unit 213, a numerical data acquisition unit 214, a text data acquisition unit 215, a prediction processing unit 216, and an evaluation output unit 217. The model storage unit 220, the application data storage unit 231, and the evaluation information storage unit 232.

The evaluation processing device 211, the important item output unit 212, the important word output unit 213, the numerical data acquisition unit 214, the text data acquisition unit 215, the prediction processing unit 216, and the evaluation output unit 217 are functional units of the evaluation device 20. This is realized by the CPU 201 provided therein by reading the program stored in the storage device 203 into the memory 202 and executing the program. The storage units of the model storage unit 220, the application data storage unit 231, and the evaluation information storage unit 232 are the evaluation unit 20. It is realized as a part of the storage area provided by the memory 202 and the storage device 203 included in.

The application data storage unit 231 (corresponding to the learning data storage unit of the present invention) stores data regarding applicants for hiring (hereinafter referred to as application data). The application data storage unit 231 stores application data relating to past applicants for employment. FIG. 5 is a diagram showing a configuration example of application data stored in the application data storage unit 231. The application data is associated with the identification information (user ID) of the applicant for employment, the type of school from which the applicant for employment is selected (university, high school, etc.), distinction between humanities and sciences (literary science), and the aspiration entered on the entry sheet. This includes motivation (motivation for ES), self-appeal written on the entry sheet (ES self-appeal), experience of part-time job, time spent working part-time, score and rank (stage) of SPI test. In the example of FIG. 5, the ES desire motive and the ES self-appeal are text data, and the other items are standard data, that is, numerical information or category information. The items of the application data are not limited to the example of FIG. 5, and can include arbitrary fixed and atypical data. Further, the application data may include not only the text data created by the applicant for hiring but also comments from the school teachers or staff who were attending school, comments from the employees who interviewed, and the like. Regarding the text data, the entire text may be included in the application data, or the file name (path) indicating the file in which the text data is stored may be included in the application data.

The evaluation information storage unit 232 stores information (hereinafter, referred to as evaluation information) indicating the evaluation of the applicant for adoption. FIG. 6 is a diagram showing a configuration example of the evaluation information stored in the evaluation information storage unit 232. As shown in the figure, in the evaluation information, a pass/fail result indicating whether the hiring candidate is adopted (passed) or not adopted (failed) is associated with the user ID indicating the hiring candidate. included. The pass/fail result shall be a value indicating either pass or fail. It should be noted that the evaluation information is not limited to the adoption/non-approval of the applicant, and may be any person evaluation to be predicted. For example, it is possible to include, as the evaluation information, the fact that the employee has retired (whether or not the employee is retired), the evaluation value of the employee's productivity (presentityism), the evaluation rank as a result of the internal personnel evaluation, and the like. ..

The model storage unit 220 stores two machine learning models, a CNN 221 (corresponding to the first prediction model of the present invention) and a machine learning model 222 (corresponding to the second prediction model of the present invention). ing. The CNN 221 is a multi-layer (four or more layers) neural network. The machine learning model 222 implements the gradient boost tree algorithm. The CNN 221 and the machine learning model 222 may use general neural network and machine learning implementations, and detailed description thereof will be omitted here.

The learning processing unit 211 learns the application data. As described above, the learning processing unit 211 performs the learning process using the convolutional neural network (CNN) and the gradient boost tree algorithm. Input data for the machine learning model (CNN 221 and machine learning model 222) is application data, and teacher data is evaluation information. First, the learning processing unit 211 extracts only the items that are text data (ES in the example of FIG. 5 and ES self appeal) from the items included in the application data, vectorizes the text data, and The vectorized data is used as the input data of the CNN 221, and the learning process is performed using the pass/fail result of the evaluation information corresponding to the user ID as the teacher data. Next, the learning processing unit 211 gives each of the items of the application data that is text data (text item) using the learned CNN 221 as a prediction model, and calculates the calculation result (possibility of pass/fail) and other items. The fixed data is used as the input data of the machine learning model 222, and the learning process is performed using the pass/fail result of the corresponding evaluation information as the teacher data. It should be noted that the category data of the fixed form data may be used as the input data by using a dummy variable. In addition, the calculation result given the item of the application data which is the text data is calculated in the middle layer of the CNN 221 instead of or in addition to the possibility of the final pass/fail. It is also possible to use an intermediate calculation result such as a feature vector.

The important word output unit 213 outputs important words among the words included in the text data among the items of the application data together with the degree of contribution to the evaluation. The important word output unit 213 can extract, as a significant word, a word indicated by a vector value of which the contribution degree is equal to or more than a predetermined value, with the weight obtained by the class activation mapping (CAM) related to the CNN 221 as the contribution degree. It should be noted that the important word output unit 213 may calculate the contribution degree calculated by the CAM (either the value of the contribution degree itself or the absolute value of the contribution degree. Hereinafter, the same applies to the contribution degree calculated by the CAM. ) (The order may be larger or smaller) may be sorted and output, or only a predetermined number of words may be output in the order of the contribution degree. Good. Also, the important word output unit 213 may extract words that are determined to be important by a method other than CAM. For example, when the RNN is used instead of the CNN 221 or in addition to the CNN 221, the words may be extracted according to the attention weight of the RNN. In addition, the important word output unit 213 may extract an important word according to the coefficient (weight) of each word when estimating a multiple regression equation using each word (vector) as an explanatory variable. .. The important word output unit 213 may output all the words together with the contribution degree. The important word output unit 213 may output the contribution degree for all words. Further, the important word output unit 213 may receive the designation of the word from the user and output the contribution degree of the received word.

The important item output unit 212 outputs the important items of the application data together with the contribution to the evaluation. The important item output unit 212 can extract a contribution of each item included in the machine learning model 222 (for example, a coefficient of an explanatory variable in the regression model) of a predetermined value or more. Note that, when the CNN is used for the machine learning model 222, the important item output unit 212 puts the items in the order of the contribution degree obtained by the CAM (the order may be large or small). The items may be sorted and output, or only a predetermined number of items may be output in the order of the contribution. Further, the important item output unit 212 may extract an item according to the weight of the attention of the RNN when the RNN is used for the machine learning model 222. In addition, the important item output unit 212 may extract an important item according to the coefficient (weight) of each word when estimating a multiple regression equation using each word (vector) as an explanatory variable. .. Note that the important item output unit 212 may output all the items together with the contribution degree. Further, the important item output unit 212 may receive an item designation from the user and output the contribution degree of the received item.

The numerical value data acquisition unit 214 acquires numerical value information or category information from the application data regarding the evaluation target person (the person who wants to be employed for the recruiting activity currently being executed). The numerical data acquisition unit 214 may accept input from the user for standard data such as the score of the SPI test relating to the evaluation subject and the type of school from which the user is evaluated, or may access another server to store the standard data. It may be acquired.

The text data acquisition unit 215 acquires text data from the application data relating to the evaluation target person. For example, the text data acquisition unit 215 may accept an input from the user for the text data described by the evaluation target person, or may accept a text file created by the evaluation target person.

The prediction processing unit 216 gives the text data acquired by the text data acquisition unit 215 to the learned CNN 221 to obtain a value, and this value and the fixed form data acquired by the numerical data acquisition unit 214 are used by the learned machine. It can be given to the learning model 222 to calculate the pass/fail probability of the evaluation target person.

The evaluation output unit 217 outputs the evaluation predicted by the prediction processing unit 216 (possibility of pass/fail of the evaluation target person in this embodiment). The evaluation output unit 217 can output the evaluation on, for example, a screen or a printer.

7 to 10 are diagrams showing the flow of learning processing by the learning processing unit 211.

FIG. 7 is a diagram showing a flow of a learning process related to the CNN 221.

The learning processing unit 211 performs the following processing on each application data stored in the application data storage unit 231.

The learning processing unit 211 extracts text data items (text items) from the items of the application data (S411), and decomposes the text data into words by morphological analysis processing or the like for each text item (S412). The learning processing unit 211 vectorizes the word by processing such as Word2Vec (S413), uses the vector value as input data, and uses the result of acceptance/rejection of the evaluation information corresponding to the user ID of the application data as teacher data to perform learning processing related to the CNN 221. Perform (S414). Note that the learning process using the CNN 221 is the same as the operation of a general neural network, and the description is omitted here. The learning processing unit 211 acquires a value characterizing the text data by the CNN 221 and sets it as the value (numerical value) of the text item (S415). As will be described later, the value is used for learning the machine learning model 222 together with items other than the text items described later.

The learning processing unit 211 executes CAM for the CNN 221 (S416), specifies a vector value having a contribution degree equal to or greater than a predetermined value from the result of the CAM, and acquires a word indicated by the identified vector value together with the contribution degree (S417).

After performing the above processing for each text item, the important word output unit 213 outputs the contribution degree for each important word (S418). The important word output unit 213 can output, for example, words whose contributions are equal to or more than a predetermined value in the order of contributions (larger or smaller). FIG. 8 is a diagram showing an output example of important words. In the example of the figure, the important word output unit 213 outputs the important word to the screen 51 together with the contribution degree (importance degree). The importance may be a value calculated (for example, standardized) based on the contribution (weight obtained from CAM). In addition, the important word output unit 213 uses the number of persons who have used the word in the text item among the applicants who have passed the evaluation (users in the passing category) and the word in the text item. It is possible to output the number of all the applicants for adoption (users in all categories) and the ratio (pass/users in all categories). The output by the important word output unit 213 is not limited to the content shown in FIG. 8, and for example, the number of application data in which the word is used (the number of applicants for adoption) is output as the number of word users. Alternatively, the number of successful candidates in the pass/fail result of the evaluation information indicated by the user ID of the application data in which this word has been used may be output as the number of successful applicants. By referring to the screen 51 as shown in FIG. 8, the user can easily understand what kind of word the candidate for adoption using can expect to pass. Note that the important word output unit 213 is not limited to output to the screen 51, may be output to a printing device such as a printer, may be transmitted to the user in the form of e-mail, a chat message, or the like, and may be voiced. The word and the contribution may be output to the speaker as voice data by processing such as synthesis.

FIG. 9 is a diagram showing a flow of a learning process according to the machine learning model 222.

The learning processing unit 211 performs the following processing on each application data stored in the application data storage unit 231.

The learning processing unit 211 divides the application data into text items and other items (S431), and converts each text item into a numerical value using the learned CNN 221 (S432). The conversion process to the numerical value using the CNN 221 can be performed by giving the CNN 221 the vectorized data as shown in steps S412 to S413 shown in FIG. 7, for example.

The learning processing unit 211 uses the numerical values obtained from the CNN 221 and the values of other fixed-form data items as input data, and uses the pass/fail result of the evaluation information corresponding to the user ID of the application data in the machine learning model 222 as teacher data. The learning process is performed (S433). Note that the learning process related to the machine learning model 222 is a learning process using a general gradient boost tree algorithm, and the description thereof will be omitted here. The machine learning model 222 may be implemented by an algorithm other than the gradient boost tree algorithm, and various learning processes related to machine learning can be used.

After performing the above processing for each application data, the important item output unit 212 outputs the contribution degree for each item (S434). The important item output unit 212 can output, for example, items whose contributions are equal to or greater than a predetermined value in the order of the contributions. FIG. 10 is a diagram showing an output example of the contribution degree of each item. In the example of the figure, the important item output unit 212 can output to the screen 52 together with the contribution degree in the order of contribution degree among the items of the application data. By referring to the output as shown in FIG. 10, the user can easily understand which item of the application data contributes to the acceptance of the applicant. Further, on the screen 52, a mark is output as “text” for the text item. Note that, in FIG. 10, the important item output unit 212 may output only the text item together with the contribution degree, or may output only the items of the fixed form data together with the contribution degree. Note that the important item output unit 212 is not limited to output to the screen 52, may be output to a printing device such as a printer, may be transmitted to the user in the form of e-mail, chat message, or the like, and may be a voice message. The word and the contribution may be output to the speaker as voice data by processing such as synthesis.

FIG. 11 is a diagram showing a flow of a prediction process regarding the employment possibility of the evaluation target person.

The text data acquisition unit 215 acquires text data of application data relating to the evaluation target person (S451). The text data acquisition unit 215 may, for example, accept text data from the user, or acquires the text data transmitted by the evaluation target person inputting it on a website or the like or attaching it to an e-mail. You may do so.

The prediction processing unit 216 divides the text data acquired by the text data acquisition unit 215 into words by morphological analysis processing or the like (S452), vectorizes the divided words by processing such as Word2Vec (S453), and has learned the vector value. The value given to the CNN 221 is obtained as the item value (S454). The processes of steps S451 to S454 can be performed for each item of text data in the application data.

Next, the numerical data acquisition unit 214 can acquire the fixed form data (numerical value information or category information) from the application data relating to the evaluation target person and set it as the corresponding item value of the application data (S455). The numerical data acquisition unit 214 may acquire the standard data transmitted by being input by the evaluation target person on a website or attached to an electronic mail. The numerical data acquisition unit 214 executes step S455 for each item of the application data that is the standard data.

The prediction processing unit 216 inputs the above item values (the value obtained from the CNN 221 and the value obtained by the numerical data obtaining unit 214) to the machine learning model 222 to obtain the possibility of pass/fail (S456), and outputs the evaluation. The unit 217 outputs the obtained possibility of success or failure (S457).

In this way, according to the evaluation device 20 of the present embodiment, it is possible to predict the result (pass/fail) of the personnel evaluation by combining the fixed and atypical data.

In addition, in the conventional personnel evaluation, the analysis was conducted by focusing only on the fixed-value data of numerical values and category values among the items of the application data, and the information contained in the non-fixed-type text data cannot be ignored. It took time and load to read by human power. On the other hand, according to the evaluation device 20 of the present embodiment, not only fixed data but also atypical data can be incorporated as variables for analysis and prediction, so that reliability of analysis and prediction results is improved. In addition, it is possible to reduce the time and effort required to read text data in personnel evaluation. Therefore, for example, in evaluating a person who is hiring a new graduate, it is possible to perform a multi-faceted evaluation in which not only the score of the SPI test or the like but also sentence data such as the motivation for the entry sheet is considered.

Although the present embodiment has been described above, the above embodiment is for the purpose of facilitating the understanding of the present invention and is not for limiting and interpreting the present invention. The present invention can be modified and improved without departing from the spirit thereof, and the present invention also includes equivalents thereof.

For example, in the present embodiment, the person evaluation service is provided by one evaluation device 20, but the present invention is not limited to this, and the system may be implemented by a system configured by a plurality of evaluation devices 20. In this case, for example, one evaluation apparatus 20 may be virtually configured by a plurality of computers. For example, a functional unit and a part of the storage unit may be provided, such as providing each storage unit in a database server. It may be provided in another computer, and the evaluation device 20 and the other computer may cooperate with each other.

Further, in the present embodiment, it is assumed that the pass/fail (reliability) of hiring a new employee is predicted. However, the present invention is not limited to this. It is also possible to predict employee productivity by using, for example, the evaluation value of employee productivity (presentityism) as teacher data. In this case, according to the evaluation device 20 of the present embodiment, words that contribute to employee productivity can be displayed as important words. By referring to these important words, the personnel manager can be expected to obtain hints that lead to organizational improvement, for example. Moreover, even if the text data is an engagement survey or the like that cannot be read by all the human hands, it is possible to quickly and easily grasp the contents by checking only the important words. In addition, if the important words are displayed in order of increasing contribution or increasing absolute value of contribution, it is possible to understand words that have a negative effect. From internal comments such as surveys, for example, it is possible to understand what kind of dissatisfaction the excellent employee group is having with the company.

Further, in the present embodiment, the atypical items of the application data include the text data created by the applicant, but for example, the employee who interviewed may create a comment and include it in the application data. May be. In this case, the personnel personnel, the management, and the like can easily understand what kind of employee is evaluated as excellent on the spot by referring to the important words extracted from the personnel evaluation comment on the spot.

Further, in the present embodiment, the success/failure of hiring a new employee is predicted, but the present invention is not limited to this. For example, the personnel evaluation result of an employee within the company may be predicted. In this case, the learning processing unit 211 uses the personnel evaluation data created by the employee's boss as input data instead of the application data, and uses the results of personnel evaluation (rank, score, etc.) as teacher data in the CNN 221 and the machine learning model 222. Just give it and learn.

Further, in the present embodiment, after the important word output unit 213 outputs the word and the contribution degree as in the screen 51 shown in FIG. 8, the important item output unit 212 makes each fixed form and the like as in the screen 52 shown in FIG. Although it has been described that the atypical item and the contribution degree are output, the screen 51 may be output after the output of the screen 52. In this case, the important item output unit 212 outputs the contribution degree of each item extracted from the machine learning model 222 to the screen 52, and then, on the screen 52, by clicking or the like, a text item (in the example of FIG. "Motivation", "ES_self-appeal" and "school type (use)" designation) is accepted, the accepted items are passed to the important word output unit 213, and the important word output unit 213 is included in the designated items. It is possible to extract the contribution degree of the word to be extracted from the CNN 221 and output it to the screen 51.

The contribution of each item and the contribution of each word included in each text item may be stored. FIG. 12 is a diagram illustrating a software configuration example of the evaluation device 20 including the contribution storage unit 233 that stores the contribution of each item and each word. FIG. 13 is a diagram showing a configuration example of the contribution storage unit 233. As shown in FIG. 13, the contribution storage unit 233 can include an item contribution 2331 and a word contribution 2332. The item contribution degree 2331 can include, for each item, a contribution degree of the item and a text flag indicating whether the item is non-standard data in a text format. The item of the item contribution degree 2331 and the contribution degree can be extracted from the machine learning model 222 as described above. In addition, the word contribution degree 2332 can include, for each text item, the contribution degree to the evaluation of the word in the item, in association with the item and the word included in the item. The word and the contribution degree of the word contribution degree 2332 can be extracted from the CNN 221 using the CAM as described above. The important item output unit 212 reads the item contribution degree 2331 from the contribution degree storage unit 233 and outputs the screen 52 shown in FIG. 10. When the item is designated by clicking or the like on the screen 52, the important word output unit 213 The word contribution degree 2332 corresponding to the designated item can be read from the contribution degree storage unit 233 and the screen 51 shown in FIG. 8 can be output. This allows the user to understand the items that contribute to the employment possibility of the applicant and also to drill down from the item to a word to see what word contributes to the employment possibility.

Further, in the present embodiment, although the success/failure of hiring a new employee is predicted, the present invention is not limited to this. For example, the retirement possibility of an employee may be predicted. When predicting the retirement possibility, the learning processing unit 211 can use data as shown in FIG. 14 as application data. In the example of FIG. 14, the application data includes the department to which the employee belongs, the post, the result of the motivation questionnaire, the boss's comment, the number of days absent, etc. in association with the user ID identifying the current or past employee. be able to. Further, as the evaluation information, as shown in FIG. 15, the result (rank) of the in-house personnel evaluation, the retirement flag indicating whether the employee indicated by the user ID has retired, in association with the user ID for identifying the employee, And the retirement date if the retirement flag is true (the employee has retired). As a result, the learning processing unit 211 can evaluate the retireability of the employee (evaluation of whether or not the employee is leaving) using the same processing as the processing described in the present embodiment. That is, the text items (motivation questionnaire and boss comment in the example of FIG. 14) included in the application data shown in FIG. 14 are vectorized as input data, and the retirement flag of the evaluation information shown in FIG. 15 is given to the CNN 221 as teacher data. In addition to learning, the value calculated by CNN 221 (prediction result of retirement possibility or feature vector of middle class) and the value of items other than text data of application data are given to machine learning model 222 for learning. , The possibility of final retirement (the possibility that the retirement flag will become true) can be predicted. Here, the rank indicating the personnel evaluation of the employee included in the evaluation information may be added as input data to the CNN 221. Also, the history of ranks that have been performed every year is accumulated in the evaluation information, and the change pattern of ranks in a predetermined number of years in the past (for example, rank A 3 years ago, rank B 2 years ago, rank C last year). It is also possible to give "ABC") indicating that it has become input to the machine learning model 222 as input data to make a prediction in consideration of the transition of personnel evaluation. Further, the retirement date may be given to the CNN 221 and the machine learning model 222 as an evaluation value to predict the retirement date. In this case, for employees who have not retired (retirement flag is false), the retirement date can be set as the date when they are scheduled to retire when they have worked until retirement and learning can be performed.

Further, the learning processing unit 211 can also perform learning by, for example, using the text of an e-mail of an employee as input data and giving the result of personnel evaluation as teacher data to the CNN 221 and the machine learning model 222. In this case, it is possible to estimate the ability of the employee based on the communication data of the employee (e-mail, daily report, chat tool, etc.), and use the same predictive model (CNN221 and machine learning model 222 already trained). Then, it is possible to determine whether or not the salesman is excellent from the text of the email from the salesman of another company.

Further, in the present embodiment, the pass/fail result is used as the teacher data, but the present invention is not limited to this, and the evaluation value is stored as numerical data (e.g., an evaluation value of 10 levels or a score of 100 points). Machine learning may be performed using this as teacher data.

Further, in the present embodiment, the prediction model for predicting the evaluation is created by performing the two machine learnings of the CNN 221 and the machine learning model 222, but one machine learning model 222 (CNN may be used. , RNN, the implementation of a gradient boost tree algorithm, or a statistical model related to multiple regression analysis) may be used for learning. In this case, the text of the non-standard data may be converted into a word vector and then machine learning may be performed by using this vector value as input data together with other standard data. The prediction processing unit 216 can perform prediction using only the machine learning model 222.

Further, in the present embodiment, the contribution degree of the word in the CNN 221 and the contribution degree of the text item in the machine learning model 222 are not particularly linked, but for example, the contribution degree of the word in the CNN 221 (determined by CAM or the like). Standardization may be performed so that the sum of weights becomes the contribution of the text item.

10 user terminal 20 evaluation device 211 learning processing unit 212 important item output unit 213 important word output unit 214 numerical data acquisition unit 215 text data acquisition unit 216 prediction processing unit 217 evaluation output unit 221 CNN
222 machine learning model 231 application data storage unit 232 evaluation information storage unit

Claims (29)

A learning data storage unit that stores at least non-standard data in text format and standard data in non-text format related to a person,
An evaluation information storage unit that stores the evaluation of the person,
Calculated by creating a first prediction model by machine learning using word data relating to words included in the atypical data as input data and using the evaluation as teacher data, and giving the atypical data to the first prediction model. A learning processing unit that creates a second prediction model by machine learning using the calculated value and the standard data as input data and the evaluation as teacher data;
A contribution degree output unit that extracts a contribution degree to the evaluation for each of the atypical data and the boilerplate data from the second prediction model, and outputs the extracted contribution degree;
An information processing apparatus comprising: The information processing apparatus according to claim 1, wherein
Further comprising a word contribution output unit that extracts the contribution of each word to the evaluation from the first prediction model and outputs the extracted contribution of each word.
An information processing device characterized by: The information processing apparatus according to claim 1, wherein
The learning processing unit creates at least the first prediction model using a convolutional neural network;
An information processing device characterized by: The information processing apparatus according to claim 3, wherein
The method further comprises a word contribution output unit that extracts the contribution of each word to the evaluation from the first prediction model by class activation mapping for the convolutional neural network and outputs the extracted contribution of each word. thing,
An information processing device characterized by: The information processing apparatus according to any one of claims 1 to 4,
An evaluation data acquisition unit that acquires the atypical data and the atypical data created by an evaluation subject,
An evaluation prediction unit that predicts the evaluation of the evaluation target person based on the atypical data, the fixed data, and the first and second prediction models,
An evaluation output unit that outputs the predicted evaluation,
An information processing apparatus, further comprising: A learning data storage unit that stores at least non-standard data in text format and standard data in non-text format related to a person,
A prediction model storage unit that stores a prediction model for predicting an evaluation for the person based on the atypical data and the atypical data,
A contribution degree output unit that extracts the contribution degree for each of the atypical data and the fixed form data from the prediction model, and outputs the extracted contribution degree,
An information processing apparatus comprising: The information processing apparatus according to claim 6,
For a word included in the atypical data, further comprising a word contribution output unit that extracts the contribution of each word to the evaluation from the prediction model, and outputs the extracted contribution of each word,
An information processing device characterized by: The information processing apparatus according to claim 7, wherein
The word contribution output unit classifies and outputs the word that contributes to increase the evaluation and the word that contributes to decrease the evaluation,
An information processing device characterized by: Computer
A step of storing at least text-type atypical data relating to a person and non-text-type atypical data;
Storing a rating for the person,
Creating a first prediction model by machine learning using word data relating to words included in the atypical data as input data and using the evaluation as teacher data;
Creating a second predictive model by machine learning using the calculated value obtained by giving the atypical data to the first predictive model and the standard data as input data, and using the evaluation as teacher data.
Extracting the contribution to the evaluation for each of the atypical data and the atypical data from the second prediction model,
Displaying the extracted contribution,
How to display. The display method according to claim 9,
The computer is further
A method of extracting the contribution of each word to the evaluation from the first prediction model, and displaying the extracted contribution of each word. The display method according to claim 9,
The computer creates at least the first predictive model using a convolutional neural network;
Display method characterized by. The display method according to claim 11,
The computer is further
A step of extracting the contribution degree of each word to the evaluation from the first prediction model by class activation mapping for the convolutional neural network, and displaying the extracted contribution degree of each word. Information processing device. The display method according to any one of claims 9 to 12,
The computer is further
Acquiring the atypical data and the atypical data created by an evaluation subject,
Predicting the evaluation of the evaluation subject based on the atypical data, the typical data, and the first and second prediction models;
Displaying the predicted assessment,
Display method characterized by executing. Computer
A step of storing at least text-type atypical data relating to a person and non-text-type atypical data;
Storing a prediction model for predicting an evaluation for the person based on the atypical data and the atypical data;
Extracting the contribution of each of the atypical data and the boilerplate data from the prediction model, and displaying the extracted contribution.
Display method characterized by executing. The display method according to claim 14, wherein
The computer is further
A display method comprising: performing a step of extracting a contribution degree of each word to the evaluation from the prediction model for a word included in the atypical data and outputting the extracted contribution degree of each word. The display method according to claim 15, wherein
The computer classifies and outputs the word that contributes to increase the evaluation and the word that contributes to decrease the evaluation,
Display method characterized by. On the computer,
A step of storing at least text-type atypical data relating to a person and non-text-type atypical data;
Storing a rating for the person,
Creating a first prediction model by machine learning using word data relating to words included in the atypical data as input data and the evaluation as teacher data;
Creating a second predictive model by machine learning using the calculated value obtained by giving the atypical data to the first predictive model and the standard data as input data, and using the evaluation as teacher data,
Extracting the contribution to the evaluation for each of the atypical data and the atypical data from the second prediction model,
Displaying the extracted contribution,
A program to execute. The program according to claim 17,
Further on the computer,
A program for executing a step of extracting a contribution degree of each word to the evaluation from the first prediction model and displaying the extracted contribution degree of each word. The program according to claim 17,
Causing the computer to create at least the first predictive model using a convolutional neural network;
A program characterized by. The program according to claim 19,
Further on the computer,
A program for executing a step of extracting the contribution degree of each word to the evaluation from the first prediction model by class activation mapping for the convolutional neural network, and displaying the extracted contribution degree of each word. The program according to any one of claims 18 to 20,
Further on the computer,
Acquiring the atypical data and the atypical data created by an evaluation subject,
Predicting the evaluation of the evaluation subject based on the atypical data, the typical data, and the first and second prediction models;
Displaying the predicted assessment,
A program to execute. On the computer,
A step of storing at least text-type atypical data relating to a person and non-text-type atypical data;
Storing a prediction model for predicting an evaluation for the person based on the atypical data and the atypical data;
Extracting the contribution of each of the atypical data and the boilerplate data from the prediction model, and displaying the extracted contribution.
A program to execute. The program according to claim 22,
Further on the computer,
A program for executing a step of extracting, for words included in the atypical data, a contribution degree of each word to the evaluation from the prediction model and outputting the extracted contribution degree of each word. The program according to claim 23,
Causing the computer to classify and output the words that contribute to increase the evaluation and the words that contribute to decrease the evaluation,
A program characterized by. A display method for outputting the degree of contribution to an evaluation of a person of an item including a plurality of words, which is composed of non-standard data in a text format,
Display the contribution to the evaluation of the person for each of the multiple items,
A display method of displaying, for each of one or more words included in each item, the degree of contribution to the evaluation. The display method according to claim 25,
Accepting a selection request from the user for one of the plurality of items,
A display method of displaying, for each of one or more words included in the one item, the degree of contribution to the evaluation. The display method according to claim 25,
The display method, wherein the contribution degree displayed for each of the plurality of words included in the one item is a breakdown of the contribution degree displayed for each item of the plurality of items. The display method according to claim 25,
Among the words included in the one item, a word having a predetermined contribution is displayed. The display method according to claim 25,
A method of displaying, wherein the degree of contribution to an evaluation of a person for an item configured by standard data other than the text is displayed together with the degree of contribution to an evaluation of a person for each of the plurality of items.