CN117973949A - Evaluation generation method, device, equipment and storage medium based on large model - Google Patents
Evaluation generation method, device, equipment and storage medium based on large model Download PDFInfo
- Publication number
- CN117973949A CN117973949A CN202410385149.XA CN202410385149A CN117973949A CN 117973949 A CN117973949 A CN 117973949A CN 202410385149 A CN202410385149 A CN 202410385149A CN 117973949 A CN117973949 A CN 117973949A
- Authority
- CN
- China
- Prior art keywords
- employee
- evaluation
- model
- data
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 253
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000012216 screening Methods 0.000 claims abstract description 13
- 230000002457 bidirectional effect Effects 0.000 claims abstract description 12
- 230000006399 behavior Effects 0.000 claims description 97
- 239000013598 vector Substances 0.000 claims description 36
- 230000006870 function Effects 0.000 claims description 29
- 238000005457 optimization Methods 0.000 claims description 19
- 230000003542 behavioural effect Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 11
- 239000011159 matrix material Substances 0.000 claims description 10
- 230000001186 cumulative effect Effects 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 5
- QVFWZNCVPCJQOP-UHFFFAOYSA-N chloralodol Chemical compound CC(O)(C)CC(C)OC(O)C(Cl)(Cl)Cl QVFWZNCVPCJQOP-UHFFFAOYSA-N 0.000 claims description 5
- 230000008569 process Effects 0.000 abstract description 12
- 238000013473 artificial intelligence Methods 0.000 abstract description 5
- 238000004891 communication Methods 0.000 description 7
- 238000012549 training Methods 0.000 description 7
- 101100481876 Danio rerio pbk gene Proteins 0.000 description 5
- 101100481878 Mus musculus Pbk gene Proteins 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000007423 decrease Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000002787 reinforcement Effects 0.000 description 2
- 235000002198 Annona diversifolia Nutrition 0.000 description 1
- 241000287196 Asthenes Species 0.000 description 1
- 241000282842 Lama glama Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000008014 freezing Effects 0.000 description 1
- 238000007710 freezing Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06398—Performance of employee with respect to a job function
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/105—Human resources
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Economics (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Educational Administration (AREA)
- Development Economics (AREA)
- General Business, Economics & Management (AREA)
- Operations Research (AREA)
- Marketing (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Game Theory and Decision Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application relates to the technical field of artificial intelligence, and discloses an evaluation generation method, device, equipment and storage medium based on a large model, wherein the method comprises the following steps: generating employee portrayal labels of target employees according to a preset employee portrayal label library and preset target behavior data; screening employee behavior data of the target employee according to label scores of the employee portrait labels; inputting employee portrait labels and employee behavior data into a preset large language generation model, and outputting employee evaluation data; and fine tuning the large language generating model according to the preset bidirectional evaluation data and employee evaluation data to obtain an evaluation generating model, and generating target evaluation data of target employees by using the evaluation generating model. By implementing the method, the staff evaluation adopts two evaluation systems, namely the staff portrait label and the large language model generation evaluation, and compared with the evaluation mode adopting a single index, the method can evaluate the staff in the whole working process, and improve the accuracy of staff evaluation results.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a method, an apparatus, a device, and a storage medium for generating an evaluation based on a large model.
Background
The evaluation of the work performance and the work capacity of the staff is an important part of the management of the company all the time, the contribution of the staff to the company is related to the operation efficiency of the whole company, and the contribution is influenced by the personal capacity of the staff, the organization efficiency of the company and the work attitude of the staff in many aspects.
The existing evaluation method is mainly used for evaluating the work performance of the staff based on the evaluation index, in practical application, the staff is evaluated from the work output of the staff based on the performance index, the staff is evaluated from the result dimension, the machine is compared, the staff is not comprehensively evaluated only from the final result, and the artificial judgment in the process of evaluating the staff has great subjectivity, so that the evaluation of the staff cannot be fair and fair, and the accuracy in the process of evaluating the staff is low.
Disclosure of Invention
The application provides an evaluation generation method, device, equipment and storage medium based on a large model, and mainly aims to solve the problem of low accuracy in an employee evaluation mode provided in the related technology.
In order to achieve the above object, the present application provides a method for generating an evaluation based on a large model, including:
Generating employee portrayal labels of target employees according to a preset employee portrayal label library and preset target behavior data; screening employee behavior data of the target employee according to label scores of the employee portrait labels; inputting employee portrait labels and employee behavior data into a preset large language generation model, and outputting employee evaluation data; and fine tuning the large language generating model according to the preset bidirectional evaluation data and employee evaluation data to obtain an evaluation generating model, and generating target evaluation data of target employees by using the evaluation generating model.
In order to solve the above problems, the present application also provides an evaluation generating device based on a large model, including:
The staff portrait tag generation module is used for generating staff portrait tags of target staff according to a preset staff portrait tag library and preset target behavior data;
the staff behavior data screening module is used for screening staff behavior data of the target staff according to the label scores of the staff portrait labels;
the employee evaluation data output module is used for inputting the employee portrait tag and the employee behavior data into a preset large language generation model and outputting the employee evaluation data;
The target evaluation data generation module is used for fine-tuning the large language generation model according to the preset bidirectional evaluation data and employee evaluation data to obtain an evaluation generation model, and generating target evaluation data of target employees by using the evaluation generation model.
In order to solve the above-mentioned problems, the present application also provides an electronic device including:
At least one processor; and
A memory communicatively coupled to the at least one processor; wherein,
The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the large model-based evaluation generation method described above.
In order to solve the above-mentioned problems, the present application also provides a computer-readable storage medium in which at least one computer program is stored, the at least one computer program being executed by a processor in a device to implement the above-mentioned large model-based evaluation generation method.
According to the embodiment of the application, the staff behavior data is collected from daily description, work content and work report of staff through the label library and the label description in the staff portrait label library, and then the similarity matching is carried out on the label description and the behavior data through the semantic representation model, so that the portrait label of the staff is extracted, a training model is not needed, a supervision corpus is not needed, and the generated evaluation is limited, so that the generated evaluation is more in line with reality; the employee portrait tag and the employee behavior data are input into the large language generation model, and the employee evaluation data are output, so that the illusion phenomenon in the large language generation process can be limited; the large language generation model is finely adjusted by using a direct preference optimization mode based on employee evaluation feedback data, so that the preference of human evaluation can be fitted, the evaluation meets the actual requirement, and the bias and the incorrectness of the human evaluation are eliminated. Therefore, the evaluation generating method, the device, the equipment and the storage medium based on the large model can solve the problem of lower accuracy in the process of evaluating staff.
Drawings
FIG. 1 is a flow chart of a large model-based evaluation generation method according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of generating employee portrait tags according to an embodiment of the present application;
FIG. 3 is a flow chart of a fine-tuning large language generation model according to an embodiment of the present application;
FIG. 4 is a functional block diagram of a large model-based evaluation generating device according to an embodiment of the present application;
Fig. 5 is a schematic structural diagram of an electronic device for implementing a large model-based evaluation generation method according to an embodiment of the present application.
The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
In the prior art, the work performance of staff is evaluated mainly through key assessment indexes, the contribution of staff to a company is evaluated by using the completion degree of staff to the assessment indexes, however, the assessment mode based on the performance indexes is to evaluate staff from the work output of staff, the staff is assessed from the result dimension, the assessment of staff is quite often not quite fair and can not be quite fair, and the accuracy of staff assessment is lower.
Referring to fig. 1, a flow chart of a large model-based evaluation generation method according to an embodiment of the present application is shown. In this embodiment, the large model-based evaluation generation method includes:
S11, generating employee portrayal labels of target employees according to a preset employee portrayal label library and preset target behavior data.
In the embodiment of the application, the employee portrait tag library consists of a series of tag names and tag descriptions, namely { tag names: tag description }, portrait tag names are descriptions of employee behaviors, such as diligence, timekeeping, active work, affinity and the like, tag descriptions are specific descriptions of tag contents, such as tag names corresponding to diligence are described as employee vs. work, industry, effort is made to complete tasks, the task is not afraid of being hard, and the work is full of enthusiasm and spirit; the label name is the label corresponding to the time keeping, which is described as the time when the staff arrives at the work and the meeting, so that the time can be effectively managed to ensure the smooth proceeding of the workflow, etc.
In the embodiment of the present application, referring to fig. 2, generating an employee portrait tag of a target employee according to a preset employee portrait tag library and preset target behavior data includes: s21, extracting a label name and a label description in a preset employee portrait label library; s22, dividing preset target behavior data into target paragraph behavior data; s23, performing semantic conversion on the behavior data of the target paragraph to obtain a behavior data semantic vector; s24, carrying out semantic conversion on the tag description according to the tag name to obtain a tag description semantic vector; s25, generating employee portrait tags of the target employees according to the behavior data semantic vectors and the tag description semantic vectors.
In detail, the tag name and tag description in the employee portrayal tag library may be expressed as follows: the label and the label description are extracted based on an image label library, wherein the employee image label library can be collected manually or can be extracted from an image label corpus through machine learning.
Specifically, to evaluate an employee, user portraying is performed on the personality of the employee, and records in daily work of the employee need to be collected, the target behavior data is behavior data collected based on related data such as a work log, a work report, a business report and the like, for example, the daily work content, the task completion condition and the spent time of the employee can be known from the work log; work reports can be used for knowing work results, project progress and difficulties faced by staff in a specific time period; furthermore, the behavioral data of the staff can be divided according to paragraphs, all the behavioral data are divided into a series of paragraph text forms, namely, target paragraph behavioral data behavioral _data= (data_1, data_2), and analysis and evaluation of each behavior or event can be more convenient.
Illustratively, text paragraph partitioning for employee behavioral data may be based on diligence and timekeeping, such as where employees have shown extremely high diligence over the past several months. They arrive at the office on time all the time and exhibit a great deal of concentration at work. This is especially true when the project expiration date is near, where employees are typically willing to overtime to ensure that the task is completed.
Further, the collected daily data of the staff is in a text form, and cannot be directly calculated, semantic representation is performed on the text of the target paragraph behavior data by adopting a pre-training model, the collected data is represented as a series of semantic vectors, and further semantic conversion is performed on the target paragraph behavior data to obtain a behavior data semantic vector, wherein the target paragraph behavior data can be represented as a series of semantic vector sets by using the pre-training model BERT and a large language model (Large Language Model, LLM) (such as lalama, chatglm, chatgpt, etc.), LLM _ embedding (data) =data_ embedding, the semantic vector is an N-dimensional vector, and generally the N value can be n= 687,1024, i.e. one semantic vector contains N components: embedding _vector= (embedding _1, casting_2,., embedding _n), the set of semantic representation vectors of all target paragraph behavior data can be represented as: behavioral _data_ embedding = (data_1_emumbedding ).
Furthermore, in order to match the collected behavior data with the pre-established user tag, similarly, the semantic vector representation needs to be performed on the tag description corresponding to the tag name, that is llm_embedding(label_description)=label_description_embedding,label_table_embedding=(label_1:label_1_description_embedding,label_2:label_2_description_embedding,...),, and then, according to the distance calculation between the semantic vector of the behavior data and the semantic vector of the tag description, a more accurate employee portrait tag of the target employee is generated.
In the embodiment of the application, the staff portrait tag is a portrait tag which is selected based on the similarity between the behavior data of the target staff and the tag description and can describe the target staff more accurately, and the staff portrait is generated by using language representation and similarity matching, so that a model is not required to be trained and a corpus is not required to be supervised.
In the embodiment of the application, the staff portrait tag of the target staff is generated according to the behavior data semantic vector and the tag description semantic vector, and the staff portrait tag comprises: calculating a distance value between the behavior data semantic vector and the tag description semantic vector; generating a distance matrix of the tag description and the target behavior data according to the distance value; calculating label scores of target staff according to the distance values in the distance matrix, wherein a label score calculation formula is as follows:
Wherein, For/>Tag scoring of individual tag descriptions,/>For/>Personal behavior data and No. >Distance value between individual tag descriptions,/>The number of the behavior data; sorting the label names corresponding to the label descriptions according to the label scores to obtain a label sorting queue; and screening portrait labels in the label arrangement queue according to a preset label threshold value to serve as target portrait labels of target staff.
In detail, the distance between the behavior data semantic vector and the tag description semantic vector is calculated through a cosine distance function, wherein the distance is thatValues within the range, i.e. by cosine distance function/>Calculating a distance value, wherein/>Representing distance value,/>Representing behavioural data semantic vectors,/>First/>, representing behavioral data semantic vectorsComponent,/>Representing tag description semantic vectors,/>First/>, representing tag description semantic vectorComponent,/>Representing the dimension of the semantic vector, performing cosine distance calculation on the semantic vector of the behavior data and the semantic vector of the tag description two by two to obtainDist (data_i_ embedding, label_j_description_ embedding), where/>For behavioural data/>And tag description/>And the distance values between the labels are used for generating a distance matrix according to the distance values, the distance matrix is represented by taking the behavior data as a row, the labels are described as column representations, and the scoring values of staff on different labels can be calculated according to the distance matrix.
Specifically, the label scores of the staff on the different labels, namely score= (score 1,score2,…,scoreK), can be calculated by a label score calculation formula, whereinFor the number of portrait tags,/>Tag/>, all behavioral data for staffScoring the sum; furthermore, for an employee, the first topK labels to be scored need to be obtained and used as portrait labels of the employee, namely, the scoring list is ordered from large to small according to the label scores to obtain a label ordering queue, then label values corresponding to the first topK scoring values are taken, the label threshold value is topK, the value of topK can be taken according to requirements, for example, topK =8, and the target portrait labels of the target employee can be expressed as follows: /(I)。
Further, the generated evaluation is limited by using the employee portrayal tag and the behavior data, so that the illusion phenomenon in the large language model generation process is limited, the generated evaluation is more realistic, and in order to generate objective evaluation on the employee by using the large language model, the employee behavior data is screened based on the employee portrayal tag of the target employee, and then the employee portrayal tag and the employee behavior data are used as model input data, so that objective employee evaluation data is generated.
S12, screening employee behavior data of the target employee according to label scores of the employee portrait labels.
In the embodiment of the application, the portrait tag of the employee is the employee tag label_employee, and the behavior data of each employee is very much, and in consideration of the limitation of the context length of a large language model, the employee behavior data needs to be screened out according to the tag score corresponding to each item of the employee portrait tag label_employee.
In the embodiment of the application, the staff behavior data of the target staff is screened according to the label scores of staff portrait labels, which comprises the following steps: selecting expected behavior data corresponding to employee portrait labels one by one in the distance matrix; generating a label grading list according to label grading of employee portrait labels and expected behavior data one by one; and selecting expected behavior data corresponding to the maximum label score in the label score list as employee behavior data of the target employee.
In detail, the expected behavior data is selected from the label description and the target behavior data in the distance matrix, the expected behavior data is a part of data selected from the target behavior data, a label grading list is generated according to the selected expected behavior data and label grading of the employee portrait labels, the behavior data with the largest label grading corresponding to the employee portrait labels is selected from the label grading list to serve as employee behavior data, and the corresponding employee behavior data is selected based on each employee portrait label.
Illustratively, the employee portrayal tab may be represented asThen select/>Corresponding behavioural data/>Behavior data with highest medium label score asCorresponding behavior data, if/>Corresponding behavioural data/>The corresponding label score is the largest, then/>The corresponding behavior data is/>And so on, choose/>Corresponding behavior data until selected/>And the corresponding behavior data is obtained by taking all the selected behavior data as a behavior data set.
Further, by combining employee portrayal labels (e.g., skills, experience, personality, etc.) with employee behavioral data, the generated employee assessment data can be more personalized, and the model can take into account the unique characteristics of each employee, thereby providing a more targeted assessment; by using a model, it can be ensured that the generated assessment remains consistent in intonation and content, helping to avoid subjectivity or inconsistency, making the assessment more fair and comparable.
S13, inputting the employee portrait tag and the employee behavior data into a preset large language generation model, and outputting employee evaluation data.
In the embodiment of the application, the large language model adopts a similar transformation former model architecture, and pretrains through a large amount of corpus, so that the method has the characteristic of fitting human language well; the large language model adopts an end-to-end generation mode, and can generate smooth sentences according to prompt and input; the generated sentences of the large language model have some language illusions, namely the generated contents are smooth, but the generated contents can not confirm the fact correctness of the generated contents or the generated contents do not accord with the original text; the generation of the large language model can be limited by portrait tags of staff and staff behavior data, and the generated result can well eliminate the illusion of the large language model, so that the generated content is real and reliable, and meets the general requirements of people on staff evaluation, wherein the large language model (Large Language Model, LLM) comprises, but is not limited to, llama, chatglm and bloomz models.
Further, before generating employee evaluation data according to the employee portrait tag and the employee behavior, fine tuning is required to be performed on the large language model, so that the large language model can identify instructions of the employee evaluation data, and corresponding employee evaluation data is generated according to the fine-tuned large language model.
In the embodiment of the application, before the employee portrait tag and the employee behavior data are input into the preset large language generation model and the employee evaluation data are output, the method further comprises the steps of: taking a preset employee evaluation data case as an employee evaluation instruction corpus; inputting employee evaluation instruction corpus into a preset large language model to obtain employee model evaluation data; calculating an evaluation loss value of employee model evaluation data through a preset loss function; and fine tuning model parameters in the large language model through a preset LoRA model and an evaluation loss value to obtain a large language generation model.
In detail, the large language model has strong small sample learning capability, in order for the model to recognize the instruction for generating employee evaluation data, a plurality of cases for generating the employee evaluation data based on the employee portrait label need to be manually written, and the employee evaluation data cases are used as the corpus finely adjusted by the instruction, for example, the employee work diligence can always complete the task on time; "he exhibits excellent leadership in team, coordination is strong"; "excellent under pressure, good at handling challenges and finding solutions"; the large model can be fine-tuned by employee evaluation data cases teaching it how to generate employee evaluation data on command.
Specifically, a LoRA fine tuning mode is adopted, so that a large model can identify an instruction for generating employee evaluation data, corresponding employee evaluation data can be generated according to the instruction by utilizing the fine-tuned large model, namely prepared employee evaluation data cases can be arranged into a text file, each case occupies one line, a LoRA fine tuning method is adopted, the cases are used as training data, fine tuning is carried out through the model, the model is learned to generate similar employee evaluation data, after fine tuning is completed, the performance of the model in terms of the raw member engineering evaluation data is evaluated, wherein the basic principle of LoRA is to freeze pre-trained model weight parameters, and under the condition of freezing original model parameters, additional network layers are added into the model, and only the newly added network layer parameters are trained, so that the performance of the model on tasks is improved.
Further, based on the trimmed large language model, namely a large language generation model, corresponding employee evaluation data can be generated according to the instruction, and the input format of the large language generation model is input { sample }, namely employee evaluation is generated according to the employee portrait tag and the behavior data; input: employee portrayal tag, employee behavior data }, output as output { output: employee evaluation statement }, i.e., output= llm _ generate (input), resulting in employee evaluation data generated by the large language generation model.
Furthermore, the employee evaluation data generated by using the large language generation model has excellent performance in terms of language fluency and evaluation objectivity, but the evaluation of the employee has a certain subjective preference and a certain standard, and the large model cannot completely reach the human judgment standard. Therefore, the evaluation of different persons on the same employee is collected, the advantages and disadvantages of different evaluations of the same employee are judged manually, corpus pairs (input, good evaluation and poor evaluation) are formed, and the generated model is finely tuned by using the corpus pairs in a reinforcement learning mode, so that the finely tuned model can accurately reflect the characteristics of human evaluation.
S14, fine tuning is conducted on the large language generation model according to the preset bidirectional evaluation data and employee evaluation data, an evaluation generation model is obtained, and target evaluation data of target employees are generated by means of the evaluation generation model.
In the embodiment of the application, the bidirectional evaluation data comprises employee self-evaluation data and employee direct-lead evaluation data, namely, for one employee, the evaluation of the employee needs to be acquired from three aspects, the employee self-evaluation data, the employee direct-lead evaluation data and model generation evaluation, and the large language generation model is subjected to fine adjustment through the evaluation corpus pair, so that the fine-adjusted model can accurately reflect the characteristics of human evaluation.
In the embodiment of the present application, referring to fig. 3, fine tuning is performed on a large language generation model according to preset bidirectional evaluation data and employee evaluation data to obtain an evaluation generation model, including: s31, extracting employee self-evaluation data and employee direct leading evaluation data in preset bidirectional evaluation data; s32, generating a first corpus pair according to employee self-evaluation data and employee evaluation data; s33, generating a second corpus pair according to the employee direct lead evaluation data and the employee evaluation data; s34, constructing a model optimization corpus pair according to the first corpus pair and the second corpus pair; s35, inputting the model optimization corpus pair into the large language generation model for fine adjustment, and obtaining the evaluation generation model.
In detail, the bi-directional evaluation data includes employee self-evaluation data and employee direct-leading evaluation data, wherein the employee self-evaluation data and the employee direct-leading evaluation data in the bi-directional evaluation data can be extracted from a pre-stored storage area by a computer sentence (such as a Java sentence, a Python sentence, etc.), wherein the storage area includes but is not limited to a database and a blockchain.
Specifically, a first corpus pair is constructed by model input, employee self-evaluation data and employee evaluation data generated by a large language generation model, namely the first corpus pair is (input, employee self-evaluation data and employee evaluation data); constructing a second corpus pair, namely the second corpus pair is (input, employee direct lead evaluation data, employee evaluation data), by using the model input, the employee direct lead evaluation data and the employee evaluation data generated by the large language generation model, and further constructing a corpus pair for fine tuning the large language generation model according to the first corpus pair and the second corpus pair, namely the first corpus pair and the second corpus pair form a model optimization corpus pairAnd then, the large language generation model is finely adjusted according to the model optimization corpus, so that the finely adjusted model can accurately reflect the characteristics of human evaluation.
Further, it is necessary to collect the evaluations of different persons on the same employee, and manually determine the merits of the different evaluations of the same employee, so as to form (input, good evaluation data, poor evaluation data) corpus pairs, wherein the employee self-evaluation data and the employee direct-lead evaluation data correspond to the good evaluations, and the employee evaluation data generated by the large language generation model correspond to the poor evaluations, so that the fine-tuned model can accurately reflect the characteristics of the human evaluation in consideration of the merits of the different evaluations of the same employee.
In the embodiment of the application, the evaluation generation model is a model which can accurately reflect human evaluation characteristics after fine tuning of the large language generation model based on (input, good evaluation data and poor evaluation data).
In the embodiment of the application, a model optimization corpus pair is input into a large language generation model for fine adjustment to obtain an evaluation generation model, and the method comprises the following steps: determining a reference model and a strategy model according to the large language generation model; inputting the model optimization corpus pairs into a reference model to obtain reference evaluation data; inputting the model optimization corpus pairs into a strategy model to obtain strategy evaluation data; calculating a loss value of the strategy model according to the reference evaluation data and the strategy evaluation data through a preset loss function, wherein the loss function is as follows:
Wherein, For loss value,/>Evaluating cumulative probability of data for policy,/>For reference to the cumulative probability of the evaluation data,/>Is the expected value/>Optimizing a model for direct preference,/>Is a corpus data set,/>Optimizing input parameters in corpus pairs for a model,/>Optimizing dominant evaluation of corpus pairs for the model; /(I)Inferior evaluation in corpus pair for model optimization,/>As a logarithmic function,/>As a sigmoid function,/>Is a parameter factor; and when the loss value is smaller than a preset loss threshold value, taking the large language generation model as an evaluation generation model.
In detail, a reinforcement learning mode is utilized to fine tune a large language generation model, namely DPO (DIRECT PREFERENCE Optimization) is adopted, two models, namely a reference model (REFERENCE MODEL) and a strategy model (policy model), are utilized in the Optimization process, the reference model and the strategy model are the same model when model training is started, the model is a large language generation model, the reference model does not update weight in the training process, the reference model is used as a comparison object, the strategy model parameters are regulated through a gradient descent method in the training process, optimization corpus of the Optimization model is further input into the reference model and the strategy model respectively, so that reference evaluation data probability output by the reference model and strategy evaluation data probability output by the strategy model are obtained, parameter Optimization is carried out on the large language generation model according to an output result of the model according to an optimized loss function, and a model capable of accurately reflecting human evaluation characteristics is obtained.
Specifically, in the loss functionFor sigmoid function/>，/>As parameters, values are generally between 0.1 and 0.5,/>Is for input parameters/>Feedback of better results,/>Is for input parameters/>Feedback of poor results,/>Refers to a given input parameter/>Cumulative probability (sum of probability of each generated character) of good generated evaluation data (good response) generated by the current policy model,/>Is given input parameter/>The cumulative probability (sum of the probabilities of each generated character) of good generated evaluation data (good response) generated by the original reference model, the left half part of the loss function is good generated evaluation data (good response) compared with the cumulative probability difference between the untrained time, the right half part of the loss function represents the bad generated evaluation data (bad response) compared with the cumulative probability difference before untrained time, if the cumulative probability difference is bigger, the left side of the loss function is bigger, the right side is smaller, and ideally, the good generated evaluation data probability is improved, and the bad generated evaluation data probability is decreased; if the left side of the loss function is smaller and the right side is smaller, the probability of good generated evaluation data is reduced, but the probability of poor generated evaluation data is reduced more, and good generated evaluation data is still prone to be generated when the bad generated evaluation data is generated; if the left side of the loss function becomes larger and the right side becomes only a bit larger, the probability of good generated evaluation data decreases, but the probability of poor generated evaluation data decreases more, and the generation tends to be good, so the loss function tends to be more good.
Further, the loss value of the large language generation model is calculated according to the loss function until the loss value is limited to a preset loss threshold value, the large language generation model at the moment is input as an evaluation generation model, and more accurate target evaluation data of target staff can be output by utilizing the evaluation generation model based on the portrait tag and the behavior data of the staff, so that the staff evaluation is completely generated by the model, the bias and the unfair of the manual evaluation are eliminated, the preference of the human evaluation can be fitted, the evaluation is more in line with the actual requirement, the staff evaluation adopts two evaluation systems of the staff portrait tag and the large language model generation evaluation, the staff is found out to have a point in the daily life of the staff, the single index is avoided from being adopted by the traditional evaluation system, and the staff can be evaluated in the whole working process.
Fig. 4 is a functional block diagram of a large model-based evaluation generating device according to an embodiment of the present application.
The large model-based evaluation generating apparatus 400 of the present application may be installed in a device. Depending on the functions implemented, the large model-based evaluation generation apparatus 400 may include an employee portrayal tag generation module 401, an employee behavior data screening module 402, an employee evaluation data output module 403, and a target evaluation data generation module 404. The module of the application, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the device, capable of being executed by the processor of the device and of performing a fixed function.
In the present embodiment, the functions concerning the respective modules/units are as follows:
The employee portrayal tag generation module 401 is configured to generate an employee portrayal tag of a target employee according to a preset employee portrayal tag library and preset target behavior data;
An employee behavior data screening module 402, configured to screen employee behavior data of a target employee according to label scores of employee portrayal labels;
The employee evaluation data output module 403 is configured to input the employee portrait tag and the employee behavior data into a preset large language generation model, and output employee evaluation data;
The target evaluation data generating module 404 is configured to fine tune the large language generating model according to preset bidirectional evaluation data and employee evaluation data, obtain an evaluation generating model, and generate target evaluation data of a target employee by using the evaluation generating model.
In detail, each module in the large model-based evaluation generating device 400 in the embodiment of the present application adopts the same technical means as the large model-based evaluation generating method in fig. 1 to 3, and can generate the same technical effects, which are not described herein.
Fig. 5 is a schematic structural diagram of an electronic device for implementing a large model-based evaluation generation method according to an embodiment of the present application.
The electronic device may include a processor 501, a memory 502, a communication bus 503, and a communication interface 504, and may also include a computer program stored in the memory 502 and executable on the processor 501, such as a large model based evaluation generation program.
The processor 501 may be formed by an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be formed by a plurality of integrated circuits packaged with the same function or different functions, including one or more central processing units (Central Processing unit, CPU), a microprocessor, a digital processing chip, a combination of a graphics processor and various control chips, etc. The processor 501 is a Control Unit (Control Unit) of the electronic device, connects various components of the entire electronic device using various interfaces and lines, and executes various functions of the electronic device and processes data by running or executing programs or modules stored in the memory 502 (for example, executing a large model-based evaluation generation program, etc.), and calling data stored in the memory 502.
Memory 502 includes at least one type of computer-readable storage medium including flash memory, removable hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 502 may in some embodiments be an internal storage unit of the electronic device, such as a mobile hard disk of the electronic device. The memory 502 may also be an external storage device of the electronic device in other embodiments, such as a plug-in mobile hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD), etc. that are provided on the electronic device. Further, the memory 502 may also include both internal storage units and external storage devices of the electronic device. The memory 502 may be used not only for storing application software installed in an electronic device and various types of data, such as code of an evaluation generation program based on a large model, but also for temporarily storing data that has been output or is to be output.
The communication bus 503 may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, or the like. The bus may be classified as an address bus, a data bus, a control bus, etc. The bus is arranged to enable connected communication between the memory 502 and the at least one processor 501 etc.
The communication interface 504 is used for communication between the electronic device and other devices described above, including network interfaces and user interfaces. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), typically used to establish a communication connection between the electronic device and other devices. The user interface may be a Display (Display), an input unit such as a Keyboard (Keyboard), or alternatively a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device and for displaying a visual user interface.
Only an electronic device having components is shown, and it will be understood by those skilled in the art that the structures shown in the figures do not constitute limitations on the electronic device, and may include fewer or more components than shown, or may combine certain components, or a different arrangement of components.
For example, although not shown, the electronic device may further include a power source (such as a battery) for powering the respective components, and the power source may be logically connected to the at least one processor 501 through a power management device, so as to perform functions of charge management, discharge management, and power consumption management through the power management device. The power supply may also include one or more of any of a direct current or alternating current power supply, recharging device, power failure detection circuit, power converter or inverter, power status indicator, etc. The electronic device may also include various sensors, bluetooth modules, wi-Fi modules, etc., which are not described in detail herein.
It should be understood that the examples are for illustrative purposes only and are not limited to this configuration in the scope of the patent application.
In particular, the specific implementation method of the above instruction by the processor 501 may refer to the description of the relevant steps in the corresponding embodiment of the drawings, which is not repeated herein.
Further, the electronic device integrated modules/units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. The storage medium may be volatile or nonvolatile. For example, a computer-readable storage medium may include: any entity or device capable of carrying computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM).
The present application also provides a computer-readable storage medium storing a computer program which, when executed by a processor of an electronic device, can implement the large model-based evaluation generation method of any of the above embodiments. The computer-readable storage medium may be volatile or nonvolatile. For example, a computer-readable storage medium may include: any entity or device capable of carrying computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM).
In the embodiments provided in the present application, it should be understood that the disclosed electronic device, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of modules is merely a logical function division, and other manners of division may be implemented in practice.
The modules illustrated as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.
It will be evident to those skilled in the art that the application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.
The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the foregoing description, and all changes which come within the meaning and range of equivalency of the scope of the application are therefore intended to be embraced therein.
The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Wherein artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) is the theory, method, technique, and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend, and expand human intelligence, sense the environment, acquire knowledge, and use knowledge to obtain optimal results.
Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or means recited in the system claims can also be implemented by means of software or hardware by means of one unit or means. The terms first, second, etc. are used to denote a name, but not any particular order.
Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present application and not for limiting the same, and although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present application without departing from the spirit and scope of the technical solution of the present application.
Claims (10)
1. A method for generating an evaluation based on a large model, the method comprising:
generating employee portrayal labels of target employees according to a preset employee portrayal label library and preset target behavior data;
screening employee behavior data of the target employee according to label scores of the employee portrait labels;
Inputting the employee portrait tag and the employee behavior data into a preset large language generation model, and outputting employee evaluation data;
And fine tuning the large language generating model according to preset bidirectional evaluation data and employee evaluation data to obtain an evaluation generating model, and generating target evaluation data of the target employee by using the evaluation generating model.
2. The large model-based evaluation generation method as claimed in claim 1, wherein the generating the employee portrayal tag of the target employee according to the preset employee portrayal tag library and the preset target behavior data comprises:
Extracting a label name and a label description in a preset employee portrait label library;
dividing preset target behavior data into target paragraph behavior data;
performing semantic conversion on the behavior data of the target paragraph to obtain a behavior data semantic vector;
carrying out semantic conversion on the tag description according to the tag name to obtain a tag description semantic vector;
And generating employee portrait tags of the target employees according to the behavioral data semantic vectors and the tag description semantic vectors.
3. The large model based evaluation generation method according to claim 2, wherein the generating the employee portrayal tag of the target employee from the behavior data semantic vector and the tag description semantic vector comprises:
Calculating a distance value between the behavior data semantic vector and the tag description semantic vector;
generating a distance matrix of the tag description and the target behavior data according to the distance value;
Calculating label scores of the target staff according to the distance values in the distance matrix, wherein the label score calculation formula is as follows:
Wherein, For/>Tag scoring of individual tag descriptions,/>For/>Personal behavior data and No. >Distance value between individual tag descriptions,/>The number of the behavior data;
sorting the tag names corresponding to the tag description according to the tag scores to obtain a tag sorting queue;
and screening portrait labels in the label arrangement queue according to a preset label threshold value to serve as target portrait labels of the target staff.
4. The large model based valuation generation method of claim 3, wherein the screening of employee behavioral data of the target employee based on the label score of the employee portrayal label comprises:
selecting expected behavior data corresponding to the employee portrait tags one by one from the distance matrix;
Generating a label grading list one by one according to the label grading of the employee image labels and the expected behavior data;
and selecting expected behavior data corresponding to the largest label score in the label score list as employee behavior data of the target employee.
5. The large model-based evaluation generation method as claimed in claim 1, further comprising, before said inputting the employee portrayal tag and the employee behavior data into a preset large language generation model, outputting employee evaluation data:
Taking a preset employee evaluation data case as an employee evaluation instruction corpus;
Inputting the employee evaluation instruction corpus into a preset large language model to obtain employee model evaluation data;
calculating an evaluation loss value of the employee model evaluation data through a preset loss function;
And fine tuning model parameters in the large language model through a preset LoRA model and the evaluation loss value to obtain a large language generation model.
6. The big model-based evaluation generation method of claim 1, wherein the fine tuning the big language generation model according to preset bidirectional evaluation data and the employee evaluation data to obtain an evaluation generation model comprises:
extracting employee self-evaluation data and employee direct leading evaluation data in preset bidirectional evaluation data;
Generating a first corpus pair according to the employee self-evaluation data and the employee evaluation data;
generating a second corpus pair according to the employee direct lead evaluation data and the employee evaluation data;
Constructing a model optimization corpus pair according to the first corpus pair and the second corpus pair;
And inputting the model optimization corpus pair into the large language generation model for fine adjustment to obtain an evaluation generation model.
7. The method for generating a large model-based evaluation according to claim 6, wherein the fine-tuning the model optimization corpus input to the large language generation model to obtain the evaluation generation model comprises:
determining a reference model and a strategy model according to the large language generation model;
inputting the model optimization corpus pairs into the reference model to obtain reference evaluation data;
inputting the model optimization corpus pairs into the strategy model to obtain strategy evaluation data;
calculating a loss value of the strategy model according to the reference evaluation data and the strategy evaluation data through a preset loss function, wherein the loss function is as follows:
Wherein, For the loss value,/>Evaluating cumulative probability of data for policy,/>For reference to the cumulative probability of the evaluation data,/>Is the expected value/>Optimizing a model for direct preference,/>Is a corpus data set,/>Optimizing input parameters in corpus pairs for the model,/>Optimizing dominant evaluation of corpus pairs for the model; /(I)Optimizing the disadvantaged evaluation of corpus pairs for the model,/>As a logarithmic function,/>As a sigmoid function,/>Is a parameter factor;
And when the loss value is smaller than a preset loss threshold value, taking the large language generation model as the evaluation generation model.
8. An evaluation generating device based on a large model, the device comprising:
The staff portrait tag generation module is used for generating staff portrait tags of target staff according to a preset staff portrait tag library and preset target behavior data;
The staff behavior data screening module is used for screening staff behavior data of the target staff according to the label scores of the staff portrait labels;
the employee evaluation data output module is used for inputting the employee portrait tag and the employee behavior data into a preset large language generation model and outputting employee evaluation data;
The target evaluation data generation module is used for fine tuning the large language generation model according to preset bidirectional evaluation data and the employee evaluation data to obtain an evaluation generation model, and generating target evaluation data of the target employee by using the evaluation generation model.
9. An electronic device, the electronic device comprising:
At least one processor; and
A memory communicatively coupled to the at least one processor; wherein,
The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the large model-based evaluation generation method of any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the large model-based evaluation generation method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410385149.XA CN117973949B (en) | 2024-04-01 | 2024-04-01 | Evaluation generation method, device, equipment and storage medium based on large model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410385149.XA CN117973949B (en) | 2024-04-01 | 2024-04-01 | Evaluation generation method, device, equipment and storage medium based on large model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117973949A true CN117973949A (en) | 2024-05-03 |
| CN117973949B CN117973949B (en) | 2024-08-02 |
Family
ID=90865068
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410385149.XA Active CN117973949B (en) | 2024-04-01 | 2024-04-01 | Evaluation generation method, device, equipment and storage medium based on large model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117973949B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118643807A (en) * | 2024-08-13 | 2024-09-13 | 北京中数睿智科技有限公司 | Large model synthesized information quality evaluation method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114399158A (en) * | 2021-12-07 | 2022-04-26 | 国能大渡河流域水电开发有限公司 | Analysis method for employee behavior and ability dimension |
| CN116757270A (en) * | 2023-06-28 | 2023-09-15 | 阿里巴巴（中国）有限公司 | Data processing method and server based on man-machine interaction model or large model |
| CN116911683A (en) * | 2023-07-25 | 2023-10-20 | 中国联合网络通信集团有限公司 | Data processing method, device and storage medium |
| CN117151338A (en) * | 2023-09-08 | 2023-12-01 | 安徽大学 | Multi-unmanned aerial vehicle task planning method based on large language model |
-
2024
- 2024-04-01 CN CN202410385149.XA patent/CN117973949B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114399158A (en) * | 2021-12-07 | 2022-04-26 | 国能大渡河流域水电开发有限公司 | Analysis method for employee behavior and ability dimension |
| CN116757270A (en) * | 2023-06-28 | 2023-09-15 | 阿里巴巴（中国）有限公司 | Data processing method and server based on man-machine interaction model or large model |
| CN116911683A (en) * | 2023-07-25 | 2023-10-20 | 中国联合网络通信集团有限公司 | Data processing method, device and storage medium |
| CN117151338A (en) * | 2023-09-08 | 2023-12-01 | 安徽大学 | Multi-unmanned aerial vehicle task planning method based on large language model |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118643807A (en) * | 2024-08-13 | 2024-09-13 | 北京中数睿智科技有限公司 | Large model synthesized information quality evaluation method |
| CN118643807B (en) * | 2024-08-13 | 2024-10-11 | 北京中数睿智科技有限公司 | Large model synthesized information quality evaluation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117973949B (en) | 2024-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110532357B (en) | ESG scoring system generation method, device, equipment and readable storage medium | |
| CN117973949B (en) | Evaluation generation method, device, equipment and storage medium based on large model | |
| US10346926B2 (en) | Context search system | |
| US20060101017A1 (en) | Search ranking system | |
| CN111475645B (en) | Knowledge point labeling method, knowledge point labeling device and computer readable storage medium | |
| CN110310012B (en) | Data analysis method, device, equipment and computer readable storage medium | |
| CN111339285B (en) | BP neural network-based enterprise resume screening method and system | |
| TWI818176B (en) | Planning aids and planning aids | |
| CN115455205A (en) | Time sequence knowledge graph-based occupational development planning method | |
| CN114722281A (en) | Training course configuration method and device based on user portrait and user course selection behavior | |
| CN114997263A (en) | Training rate analysis method, device, equipment and storage medium based on machine learning | |
| CN114139065B (en) | Talent screening and recommending method and system based on big data and readable storage medium | |
| CN110334720A (en) | Feature extracting method, device, server and the storage medium of business datum | |
| TWI833098B (en) | Intellectual quality management method, electronic device and readable storage medium | |
| CN114880449A (en) | Reply generation method and device of intelligent question answering, electronic equipment and storage medium | |
| CN112396092B (en) | Crowdsourcing developer recommendation method and device | |
| US11809980B1 (en) | Automatic classification of data sensitivity through machine learning | |
| Cao et al. | Comprehensive evaluation method of teaching effect based on particle swarm optimization neural network model | |
| US10650339B2 (en) | Assessment method and server for debt management competency | |
| CN113222471B (en) | Asset wind control method and device based on new media data | |
| CN114862141A (en) | Method, device and equipment for recommending courses based on portrait relevance and storage medium | |
| JP2022145570A (en) | Automated empathetic assessment of candidate for job | |
| CN114092057A (en) | Project model construction method and device, terminal equipment and storage medium | |
| CN110909252B (en) | Community activity recommendation method and device based on big data | |
| Elanwer | HR analytics in practice: Using machine learning to predict employee absenteeism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |