CN113537731B - Design resource capability assessment method based on reinforcement learning - Google Patents
Design resource capability assessment method based on reinforcement learning Download PDFInfo
- Publication number
- CN113537731B CN113537731B CN202110710388.4A CN202110710388A CN113537731B CN 113537731 B CN113537731 B CN 113537731B CN 202110710388 A CN202110710388 A CN 202110710388A CN 113537731 B CN113537731 B CN 113537731B
- Authority
- CN
- China
- Prior art keywords
- capability
- service provider
- reinforcement learning
- design
- design resource
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013461 design Methods 0.000 title claims abstract description 131
- 230000002787 reinforcement Effects 0.000 title claims abstract description 63
- 238000000034 method Methods 0.000 title claims abstract description 27
- 230000009471 action Effects 0.000 claims abstract description 24
- 238000011156 evaluation Methods 0.000 claims description 31
- 230000003993 interaction Effects 0.000 claims description 8
- 230000008859 change Effects 0.000 claims description 6
- 230000000875 corresponding effect Effects 0.000 description 24
- 239000003795 chemical substances by application Substances 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000013138 pruning Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000012938 design process Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000013210 evaluation model Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000008713 feedback mechanism Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
Abstract
The invention discloses a design resource capability assessment method based on reinforcement learning, which takes service provider historical transaction data and current design resource data as state data, takes service providers pushing the maximum capability assessment value as actions, adopts a tree structure to construct an experience playback set, assesses the capability breadth score and the capability width score of the service providers in various fields based on reinforcement learning, so as to know the comprehensive capability, growth capability and bearing capability of the service providers, reasonably assesses the service providers not only depending on data but also according to quality based on the self-learning evolution concept of reinforcement learning, can know the comprehensive capability, growth capability and bearing capability of the current service providers, provides more objective capability feedback information for a crowdsourcing platform, and also provides data support of the service providers for personalized pushing service of the crowdsourcing platform.
Description
Technical Field
The invention belongs to the technical field of computer data processing, and particularly relates to a design resource capability assessment method based on reinforcement learning.
Background
Capacity is the basic quality that a certain activity needs to possess. Traditional capability assessment theory divides capabilities into general and special capabilities, where general refers to the ability to cope with basic activities, such as observation and thinking; while special capabilities refer to experience-related capabilities such as design capabilities, promotional capabilities, management capabilities, etc., that are exhibited in engaging in certain specific activities. Thus, the capabilities are diversified, as are the capabilities exhibited by each facilitator/merchant in different design areas. In a service platform, a capability domain is an important component, and has inseparable relation with the operation of the platform, the completion of tasks and the accumulation of knowledge resources.
Design resource service requirements refer to the need to utilize external resource services to complete the product design process, mainly intellectual resource service requirements, knowledge resource service requirements, tool resource service requirements and other service requirements, and the attributes mainly comprise basic attributes, target attributes and service attributes. Therefore, as the number of design tasks performed by the design capability body increases, the design capability will grow continuously, and the growth characteristics determine that the design capability cannot be analyzed and evaluated by means of a static mathematical model. The method is an effective approach for evaluating the design capability level based on the number of completed design tasks and the corresponding task completion performance.
Various types of service providers exist in the crowdsourcing website, the service providers/merchants finish the design requirements of clients from various regions based on the crowdsourcing website, the large and small design resources are displayed on a platform, the correct integration of the design resources of the merchants and the capability range of the merchants becomes an important task, and compared with the 'evaluation' mode of the capability value port of the traditional merchants, the capability evaluation of the platform side to the service providers/merchants provides a more objective and direct grade distribution mode.
However, due to the limitation of the technology in various aspects, the current method still lacks dynamics, for example, the text part in the design resource cannot be quantized into a numerical form, the capability condition of the service provider/merchant is simply mastered according to the completion proportion of the learning item and the user's praise rate statistics data, the capability degree of the service provider/merchant to the specific field cannot be known, and the capability degree of the single service provider/merchant cannot be mastered. Therefore, many crowdsourcing platforms have no targeted effect evaluation and feedback mechanism for evaluating design resources in specific fields, and the problems that capability evaluation is inaccurate and clients cannot really know the level of service providers/merchants are presented, so that excellent design resources on the crowdsourcing platforms are difficult to mine and emerging design resources are difficult to develop.
The ability to evaluate design resource capability units cannot be based solely on the performance of their single design task or on the design capability unit test results at a certain time node, but rather the design capability influencing factors should be analyzed from multiple levels and given comprehensive judgment. The ability is a comprehensive quality expression, and the ability is characterized by both integrity and distinguishing property during evaluation, and can be used for explaining the comprehensive ability of individuals and expressing the advantages and the good aspects thereof.
Disclosure of Invention
The invention aims to provide a design resource capacity assessment method based on reinforcement learning, which is based on the characteristics of reinforcement learning self-evolution, dynamically designs a design resource capacity assessment method from multiple angles from the historical design task of a service provider, analyzes the design capacity of the service provider and gives out comprehensive judgment.
The invention is realized by adopting the following technical scheme:
the design resource capacity assessment method based on reinforcement learning comprises the following steps: acquiring historical transaction data and design resource data of service providers in a crowdsourcing platform; obtaining the designed resource characteristics of the service provider based on the following steps: extracting design resource commonalities obtained by the design resource information data, quantifying the historical transaction data to obtain transaction grades, extracting the historical transaction data to obtain breadth resource division data, extracting description tags obtained by the design resource information data, and obtaining capability assessment results obtained by reinforcement learning at the previous moment; constructing a tree structure set in time order as an experience playback set for reinforcement learning; taking a service provider as a unit, taking historical transaction data of the service provider and design resource characteristics of the service provider as reinforcement learning states, taking the service provider pushing the maximum capability assessment value as reinforcement learning actions, and obtaining design resource capability assessment results of the service provider based on reinforcement learning.
Further, the service provider pushing the maximum capability assessment value is an action of reinforcement learning, specifically:
designing a feedforward function f (s, action) =eval_score; where f ' (s, s ', report) =action, s is the state before the interaction of the environment agent, s ' is the state after the interaction of the environment agent, report is the Reward given by reinforcement learning, action is the recommendation probability, and eval_score is the capability evaluation value.
Further, the method further comprises: arranging the capability evaluation values corresponding to the design resource features obtained by reinforcement learning according to the generation time to obtain score sequences corresponding to the design resource features; obtaining the capability assessment result of the service provider according to the first element in each score sequence; and/or obtaining the capability growth result of the service provider on the design resource according to the numerical value change of each element of each score sequence; and/or obtaining the capability bearing result of the service provider on the design resource according to the positions of the elements larger than the preset value in the score sequence; and/or obtaining the familiarity result of the service provider to the ability of designing the resource according to the time intervals of each element of the score sequence.
Further, extracting the historical transaction data to obtain breadth resource division data specifically includes: classifying design resource data of the service providers in a grading way; according to the classification, the corresponding ability evaluation values are arranged according to the generation time, and sub-score sequences of all the classifications are obtained; and storing each sub-score sequence in a parent-child tree structure to obtain a capacity growth structure model of the service provider.
Further, obtaining the capability assessment result of the service provider according to the first element in each score sequence, including: obtaining the latest capacity score of the service provider in the field according to the first element in each score sequence; and (5) taking an average value of all the capability scores in the score sequence, and evaluating the overall capability value of the designed resources of the service provider in the field.
Further, according to the numerical value change of each element of each score sequence, the capability growth result of the service provider on the design resource is obtained, specifically: and comparing the latest capability scores and the overall capability values of the score sequences to obtain the growth speed of the designed resources of the service providers in the field.
Compared with the prior art, the invention has the advantages and positive effects that: in the design resource capacity assessment method based on reinforcement learning, historical transaction data of a service provider and current design resource data are used as state data, the service provider pushing the maximum capacity assessment value is used as action, an experience playback set is constructed by adopting a tree structure, the capacity breadth score and the capacity breadth score of the service provider are assessed in each field based on reinforcement learning, so that the comprehensive capacity, the growth capacity and the bearing capacity of the service provider are known, the concept of self-learning evolution based on reinforcement learning is adopted, the service provider is reasonably assessed according to quality and not only the comprehensive capacity, the growth capacity and the bearing capacity of the current service provider are known, more objective capacity feedback information is provided for a crowdsourcing platform, and the data support of the service provider is provided for personalized pushing service of the crowdsourcing platform.
Other features and advantages of the present invention will become more apparent from the following detailed description of embodiments of the present invention, which is to be read in connection with the accompanying drawings.
Drawings
FIG. 1 is a flow chart of a reinforcement learning-based design resource capability assessment method according to the present invention;
FIG. 2 is a schematic tree structure of an experience playback set according to the present invention;
FIG. 3 is a schematic diagram of a reinforcement learning model according to the present invention;
FIG. 4 is a tree structure diagram of an experience playback set according to an embodiment of the present invention.
Detailed Description
The following describes the embodiments of the present invention in further detail with reference to the drawings.
Reinforcement learning is one area of machine learning, which is learning "what to do (i.e., how to map a current scenario to an action) to maximize a digitized revenue signal. The learner is not informed of what actions should be taken, but must himself by trying to find out which actions will produce the most earnings. Reinforcement learning tradeoffs between "heuristics" and "development," an agent develops an existing experience to obtain revenue while heuristics are made so that better action selection space (i.e., learning from errors) can be obtained in the future.
The resources in the current crowdsourcing platform are numerous, the capability values of the resources are difficult to be defined by manpower in a full coverage manner, the idea of reinforcement learning provides an exploration mechanism, no interaction exists between clients (task publishers) and design resource providers (service providers/merchants) in the crowdsourcing platform, the reinforcement learning can completely combine the clients and the design resource providers, and task requests submitted by the clients search services meeting the requirements of the clients through the crowdsourcing service platform, so that from the service perspective of the crowdsourcing platform, certain correlation exists between design tasks and design capabilities, and the correlation can be called as task-capability matching degree.
Based on the above, the present invention proposes a design resource capability assessment method based on reinforcement learning, as shown in fig. 1, including:
step S1: historical transaction data and design resource data of service providers in the crowdsourcing platform are obtained.
The design resource data of the service provider is derived from the data uploaded by the service provider in various fields; the design resources are used for describing the content of each service provided by the service provider, the related fields, the corresponding knowledge points and the like, and can correspond to the pre-specification of different design tasks, and can also be extracted from the description files of the design resources.
When extracting from the description file of the design resource, the corresponding design resource description label can be obtained by adopting a natural language/structured text processing mode according to the format and structure of the description text. The extraction of keywords includes TF-IDF algorithm, etc.
A category label for one design resource may be applied to a plurality of design resources, and one design item may correspond to a plurality of design resource category labels.
Step S2: obtaining the designed resource characteristics of the service provider based on the following steps: extracting design resource commonalities obtained by design resource information data, quantifying historical transaction data to obtain transaction grades, extracting historical transaction data to obtain breadth resource division data, extracting description labels obtained by the design resource information data, and obtaining capability assessment results obtained by reinforcement learning at the previous moment.
The commonality of design resources indicates the similarity of tasks, e.g. service providers providing finishing services all have the basic service of painting a wall.
The transaction level of the transaction data is divided according to the size of the transaction amount.
The breadth is divided according to transaction data, the total capacity of the service providers is reflected by the width resources, the crowdsourcing platform classifies various design resources and basic information of the commercial tenant, the service provider capacity evaluation index system is utilized to carry out comprehensive capacity grade division on the data of the service providers, the breadth capacity value of the service providers to the design resources in a technical field can be evaluated in more dimensions, and the comprehensive capacity value can be understood.
The resource category labels are attribute labels provided by the crowdsourcing platform, such as that the finishing service belongs to engineering services, etc.
Step S3: and constructing a tree structure set in time order as an experience playback set for reinforcement learning.
In the design resource capability assessment method provided by the invention, the experience playback set of reinforcement learning is realized by adopting a tree structure set, as shown in fig. 2, the capability assessment value obtained by reinforcement learning of each design resource of a service provider forms a score sequence in time sequence and is stored in one branch of the tree structure.
Step S4: taking a service provider as a unit, taking historical transaction data of the service provider and design resource characteristics of the service provider as reinforcement learning states, taking the service provider pushing the maximum capability assessment value as reinforcement learning actions, and obtaining design resource capability assessment results of the service provider based on reinforcement learning.
In the application of the invention, historical transaction data of the service provider and design resource characteristics of the service provider are used as state data S of the reinforcement learning environment, the service provider pushing the maximum capability assessment value is used as action A of reinforcement learning (namely, the service provider with the maximum score is pushed to a client according to the capability assessment value of the service provider), a neural network (agent) in the reinforcement learning environment is applied as a capability assessment membership function, and the reinforcement learning is adopted to obtain the design resource capability assessment result of the service provider.
In the embodiment of the invention, the action of pushing the service provider with the maximum capability assessment value as reinforcement learning is realized by adopting the following means:
designing a feedforward function f (s, action) =eval_score; where f ' (s, s ', report) =action, s is the state before the interaction of the environment agent, s ' is the state after the interaction of the environment agent, report is the Reward given by reinforcement learning, action is the recommendation probability, and eval_score is the capability evaluation value.
Specifically, the reinforcement learning process includes:
1) Randomly initializing all the state S and the capacity evaluation value Q corresponding to the action A, randomly initializing all the parameters w of the current Q network (capacity evaluation value network), initializing the parameters w 'of the target Q network Q', and emptying the experience playback set D;
2) Initializing a state S to be the first state of a current state sequence to obtain a characteristic vector phi (S) of the state S;
3) Using phi(s) as input in a Q network to obtain Q value output corresponding to all actions of the Q network, and selecting a corresponding action A in the current Q value output by using an epsilon-greedy method;
4) Executing the current action A in the state S to obtain a feature vector phi (S ') and a reward R corresponding to the new state S', and obtaining whether to terminate the state is_end;
5) Storing { phi(s), A, R, phi (s'), is_end } in the experience playback set D;
6)s＝s′；
7) Sampling m samples { phi (Sj), aj, rj, phi (S' j), is_endj }, j=1, 2,, m from the experience playback set D, calculating the current target Q value
8) Using a mean square error loss functionUpdating all parameters w of the Q network by gradient back propagation of the neural network;
9) If T% c=1, updating the target Q network parameter w' =w; wherein T is time, C is a self-defined period;
10 If S' is the end state, the current round of iteration is completed, otherwise go to step 3).
As shown in the design resource capacity evaluation model based on reinforcement learning in the figure 3, the invention adopts a reinforcement learning method to promote the self-evolution mechanism of the evaluation standard, and in the aspect of service provider characteristics, due to the specificity of the reinforcement learning algorithm, the characteristics of each service provider are ensured to be independent of other service providers, and the phenomenon of characteristic crossing and the influence of capacity evaluation are not generated; extracting the characteristics of each service provider by the agent, assigning weights to the characteristics and carrying out characteristic fusion calculation to finally generate eval_score; the state-ACTION interaction mechanism of reinforcement learning enables an algorithm to calculate ACTION according to a certain rule, replaces a traditional reinforcement learning artificial design ACTION-SPACE part, enables the method to self-learn to a self-evolution value by adding a full-connection layer, takes the self-evolution value as a part of a service provider individual, and can be a single numerical value or a one-dimensional vector, and the value is initially assigned to be 1.
The method for evaluating the design resource capacity based on reinforcement learning according to the present invention will be described in detail in two specific embodiments.
Example 1
The crowdsourcing platform generally sets a variety of classifications of design resources according to the classifications of the design resources, sets different capability evaluation standards for each design resource, and gives different capability evaluation values according to different design resources of a service provider, including the following steps:
1) Historical transaction data and design resource data of service providers in the crowdsourcing platform are obtained.
2) Preprocessing design resource data: extracting design resource commonalities obtained by design resource information data, quantifying historical transaction data to obtain transaction grades, extracting historical transaction data to obtain breadth resource division data, extracting description labels obtained by the design resource information data, and obtaining capability assessment results obtained by reinforcement learning at the previous moment.
3) Reinforcement learning is employed to evaluate design resource capabilities on a service.
In this embodiment, the capacity evaluation of the design resources is performed on the capacity evaluation scores given by the order forming capacity of the service provider and the design resources of the belonging order in a current period, the scores may rise or fall according to corresponding data as time and algorithm evolves, and each service provider may relate to a plurality of fields, so that the capacity evaluation values obtained by each field are different, as shown in fig. 4, the score sequence of the design resources may be regarded as sub-scores of the fields, and after the capacity evaluation of all the design resource sub-scores of one service provider is completed, the total scores of the service provider may be given according to all the scores.
4) And arranging the capability evaluation values corresponding to the design resource features in the field obtained by reinforcement learning according to the generation time to obtain a sub-score sequence corresponding to the design resource features.
Specifically, the capability assessment of a design resource shows the capability degree of a service provider in a recent transaction and the field, and the capability assessment value obtained by a certain service provider in the design resource category of a certain field when the service provider is involved for the first time can integrally show the familiarity and mastering degree of the service provider on the design resource, and the higher the score is, the better the service provider is on the basic capability value of the design resource of the field, the capability assessment scores corresponding to the obtained design resources are arranged in time sequence (from big date to small date) to obtain a corresponding sub-score sequence.
5) And obtaining the capability assessment result of the service provider in one field according to the first element in each sub-score sequence.
Obtaining the latest capacity score of the service provider in the field according to the first element in each sub-score sequence; and then taking an average value of all the capacity scores in the sub-score sequences, and evaluating the overall capacity value of the designed resources of the service provider in the field.
And/or 6) obtaining the capability growth result of the service provider on the design resource according to the numerical value change of each element of each sub-score sequence.
And comparing the latest capability scores and the overall capability values of the sub-score sequences to obtain the capability growth speed of the designed resources of the service providers in the field.
In order to refine the calculation, a weighting value can be set for design resources of the corresponding domain according to the difference between the domains in the calculation.
And/or 7) obtaining the capability bearing result of the service provider on the design resource according to the positions of the elements larger than the preset value in the sub-score sequence.
For the design resource capacity evaluation value in a certain field, a preset value (threshold value) can be used for judging whether the current capacity of a service provider is enough to meet the current requirement, and if the current capacity of the service provider is lower than the threshold value, a third-party training auxiliary capacity training can be provided in the platform; in this embodiment, the score greater than the preset value in the sub-score sequence is obtained, and the judging server has enough capability to provide the corresponding service, and in specific calculation, weighting operation can be performed by using the number of orders, customer evaluation, and shop month average flow rate, etc. completed in the domain by all the servers in the crowd-sourced platform, where the design resources provided by the servers in the domain are considered to have a certain guiding property and the servers have enough bearing power on the services in the domain under the condition that the above properties are continuously increased.
And/or 8) obtaining the ability familiarity result of the service provider to the design resource according to the time intervals of each element of the sub-score sequence.
When a service provider performs order transaction on a certain field, the number of orders increases and decreases along with the advancement of time, the embodiment provides an order interval threshold, determines that when the orders of the service provider before and after the field exceed the threshold, calculates an interval weighted value according to the threshold and the total number of orders to multiply with the capacity evaluation value of the service provider, and if the threshold is not exceeded, calculates the time of continuous orders in an accumulated way, thereby calculating the familiarity value of the service provider on the design resources of the field.
Example two
According to the method and the system, the capability level of the service provider is evaluated according to design resources submitted by the service provider and merchant information on the platform line, wherein the capability level comprises the total capability breadth of the service provider and the capability width of various design resources, and capability growth values, capability bearing values and capability familiarity values of the service provider on the design resources in various fields can be given.
The method comprises the following steps:
1) And acquiring a set of data of the service providers in the crowdsourcing platform, wherein the set comprises historical transaction data of the service providers and design resource data of the service providers.
2) And classifying various design resources according to the crowdsourcing platform and classifying basic information of merchants, and classifying comprehensive capacity from coarse to fine on-service data by utilizing a service provider capacity evaluation index system.
In this embodiment, three-level division from coarse to fine is included, including a first level index, a second level index, and a third level tabulation.
3) And obtaining the breadth capability assessment result of the service provider according to the three-level index by adopting reinforcement learning.
In the embodiment, three-level index division is performed on the design resources of each field of the service provider, so that the breadth capability value of the service provider on the design resources in one field can be evaluated in more dimensions; in this embodiment, three-level classification from large to small is performed, and three-level indexes are obtained according to data values (according to operation conditions, operation capacity and operation resources) corresponding to service providers on the crowdsourcing platform. The three-level indexes are divided and stored according to a father-son tree structure, wherein the first-level indexes represent the profitability of a service provider, the second-level indexes are divided into three independent indexes according to the first-level indexes, namely operation factors, external environment and competing environment, and the three-level indexes are also the sufficient and necessary conditions for the fine division of the second-level indexes, such as the manpower, material resources, financial resources and other service provider operation factors of the service provider. Classification other than the above may be based on the service provider domain. Compared with the primary index and the tertiary index, the secondary index has a more moderate range, and the secondary index sub-score is selected as the width capability basis of the evaluation service provider, so that the design resource capability evaluation value of most service providers in the middle position interval can be obtained.
4) And arranging the corresponding capability evaluation values according to the generation time according to the grades, and obtaining the sub-score sequences of each grade.
5) And storing each sub-score sequence in a parent-child tree structure to obtain a capacity growth structure model of the service provider.
Storing each sub-score sequence in a father-son tree structure to obtain a capacity growth structure model of the service provider, and applying corresponding pruning (such as an algorithm for reducing pruning REP (reduced pruning REP), complex pruning CCP (CCP) with cost and the like) to reduce the growth error of the capacity value, wherein the capacity growth tree also avoids data redundancy.
Specifically, the capability numerical value sequence corresponding to each service provider is input into a capability growth structure model (SumToee), a continuously updated and iterated tree structure can be obtained, three-level indexes of a certain service provider are updated according to a corresponding algorithm along with the increase of the transaction amount of the service provider on a crowdsourcing platform to obtain the capability value of the service provider, the tree growth model simultaneously considers the algorithm structures such as data redundancy, data abnormality, data update and the like, the situation that a merchant has a change in part of transactions can be considered, the service provider can be dynamically modeled, and accordingly, the capability difference of each service provider can be displayed objectively, clearly and positively.
Because the transaction attributes of the fields are different, the order update time length of different service providers is also different, and in some embodiments of the invention, the capacity growth and familiarity result of the service providers in the three-level index is obtained according to the change in a single period according to the total number of monthly transactions in the fields as a period.
In the embodiment of the invention, the rules and the corresponding data structures are encapsulated into the corresponding modules in the reinforcement learning algorithm, wherein the algorithm of each index is used as a sub-function for state update, the result is used as a sub-vector in the state vector, the tree structure is used as a storage mode of the memory module, and thus a closed loop evolution process of the system is formed, and the DQN is selected as the self-evolution algorithm, so that the current algorithm has an experience playback function, namely, the weight is evaluated according to the latest memory learning network.
It should be noted that the above description is not intended to limit the invention, but rather the invention is not limited to the above examples, and that variations, modifications, additions or substitutions within the spirit and scope of the invention will be within the scope of the invention.
Claims (3)
1. A reinforcement learning-based design resource capability assessment method, comprising:
acquiring historical transaction data and design resource data of service providers in a crowdsourcing platform;
obtaining the designed resource characteristics of the service provider based on the following steps: extracting design resource commonalities obtained by the design resource information data, quantifying the historical transaction data to obtain transaction grades, extracting the historical transaction data to obtain breadth resource division data, extracting description tags obtained by the design resource information data, and obtaining capability assessment results obtained by reinforcement learning at the previous moment; the design resource commonality indicates the similarity of tasks, and the transaction grades are divided according to the transaction amount;
constructing a tree structure set in time order as an experience playback set for reinforcement learning;
taking a service provider as a unit, taking historical transaction data of the service provider and design resource characteristics of the service provider as reinforcement learning states, taking the service provider pushing the maximum capability assessment value as reinforcement learning actions, and obtaining design resource capability assessment results of the service provider based on reinforcement learning;
the service provider pushing the maximum capability assessment value is the action of reinforcement learning, specifically:
design of feed forward functionThe method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>S is the state before interaction of the environment agent, +.>For the state after the interaction of the environment agents, reward is rewarded given by reinforcement learning, action is recommendation probability, eval_score is capability evaluation value;
the method further comprises the steps of:
arranging the capability evaluation values corresponding to the design resource features obtained by reinforcement learning according to the generation time to obtain score sequences corresponding to the design resource features;
obtaining the capability assessment result of the service provider according to the first element in each score sequence;
obtaining the capability growth result of the service provider on the design resource according to the numerical value change of each element of each score sequence;
obtaining the capability bearing result of the service provider on the design resource according to the positions of the elements larger than the preset value in the score sequence;
obtaining the ability familiarity result of the service provider to the design resource according to the time intervals of each element of the score sequence;
extracting the historical transaction data to obtain breadth resource division data, which comprises the following steps:
classifying design resource data of the service providers in a grading way;
according to the classification, the corresponding ability evaluation values are arranged according to the generation time, and sub-score sequences of all the classifications are obtained;
and storing each sub-score sequence in a parent-child tree structure to obtain a capacity growth structure model of the service provider.
2. The reinforcement learning-based design resource ability assessment method according to claim 1, wherein obtaining the ability assessment result of the facilitator from the first element in each score sequence comprises:
obtaining the latest capacity score of the service provider in the field according to the first element in each score sequence;
and (5) taking an average value of all the capability scores in the score sequence, and evaluating the overall capability value of the designed resources of the service provider in the field.
3. The reinforcement learning-based design resource capability assessment method according to claim 2, wherein the capability growth result of the service provider on the design resource is obtained according to the numerical variation of each element of each score sequence, specifically:
and comparing the latest capability scores and the overall capability values of the score sequences to obtain the growth speed of the designed resources of the service providers in the field.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110710388.4A CN113537731B (en) | 2021-06-25 | 2021-06-25 | Design resource capability assessment method based on reinforcement learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110710388.4A CN113537731B (en) | 2021-06-25 | 2021-06-25 | Design resource capability assessment method based on reinforcement learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113537731A CN113537731A (en) | 2021-10-22 |
| CN113537731B true CN113537731B (en) | 2023-10-27 |
Family
ID=78096781
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110710388.4A Active CN113537731B (en) | 2021-06-25 | 2021-06-25 | Design resource capability assessment method based on reinforcement learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113537731B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114386794A (en) * | 2021-12-28 | 2022-04-22 | 中国电子技术标准化研究院华东分院 | Evaluation method for classification and grading of resource pool of industrial internet service provider |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105931055A (en) * | 2016-04-05 | 2016-09-07 | 扬州大学 | Service provider feature modeling method for crowdsourcing platform |
| WO2018176179A1 (en) * | 2017-03-27 | 2018-10-04 | 杭州呯嘭智能技术有限公司 | Rapid settlement method for cross-border payment based on big credit data |
| CN108876188A (en) * | 2018-07-05 | 2018-11-23 | 阿里巴巴集团控股有限公司 | One inter-species even served business quotient's methods of risk assessment and device |
| CN110365568A (en) * | 2019-06-18 | 2019-10-22 | 西安交通大学 | A kind of mapping method of virtual network based on deeply study |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108230057A (en) * | 2016-12-09 | 2018-06-29 | 阿里巴巴集团控股有限公司 | A kind of intelligent recommendation method and system |
| CA3044327A1 (en) * | 2018-05-25 | 2019-11-25 | Royal Bank Of Canada | Trade platform with reinforcement learning network and matching engine |
-
2021
- 2021-06-25 CN CN202110710388.4A patent/CN113537731B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105931055A (en) * | 2016-04-05 | 2016-09-07 | 扬州大学 | Service provider feature modeling method for crowdsourcing platform |
| WO2018176179A1 (en) * | 2017-03-27 | 2018-10-04 | 杭州呯嘭智能技术有限公司 | Rapid settlement method for cross-border payment based on big credit data |
| CN108876188A (en) * | 2018-07-05 | 2018-11-23 | 阿里巴巴集团控股有限公司 | One inter-species even served business quotient's methods of risk assessment and device |
| CN110365568A (en) * | 2019-06-18 | 2019-10-22 | 西安交通大学 | A kind of mapping method of virtual network based on deeply study |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113537731A (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Provost et al. | Data Science for Business: What you need to know about data mining and data-analytic thinking | |
| Wang et al. | Developing a fuzzy TOPSIS approach based on subjective weights and objective weights | |
| CN104268292B (en) | The label Word library updating method of portrait system | |
| Carrasco et al. | A fuzzy linguistic RFM model applied to campaign management | |
| Akerkar et al. | Intelligent techniques for data science | |
| Alsaadi et al. | Data-driven effort estimation techniques of agile user stories: a systematic literature review | |
| Onan et al. | The use of data mining for strategic management: a case study on mining association rules in student information system | |
| Ramesh et al. | Hybrid artificial neural networks using customer churn prediction | |
| CN113537731B (en) | Design resource capability assessment method based on reinforcement learning | |
| CN112598405B (en) | Business project data management method and system based on big data | |
| Hain et al. | The promises of Machine Learning and Big Data in entrepreneurship research | |
| Tang et al. | Multiple criteria group decision-making based on hesitant fuzzy linguistic consensus model for fashion sales forecasting | |
| Kumar et al. | Exfoliating decision support system: a synthesis of themes using text mining | |
| CN115631008B (en) | Commodity recommendation method, device, equipment and medium | |
| Hu et al. | Investor preference analysis: An online optimization approach with missing information | |
| Liu et al. | Finding the next interesting loan for investors on a peer-to-peer lending platform | |
| Thangam et al. | A Digital Assistant for Stock Market Prediction | |
| TWI419071B (en) | Active knowledge management system, method and computer program product for problem solving | |
| Mattyasovszky-Philipp et al. | Adaptive/cognitive resonance and the architecture issues of cognitive information systems | |
| Daniati et al. | TOPSIS in Decision-Making Framework Based on Twitter Sentiment Analysis | |
| Li et al. | Key product review extraction based on the RLRSA-CL algorithm | |
| CN117422314B (en) | Enterprise data evaluation method and equipment based on big data analysis | |
| Sharma | Identifying Factors Contributing to Lead Conversion Using Machine Learning to Gain Business Insights | |
| Kart et al. | Decision support system for a customer relationship management case study | |
| Wu et al. | Data mining and simulation: a grey relationship demonstration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |