CN117205548A

CN117205548A - Scoring algorithm evaluation method and related device

Info

Publication number: CN117205548A
Application number: CN202311222232.7A
Authority: CN
Inventors: 刘兆丰; 祁彦星; 韩飞扬; 赵琦; 郑力
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2023-09-20
Filing date: 2023-09-20
Publication date: 2023-12-12

Abstract

The embodiment of the application discloses a scoring algorithm evaluation method and a related device in the field of artificial intelligence, wherein the method comprises the following steps: creating a plurality of virtual objects for operating a target game, configuring initial test scores for the plurality of virtual objects respectively, then controlling the plurality of virtual objects to participate in a plurality of rounds of simulation tests based on the target game, distributing game plays for the virtual objects based on the current test scores of the virtual objects in each round of simulation tests, adopting a scoring algorithm to be evaluated, updating the test scores of the virtual objects according to the game play results of the virtual objects in the game plays, and determining accuracy data and convergence speed data corresponding to the scoring algorithm based on the game play results of the plurality of virtual objects in the game plays and the test score ranks of the plurality of virtual objects in each round of simulation tests so as to determine the suitability of the scoring algorithm relative to the target game. Thus, the accuracy and reliability of the evaluation of the scoring algorithm are improved.

Description

Scoring algorithm evaluation method and related device

Technical Field

The application relates to the technical field of computers, in particular to a scoring algorithm evaluation method and a related device.

Background

In recent years, with the increasing development of computer technology and internet technology, the state of competitive game is continuously enriched and perfected. In athletic games, a player matching mechanism is a very important link that aims to help players find opponents or teammates of comparable levels to participate in the same game play, and the player matching mechanism affects the fairness of the game play and the game experience of the players to a great extent.

Modeling predictions of player gaming levels are of great importance in the design of player matching mechanisms. Generating scoring results of the player's game level as accurately and rapidly as possible based on historical game data is a long-standing research focus in the industry; various scoring algorithms for player gaming levels have been developed, including but not limited to ELO, glicko, trueSkill, etc. However, the industry currently has no evaluation scheme for the existing scoring algorithm, and the suitability between the scoring algorithm and the game cannot be accurately and reliably determined.

Disclosure of Invention

The embodiment of the application provides a scoring algorithm evaluation method and a related device, which are used for accurately and reliably determining suitability between a scoring algorithm and a game.

The first aspect of the application provides a scoring algorithm evaluation method, which comprises the following steps:

Creating a plurality of virtual objects for operating the target game, and respectively configuring initial test scores for the plurality of virtual objects;

controlling a plurality of virtual objects to participate in a plurality of rounds of simulation tests based on the target game; in each round of simulation test, based on the current test score of the virtual object, distributing game play for the virtual object, and updating the test score of the virtual object according to the game play result of the virtual object in the game play by adopting a scoring algorithm to be evaluated;

determining accuracy data and convergence speed data corresponding to a scoring algorithm based on game play results of a plurality of virtual objects in game play in each round of simulation test and test score ranking of the plurality of virtual objects;

and determining the suitability of the scoring algorithm relative to the target game according to the accuracy data and the convergence speed data corresponding to the scoring algorithm.

The second aspect of the present application provides a scoring algorithm evaluation device, the device comprising:

the object creation module is used for creating a plurality of virtual objects for operating the target game, and respectively configuring initial test scores for the plurality of virtual objects;

the simulation test module is used for controlling the plurality of virtual objects to participate in a plurality of rounds of simulation tests based on the target game; in each round of simulation test, based on the current test score of the virtual object, distributing game play for the virtual object, and updating the test score of the virtual object according to the game play result of the virtual object in the game play by adopting a scoring algorithm to be evaluated;

The evaluation index determining module is used for determining accuracy data and convergence speed data corresponding to a scoring algorithm based on game results of a plurality of virtual objects in game games in each round of simulation test and test score ranking of the plurality of virtual objects;

and the evaluation module is used for determining the suitability of the scoring algorithm relative to the target game according to the accuracy data and the convergence speed data corresponding to the scoring algorithm.

A third aspect of the application provides a computer apparatus comprising a processor and a memory:

the memory is used for storing a computer program;

the processor is configured to execute the steps of the scoring algorithm evaluation method according to the first aspect described above according to the computer program.

A fourth aspect of the present application provides a computer-readable storage medium storing a computer program for executing the steps of the scoring algorithm evaluation method of the first aspect described above.

A fifth aspect of the application provides a computer program product or computer program comprising computer instructions stored on a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of the scoring algorithm evaluation method described in the first aspect.

From the above technical solutions, the embodiment of the present application has the following advantages:

the scoring algorithm evaluation method provided by the embodiment of the application comprises the steps of firstly creating a plurality of virtual objects for operating a target game, and respectively configuring initial test scores for the plurality of virtual objects; then, a plurality of virtual objects are controlled to participate in multiple rounds of simulation tests based on the target game, game play is distributed to the virtual objects based on the current test scores of the virtual objects in each round of simulation tests, a scoring algorithm to be evaluated is adopted, and the test scores of the virtual objects are updated according to game play results of the virtual objects in the game play; based on the game results of the multiple virtual objects in the game games in each round of simulation test and the test score ranking of the multiple virtual objects, determining accuracy data and convergence speed data corresponding to the scoring algorithm, and determining the suitability of the scoring algorithm relative to the target game according to the accuracy data and the convergence speed data corresponding to the scoring algorithm. In the embodiment of the application, the game results and the test score ranks of a plurality of virtual objects in game games are comprehensively considered, and accuracy data and convergence speed data corresponding to a scoring algorithm are determined, wherein the accuracy data can represent the matching degree between the test score given by the scoring algorithm for the virtual object and the real game level of the virtual object, the convergence speed data can represent the time required by the scoring algorithm for determining the test score matched with the real game level of the virtual object, and further, the accuracy data and the convergence speed data are used as evaluation indexes of the suitability of the scoring algorithm relative to a target game, so that whether the scoring algorithm is suitable for the target game is accurately evaluated, and the accuracy and the reliability of evaluation of the scoring algorithm are improved.

Drawings

Fig. 1 is a schematic view of a scenario of a scoring algorithm evaluation method provided in an embodiment of the present application;

FIG. 2 is a flowchart of a scoring algorithm evaluation method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a convergence speed of a winning rate according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a scoring algorithm evaluation framework according to an embodiment of the present application;

FIG. 5a is a schematic diagram of an ELO algorithm evaluation index according to an embodiment of the present application;

FIG. 5b is a schematic diagram of a Glicko algorithm evaluation index according to an embodiment of the present application;

fig. 5c is a schematic diagram of a TrueSkill algorithm evaluation index provided by an embodiment of the present application;

fig. 6 is a schematic structural diagram of a scoring algorithm evaluation device according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

In order to make the present application better understood by those skilled in the art, the following description will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In a wide variety of competitive games, a player matching mechanism is a very important link that aims to help players find opponents or teammates of comparable levels to participate in the same game play, and the player matching mechanism affects the fairness of the game play and the game experience of the players to a great extent.

As an example, assuming that player a plays a one-to-one competitive game, player a plays a level of 50, player a may play a level of 52 to player B, player a is comparable to player B's level, and experience of player a and player B may be enhanced; assuming that the game is a multi-player independent competitive game, the game level of player C is 30, the game levels of opponent player D and opponent player F of player C can be 31 and 29, and the game levels of player C, player D and player F are equivalent, so that the game experience of player C, player D and player F can be ensured.

In the related art, in order to ensure fairness of game play and game experience of players, game levels of players are estimated generally through a scoring algorithm, and then players with close game levels are regarded as teammates or opponents in the same game play, a scoring algorithm for game levels of various players has been developed, including but not limited to ELO, glicko, trueSkill and the like.

However, different games may include different rules and emphasis, and if a scoring algorithm with low suitability to the game is adopted, the true game level of the player may not be estimated accurately, and further the fairness of the game to the game and the game experience of the player may be affected. The industry currently has no evaluation scheme aiming at the existing scoring algorithm, and the suitability between the scoring algorithm and the game cannot be accurately and reliably determined.

In order to solve the above technical problems, an embodiment of the present application provides a scoring algorithm evaluation method and a related device, where the method includes: creating a plurality of virtual objects for operating a target game, configuring initial test scores for the plurality of virtual objects respectively, then controlling the plurality of virtual objects to participate in a plurality of rounds of simulation tests based on the target game, distributing game play for the virtual objects based on the current test scores of the virtual objects in each round of simulation tests, adopting a scoring algorithm to be evaluated, updating the test scores of the virtual objects according to the game play results of the virtual objects in the game play, determining accuracy data and convergence speed data corresponding to the scoring algorithm based on the game play results of the plurality of virtual objects in each round of simulation tests and the test score ranks of the plurality of virtual objects, and determining suitability of the scoring algorithm relative to the target game according to the accuracy data and the convergence speed data corresponding to the scoring algorithm.

In the embodiment of the application, the game results and the test score ranks of a plurality of virtual objects in game games are comprehensively considered, and accuracy data and convergence speed data corresponding to a scoring algorithm are determined, wherein the accuracy data can represent the matching degree between the test score given by the scoring algorithm for the virtual object and the real game level of the virtual object, the convergence speed data can represent the time required by the scoring algorithm for determining the test score matched with the real game level of the virtual object, and further, the accuracy data and the convergence speed data are used as evaluation indexes of the suitability of the scoring algorithm relative to a target game, so that whether the scoring algorithm is suitable for the target game is accurately evaluated, and the accuracy and the reliability of evaluation of the scoring algorithm are improved.

Referring to fig. 1, the diagram is a schematic view of a scenario of a scoring algorithm evaluation method provided by an embodiment of the present application, where the application scenario may include a server 101 or a terminal device 102.

The server 101 or the terminal device 102 creates a plurality of virtual objects for operating the target game, and configures initial test scores for the plurality of virtual objects, respectively. As an example, where the target game is a competitive game, 10 virtual objects may be created, the 10 virtual objects being used to simulate players in actual use, the 10 virtual objects may have an initial test score of 1 score.

The server 101 or the terminal device 102 controls a plurality of virtual objects to participate in a plurality of rounds of simulation tests based on the target game; in each round of simulation test, game play is distributed to the virtual object based on the current test score of the virtual object, a scoring algorithm to be evaluated is adopted, and the test score of the virtual object is updated according to the game play result of the virtual object in the game play. As an example, assuming that the target game is an athletic game, 10 virtual objects may be established, the initial test scores of the 10 virtual objects may be 1 score, and when the first round of simulation test is performed, since the initial test scores of the 10 virtual objects are all 1 score, game play may be randomly allocated to the 10 virtual objects, and according to the game result of the game play, the test scores of the 10 virtual objects may be updated by using a scoring algorithm to be evaluated, and the update manner of the test scores may be, for example: the virtual objects X, Y, Z, P, Q are respectively added with 1 score, and the other 5 virtual objects are not added with scores; in the second round of simulation test, since the test scores of the 5 virtual objects X, Y, Z, P, Q are all 2 points, the 5 virtual objects can be allocated to the same game play, and the other 5 virtual objects are allocated to the same game play; and executing the simulation test for a plurality of rounds according to the simulation test mode, and updating the test scores of the 10 virtual objects.

The server 101 or the terminal device 102 determines accuracy data and convergence speed data corresponding to the scoring algorithm based on the game play results of the plurality of virtual objects in the game play in each round of simulation test and the test score ranks of the plurality of virtual objects. Taking the virtual object X as an example, assuming that the result of the game in the 20 rounds of simulation test is 13-7 minus, the accuracy data and the convergence speed data corresponding to the scoring algorithm can be determined according to the winning rate of the virtual object X; and determining accuracy data and convergence speed data corresponding to the scoring algorithm according to the ranking of the test scores of the 10 created virtual objects.

The server 101 or the terminal device 102 determines suitability of the scoring algorithm with respect to the target game according to the accuracy data and the convergence speed data corresponding to the scoring algorithm. It should be understood that by determining the accuracy data and the convergence speed data corresponding to the scoring algorithm, the suitability of the scoring algorithm with respect to the target game can be further determined, that is, the higher the accuracy indicated by the accuracy data, the faster the convergence speed indicated by the convergence speed data, and the higher the suitability of the scoring algorithm with respect to the target game is indicated; otherwise, the lower the suitability of the scoring algorithm to the target game is indicated.

The scoring algorithm evaluation method provided by the embodiment of the application can be applied to a server or terminal equipment with data processing capability, wherein the server can be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and can also be a cloud server for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligent platforms. The terminal device includes, but is not limited to, a mobile phone, a tablet, a computer, an intelligent voice interaction device, an intelligent home appliance, a vehicle-mounted terminal, an aircraft, and the like. The terminal device and the server may be directly or indirectly connected through wired or wireless communication, and the present application is not limited herein.

The embodiment of the application provides a scoring algorithm evaluation method, which relates to an artificial intelligence technology.

Artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use the knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.

The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

The related data collection and processing in the application should be strictly according to the requirements of relevant national laws and regulations when the example is applied, obtain the informed consent or independent consent of the main body of personal information, and develop the subsequent data use and processing behaviors within the authorized range of laws and regulations and the main body of personal information.

Referring to fig. 2, the flowchart of a scoring algorithm evaluation method according to an embodiment of the present application is shown.

Referring to fig. 2, the scoring algorithm evaluation method provided by the embodiment of the present application may include:

s201: a plurality of virtual objects for operating the target game are created, and initial test scores are respectively configured for the plurality of virtual objects.

The target game means a game according to which the scoring algorithm is evaluated, and may be a one-to-one competitive game, a many-to-many competitive game, a multi-person independent competitive game, a multi-person team competitive game, or the like, which is not particularly limited herein.

By virtual object is meant an object created when evaluating a scoring algorithm for participation in a test, the virtual object resembling a real player for use in a simulated test.

The test score means a score for evaluating a game level of the virtual object, and the score of the test score may be 100, 1000, 345, etc., without limitation.

S202: controlling a plurality of virtual objects to participate in a plurality of rounds of simulation tests based on the target game; in each round of simulation test, game play is distributed to the virtual object based on the current test score of the virtual object, a scoring algorithm to be evaluated is adopted, and the test score of the virtual object is updated according to the game play result of the virtual object in the game play.

The simulation test means a test of a plurality of virtual objects for game play based on a game mode of a target game, and as an example, assuming that the target game includes a one-to-one competitive mode, the simulation test may test a plurality of virtual objects for game play based on a one-to-one competitive mode.

The scoring algorithm means an algorithm for scoring a game level of a virtual object according to a game result of the virtual object, and may include, but is not limited to, an ELO algorithm, a Glicko algorithm, a TrueSkill algorithm, and the like.

The game result means a game score or a game result of the virtual object after the game is finished, for example, the game result may be winning or losing; the game result may be the X-th name, etc.

The ELO algorithm (erlo algorithm) is an algorithm for calculating the relative game level of players, in which it is assumed that each player's performance in each round of game is a normally distributed random variable, and although players may play widely different games, the average value of each player's performance over a period of time varies little, and the average value of the random variable may be used to represent the true level of the player.

The Glicko algorithm is an algorithm for determining the reliability of a score, and is an improvement over this problem because the ELO algorithm cannot determine the confidence level of a player's score. Assuming that in a game, players a and B, both scored 1700, after a match, a wins, player a scores will increase by 16 and the corresponding player B score will decrease by 16 under the ELO algorithm. However, if player a is not playing for a long time, but player B is playing weekly, then player a's 1700 score is not reliably rated for player a's strength in the above case, and player B's 1700 score is more trustworthy.

The TrueSkill algorithm is a skill-based ranking algorithm under multiplayer competition. The ELO algorithm is simple and easy to use in calculation, but is mainly used for 1-to-1 type countermeasure results. The ELO algorithm has limitations assuming a form of combat that is a multi-person, multi-team, or more complex form of combat. The TrueSkill algorithm is applicable to complex team formation and is more general; and the method is placed in a more perfect modeling system and is easy to expand.

It should be noted that, in the embodiment of the present application, the scoring algorithm to be evaluated is used to score the game result of the virtual object, and the test score is updated based on the scoring result, and as an example, it is assumed that the current test score of the virtual object is 10 points, and the test score of the virtual object is determined to be improved by 10 points according to the game result of the virtual object by the scoring algorithm, so that the test score of the virtual object is updated, and the updated test score is 20 points.

S203: and determining accuracy data and convergence speed data corresponding to the scoring algorithm based on game results of the plurality of virtual objects in game games in each round of simulation test and the test score ranking of the plurality of virtual objects.

The test score ranking means ranking the plurality of virtual objects in order from high to low based on the test scores corresponding to the plurality of virtual objects, and as an example, assuming that virtual object A, B, C is included and the test scores corresponding to virtual object A, B, C are 50, 70, 20, the corresponding test ranks may be virtual object B, virtual object a, virtual object C.

The accuracy data means data for characterizing the accuracy of the test score of the scoring algorithm for the game level of the virtual object, i.e., the degree of fitting of the test score determined by the scoring algorithm to the real game level of the virtual object; the convergence speed data means data for characterizing a speed at which a test score of the scoring algorithm for the game level of the virtual object approaches the real game level of the virtual object, i.e., a time required for the test score determined by the scoring algorithm to fit the real game level of the virtual object.

It should be understood that, in the embodiment of the present application, as the test score of the virtual object is updated, the test score thereof will gradually approach the real test score; accordingly, the game prediction result determined based on the scoring algorithm is more and more accurate, and the player scoring ranking is more and more close to the real scoring ranking, namely, from two dimensions of the game result of the target game and the test scoring ranking, the accuracy data and the convergence speed data corresponding to the scoring algorithm are determined, and accordingly the suitability of the scoring algorithm relative to the game according to the test is determined.

S204: and determining the suitability of the scoring algorithm relative to the target game according to the accuracy data and the convergence speed data corresponding to the scoring algorithm.

Suitability means that the matching degree of the scoring algorithm with respect to the target game, as an example, assuming that there are a scoring algorithm a and a scoring algorithm B, the convergence speeds of which with respect to the target game are identical, the accuracy of the scoring algorithm a in evaluating the game level of the player in the target game is 95%, and the accuracy of the scoring algorithm B in evaluating the game level of the player in the target game is 50%, the suitability of the scoring algorithm a with respect to the target game is higher, and the scoring algorithm a should be selected for scoring the game level of the player in the target game.

To further illustrate the process of obtaining accuracy data based on the scoring algorithm evaluation method provided in the above embodiment, in some possible implementations, the accuracy data may include a winning rate prediction error, where the winning rate prediction error is determined by:

a1: for each virtual object, determining the real winning rate corresponding to the virtual object according to the game play result of the virtual object in each reference game play in which the virtual object participates; and determining the predicted winning rate corresponding to the virtual object according to the predicted game winning rates of the virtual object in each reference game.

The winning rate prediction error means a difference between a predicted winning rate and a real winning rate of a virtual object.

Predicting the game winning rate means predicting probability that the current game result of the virtual object is winning, and the predicted winning rate corresponding to the virtual object can be further determined by predicting the game winning rate.

The real winning rate of the virtual object can be calculated in different ways based on different combat modes. As an example, assuming that the fight mode is a multi-player independent competition, the game play includes 100 virtual objects, if the rank of the virtual object a is 2, it can be considered that the virtual object a wins the 98 virtual objects ranked behind it, then the single-field winning rate of the virtual object a corresponding to the game play is 98%, and then the average value of the single-field winning rates of the game plays is calculated as the true winning rate of the virtual object a corresponding to each of the game plays in combination with the single-field winning rate of each of the plays before the on-site game play. As another example, assuming that the fight mode is team competition, the win or lose of a team may be regarded as the win or lose of each virtual object within the team, and assuming that the fight result of a certain team in the game fight is the win, then the fight result of the virtual object B in the team may be regarded as the win; and combining the winning or losing results of team competition in which the virtual object B historically participates, and using the ratio of the number of winning game plays to the number of total game plays participated as the winning rate of the virtual object B.

Wherein the predicted game play odds are determined based on a test score of the virtual object prior to participation in the reference game play. As an example, assuming that the combat mode is a multi-person independent competition, the predicted ranking of the virtual object may be determined according to the current test score of the virtual object and the current test score of each virtual object in the reference game pair, and the predicted winning rate corresponding to the virtual object may be determined according to the predicted ranking. As another example, an ELO algorithm may be employed to calculate the predicted game odds based on a test score of a virtual object prior to participation in a reference game and a current test score of an opponent virtual object of the virtual object in the reference game.

And determining an object winning rate prediction error corresponding to the virtual object according to the real winning rate and the predicted winning rate corresponding to the virtual object. As an example, assuming that the predicted winning rate of a virtual object may be 90% and the real winning rate of the virtual object is 80%, the object winning rate prediction error corresponding to the virtual object is considered to be 10%.

A2: and determining the winning rate prediction error corresponding to the scoring algorithm according to the object winning rate prediction error corresponding to each virtual object.

For each virtual object P _i The object winning rate prediction error corresponding thereto can be obtained through the above step A1, and it should be understood that as the scores of the virtual objects get closer to the true scores, the predicted winning rate of each virtual object should be close to the true winning rate. Further, the winning rate error of the scoring algorithm can be determined from the object winning rate prediction error corresponding to each virtual object by the following formula (1).

RMS＝∑ _1≤i≤N (S(P _i )-R(P _i )) ² (1)

Where RMS represents the winning error of the scoring algorithm, S (P _i ) Representing virtual object P _i Is a predicted winning rate of R (P) _i ) Representing the real winning rate of the virtual objects, N is the total number of virtual objects constructed.

It will be appreciated that the smaller the winning prediction error, the higher the suitability of the scoring algorithm with respect to the target game.

The embodiment of the application creatively provides the evaluation index for evaluating the scoring algorithm, namely the winning rate prediction error, wherein the evaluation index is more matched with the related requirements in the game scene, can accurately reflect the accuracy of the scoring algorithm, and is correspondingly beneficial to accurately evaluating the suitability between the scoring algorithm and the target game according to the index.

In order to further illustrate the process of obtaining accuracy data based on the scoring algorithm evaluation method provided in the foregoing embodiment, in another possible implementation manner, the accuracy data may include a ranking prediction accuracy, where the ranking prediction accuracy is determined by:

B1: and determining a predictive ranking sequence according to the current test scores of the virtual objects.

The test score means a score of the scoring algorithm for a game level of the virtual object in the target game; the predictive ranking sequence means that each virtual object is ranked first and virtual object a is ranked second based on the ranking corresponding to the current test score, assuming that the test score of virtual object a is 30 points and the test score of virtual object B is 50 points, as an example.

B2: and determining the ranking prediction accuracy corresponding to the scoring algorithm according to the predicted ranking sequence and the real ranking sequence.

The real ranking sequence means ranking of each virtual object based on a real score, wherein the real score is a real game level of the virtual object in the target game, and the real game level can be preset, namely, the corresponding real score of the virtual object can be set when the virtual object is created, and the real ranking is not particularly limited herein.

As one example, at any time, the predicted ranking sequence of the predicted virtual object ranks is set to L', while the true ranking sequence may be set to L. Let L (k) be the first k virtual objects of list L, P (L '(k), L (k)) be the degree of matching between L' (k) and L (k). The rank prediction accuracy can be evaluated using the following equation (2).

Where P (L '(k), L (k)) represents the degree of matching between the top k virtual objects in the predicted rank sequence L' and the top k virtual objects in the true rank sequence L, RS represents the rank prediction accuracy, and N represents the total number of virtual objects.

It should be appreciated that in equation (2), the ranking prediction accuracy corresponding to the scoring algorithm is determined by determining the degree of matching between the predicted ranking sequence and the true ranking sequence of the virtual object; the matching results of the N ranking sequences are accumulated and divided by the number N of the virtual objects, so that the ranking prediction accuracy can be obtained.

It should be appreciated that the higher the ranking prediction accuracy, the higher the suitability of the scoring algorithm with respect to the target game.

The embodiment of the application creatively provides the evaluation index for evaluating the scoring algorithm, namely the ranking prediction accuracy, wherein the evaluation index is more matched with the related requirements in the game scene, can accurately reflect the accuracy of the scoring algorithm, and is correspondingly beneficial to accurately evaluating the suitability between the scoring algorithm and the target game according to the index.

To further illustrate the process of acquiring convergence speed data, in some possible implementations, the convergence speed data includes a winning rate convergence speed, which is determined by:

C1: and determining a winning rate prediction error corresponding to each round of simulation test. The winning rate prediction error corresponding to the simulation test is determined according to the object winning rate prediction error of each virtual object in the simulation test.

It should be appreciated that in embodiments of the present application, the faster the winning error becomes smaller in each round of simulation testing, the faster the convergence speed is indicated. The specific manner of determining the winning rate prediction error may be found in the above description, and will not be described in detail herein.

C2: in each round of simulation test, determining the simulation test with the minimum corresponding winning rate prediction error as a first target simulation test.

As an example, assuming that 10 rounds of simulation tests are performed, in the 9 th round of simulation test, the 9 th round of simulation test is taken as the first target simulation test, where the winning error of the 9 th round is the smallest value among the plurality of winning errors to which the 10 rounds of simulation tests correspond respectively.

And C3: and determining the winning rate convergence speed corresponding to the scoring algorithm according to the winning rate prediction error corresponding to the first target simulation test and the winning rate prediction error corresponding to each round of simulation test before the first target simulation test.

In one possible implementation, the winning error of the virtual object in the first M-field simulation test may be obtained, and then, when determining the winning error of the minimum value, the winning rate convergence speed of the scoring algorithm may be determined, which may be obtained by the following equation (3):

Wherein,

m ₀ ＝argmin _1≤m≤M RMS(m)

where M is the number of rounds of simulation test, M is the total number of rounds of simulation test performed, CR _winning Representing the winning rate convergence speed, m of the scoring algorithm ₀ Representing the number of turns corresponding to the first target simulation test; RMS (t) represents the winning rate prediction error corresponding to the t-th round of simulation test, t is greater than or equal to 1 and less than or equal to m ₀ ，RMS(m ₀ ) And testing the corresponding winning rate prediction error for the first target simulation.

The above-mentioned manner of calculating the rate of convergence of the winning rate corresponding to the scoring algorithm essentially calculates the Area Under the Curve (AUC) corresponding to the index of the winning rate prediction error, and the smaller the calculated Area Under the Curve, the faster the rate of convergence of the winning rate prediction error of the scoring algorithm, and the larger the calculated Area Under the Curve, the slower the rate of convergence of the winning rate prediction error of the scoring algorithm.

By way of example, the above formula (3) may be represented by an area in a coordinate system, referring to fig. 3, which is a schematic diagram of a convergence speed of a winning rate according to an embodiment of the present application. With reference to fig. 3, assume that 3 rounds of simulation tests are included, the respective corresponding winning rate prediction errors are 70%, 30% and 5%, each round of simulation tests is taken as an abscissa, the winning rate prediction errors are taken as an ordinate, coordinate points corresponding to the three winning rate errors are connected to obtain a curve S, then the 3 rd round of simulation tests are taken as a first target simulation test, the coordinate passing through the first round of simulation tests is parallel to a first straight line M of a vertical axis, the coordinate passing through the first target simulation test is parallel to a second straight line K of a horizontal axis, and an area X surrounded by the curve S is the winning rate convergence speed corresponding to the scoring algorithm.

As shown in fig. 3, the smaller the area of the area X, the faster the winning rate convergence speed of the scoring algorithm is proved, and the higher the suitability of the scoring algorithm with respect to the target game is.

The embodiment of the application creatively provides the evaluation index for evaluating the scoring algorithm, namely the convergence speed of the winning rate, wherein the evaluation index is more matched with the related requirements in the game scene, can accurately reflect the convergence speed of the scoring algorithm, and is correspondingly beneficial to accurately evaluating the suitability between the scoring algorithm and the target game according to the index.

Based on the scoring algorithm evaluation method provided by the foregoing embodiment, in order to further illustrate the process of obtaining the convergence speed data, in other possible implementation manners, the convergence speed data may include a ranking convergence speed, where the ranking convergence speed is determined by:

d1: and determining the ranking prediction accuracy corresponding to each round of simulation test.

The ranking prediction accuracy corresponding to the simulation test is determined according to the predicted ranking sequence and the real ranking sequence of each virtual character in the simulation test. The above-mentioned process for obtaining the accuracy of ranking prediction can be referred to specifically, and will not be described in detail herein.

D2: and in each round of simulation test, determining the simulation test with the highest ranking prediction accuracy as a second target simulation test.

As an example, assuming that 10 rounds of simulation tests are performed, the 4 th round of simulation test corresponds to the highest ranking accuracy, the 4 th round of simulation test is taken as the second target simulation test.

D3: and determining the ranking convergence speed corresponding to the scoring algorithm according to the ranking prediction accuracy corresponding to the second target simulation test and the ranking prediction accuracy corresponding to each round of simulation test before the second target simulation test.

As an example, the first round of simulation tests may be sequentially fetched into the M rounds of simulation tests, all of which are each associated with a rank prediction accuracy. And then, drawing a curve of ranking prediction accuracy RS corresponding to each round of simulation test in a coordinate system. That is, when the RS should reach the maximum value, the test score of the virtual object can be considered to converge. Thus, the ranking convergence speed of the scoring algorithm may be defined as a score or similar numerical score form, as specifically shown in equation (4) below:

wherein,

m ₁ ＝argmax _1≤m≤M RS(m)

wherein M is the number of rounds of the simulation test, M is the total number of rounds of the simulation test performed, CR _rating Representing the rate of rank convergence of the scoring algorithm. m is m ₁ Representing the number of rounds corresponding to the second target simulation test; RS (t) represents ranking prediction accuracy corresponding to the t-th round simulation test, wherein t is greater than or equal to 1 and less than or equal to m ₁ ，RS(m ₁ ) And testing the corresponding ranking prediction accuracy for the second target simulation.

The above method for calculating the ranking convergence speed corresponding to the scoring algorithm essentially calculates the area under the curve AUC corresponding to the index of the ranking prediction accuracy, and the smaller the calculated area under the curve is, the faster the ranking prediction accuracy convergence speed of the scoring algorithm is, the larger the calculated area under the curve is, and the slower the ranking prediction accuracy convergence speed of the scoring algorithm is.

It should be appreciated that the area under the curve is characterized by numerical credits, which better suits the rate of rank convergence of the scoring algorithm with respect to the target game.

The embodiment of the application creatively provides the evaluation index for evaluating the scoring algorithm, namely the ranking convergence speed, wherein the evaluation index is more matched with the related requirements in the game scene, can accurately reflect the convergence speed of the scoring algorithm, and is correspondingly beneficial to accurately evaluating the suitability between the scoring algorithm and the target game according to the index.

Because the scoring algorithm evaluation method provided by the embodiment of the present application may include one-to-one competition, team competition, single person competition or team mixed competition, and different competition modes may include different scoring mechanisms, and may further include different scoring update mechanisms, that is, mechanisms for updating the scoring of the player, in some possible implementation manners, the scoring algorithm evaluation method provided by the embodiment of the present application may further include:

e1: multiple virtual objects are controlled to participate in multiple rounds of simulation testing based on the target game.

In each round of simulation test, game play is distributed to the virtual object based on the current test score of the virtual object, a scoring algorithm and a score updating mechanism to be evaluated are adopted, and the test score of the virtual object is updated according to the game play result of the virtual object in the game play.

As an example, assuming that the current test score of the virtual object W is 10 points, the fight mode is team competition, matching the virtual object W with teammates or opponents with test scores of 9 points, 10 points or 11 points, assuming that the game is winning as a game result, determining that the test score of the virtual object W should be increased by 20 points by adopting a scoring algorithm and a specific score updating mechanism, and adding the 20 points to the 10 points to obtain the updated test score of 30 points of the virtual object.

E2: and determining accuracy data and convergence speed data corresponding to a score updating mechanism based on game results of the plurality of virtual objects in game games in each round of simulation test and the test score ranking of the plurality of virtual objects.

It should be understood that, in the embodiment of the present application, a specific scoring algorithm may be adopted, and the score updating mechanism to be evaluated is utilized to update the test score of the virtual object, where the capability score will gradually approach the real test score; accordingly, the game result determined based on the score updating mechanism will be more accurate, and the player score ranking will be more and more close to the true score ranking, namely, from two dimensions of the game result of the target game and the test score ranking, the accuracy data and the convergence speed data corresponding to the score updating mechanism are determined, and accordingly the suitability of the score updating mechanism relative to the target game according to the test is determined.

It should be noted that, in this step, the implementation manner of determining the accuracy data and the convergence speed data corresponding to the score updating mechanism is similar to the implementation manner of determining the accuracy data and the convergence speed data corresponding to the scoring algorithm described above, and details of the implementation manner may be referred to the related description content above, which is not repeated herein.

E3: and determining the suitability of the score updating mechanism relative to the target game according to the accuracy data and the convergence speed data corresponding to the score updating mechanism.

In the embodiment of the application, accuracy data and convergence speed data corresponding to a score updating mechanism are determined by considering two dimensions of game results of a plurality of virtual objects in game games and test score ranking of the plurality of virtual objects in each round of simulation test, so that suitability of the score updating mechanism relative to a target game is accurately determined through the accuracy data and the convergence speed data.

In one possible implementation, step E1 may include: and aiming at the target virtual object in the game, updating the test score of the target virtual object by adopting a scoring algorithm according to the game result of the target virtual object in the game relative to other virtual objects.

When the game comprises at least three virtual objects, if the game support takes the object as a unit to participate, the game result is determined according to the ranking of the target virtual objects; if the game play support participates in units of team, the play result is determined according to the play result of the team to which the target virtual object belongs.

That is, when the game is played in units of objects, the fight mode may be a multi-person independent competition, the fight result may be determined according to the ranking of the virtual objects in the game, for example, the virtual object ranks first among 100 virtual objects, the fight result is first, the virtual object is compared with other virtual objects, and the test score is updated based on all the comparison results; when the game is played in units of team, that is, the fight mode is team fight, each virtual object in the team may be compared with each virtual object in the opponent team, and the fight result of the target virtual object may be determined.

As one example, assuming that the game play is in team units, assuming that team a wins team B, then all virtual objects in team a are considered to win all virtual objects of team B. Each virtual object updates the test score by comparing the results with each opponent in team B. Taking an ELO algorithm as an example, the predicted winning rate between a target virtual object and each opponent virtual object in an opponent team can be calculated first, namely, for one opponent virtual object, the predicted winning probability of the target virtual object relative to the opponent virtual object can be determined according to the respective current test scores of the target virtual object and the opponent virtual object; then, determining the real opponent virtual object corresponding to each opponent virtual object according to the actual opponent results of the team to which the target virtual object belongs and the opponent team; further, for each opponent virtual object, an updated test score for the opponent virtual object relative to the target virtual object is determined based on the true opponent result and the predicted winning rate of the target virtual object relative to the opponent virtual object. Thus, according to the mode, the updated test scores of the opponent virtual objects relative to the target virtual object are determined, and accordingly the test scores of the target virtual object are comprehensively updated. A similar approach can be used for the Glicko algorithm.

In both of the above algorithms, the following update formula (5) may be employed:

R _new ＝R _old +K∑H(S _match -E _match ) (5)

where K and H are the percentage contribution of each virtual object to the final test score update, R _new Representing updated test scores, R _old Represents an unexplored test score, K ΣH (S _match -E _match ) Representing updated test scores for target virtual objects, S _match E is true of the result of the game _match To predict winning probabilities.

In another possible implementation, when the game play is played in units of team, step E1 may include:

f1: when the game play support participates in the unit of team, determining the team score corresponding to each team in the game play; team scores corresponding to the team are determined according to the current test scores of the virtual objects in the team.

Among the current test scores of the virtual objects in the team, the highest test score, the lowest test score or the average test score may be used as the team score, or the median or the mode of the multiple test scores may be used as the team score, which is not particularly limited herein.

F2: and determining team update scores corresponding to each team according to the team scores corresponding to each team in the game and the game result by adopting a scoring algorithm.

F3: and updating the test scores of the virtual objects based on the team update scores corresponding to each team for each virtual object in each team.

It should be understood that, in this embodiment, the team score is used as the score of each virtual object, and based on the team score and the game result, the team update score corresponding to each team is determined, and based on the team update score, the test score of each virtual object is updated, so as to increase the team cooperation capability of the player when participating in the team competition.

In yet another possible implementation, when the game play is in team units, step E1 may include:

g1: aiming at a target virtual object in game opponents, a scoring algorithm is adopted, and a mixed update score is determined according to the current test score of the target virtual object, the team score corresponding to an opponent team and the opponent result.

The opponent team comprises other virtual objects opposite to the target virtual object in the game opponent, and the team score corresponding to the opponent team is determined according to the current test scores of the other virtual objects opposite to the target virtual object.

And G2: based on the hybrid update scores, the test scores of the virtual objects are updated.

It should be understood that, in the embodiment of the present application, based on the game result, the test score of the target test object in the team is compared with the team score corresponding to the opponent team, and then the hybrid update score is determined, and the test score of the virtual object is updated.

Thus, through the above modes, the scoring algorithm which is only suitable for one-to-one game in the related technology, such as the ELO algorithm and the Glicko algorithm, can be applied to other game business states, and the application scenes of the algorithms are effectively expanded.

Referring to fig. 4, a schematic diagram of a scoring algorithm evaluation framework according to an embodiment of the present application is shown.

Referring to fig. 4, the scoring algorithm evaluation framework provided by the embodiment of the present application may include:

the match mode selection module 401 (match mode) is configured to select a plurality of match modes, such as one-to-one (two players fight each other to generate three results of win, lose and tie), many-to-many (two teams fight each other to generate two results of lose or win), many-person independent competition (multiple players compete with each other, the last one remaining after the successive elimination is the winner), and multiple team competition (multiple teams fight, and the last team winning is the winner).

Scoring module 402 (ranking) is configured to provide a corresponding scoring algorithm, such as ELO algorithm, glicko algorithm, trueSkill algorithm, and the like, for the combat mode selected by combat mode selection module 51. Among them, the conventional ELO algorithm and Glicko algorithm support only one-to-one combat mode, while TrueSkill algorithm can support combat of a plurality of players by iteratively updating ratings through factor graphs.

Matching module 403 (matching generation) for grouping or matching appropriate games based on the game level of the virtual object, matching module 503 may be composed of a skill generator and a result generator. The skill generator is modeled by a built-in probability distribution function containing parameters, and the result generator calculates the game result of the current game through the game level of the virtual object and divides the game result into a deterministic game result generation method and a non-deterministic game result generation method. In one possible implementation, a non-deterministic outcome generator may be selected to fit noise in the game and player performance fluctuations.

An algorithm Evaluation module 404 (Evaluation) is configured to evaluate suitability of the scoring algorithm with respect to the target game based on accuracy data and convergence speed data corresponding to the scoring algorithm. The accuracy data may include a winning rate error and a ranking prediction accuracy, and the convergence speed data may include a winning rate convergence speed and a ranking convergence speed.

Referring to fig. 5a, the diagram is a schematic diagram of an ELO algorithm evaluation index provided in an embodiment of the present application. Referring to fig. 5b, the graph is a schematic diagram of a Glicko algorithm evaluation index provided in an embodiment of the present application. Referring to fig. 5c, the diagram is a schematic diagram of a TrueSkill algorithm evaluation index provided by the embodiment of the present application.

As shown in connection with fig. 5a, 5b and 5c, it is assumed that a target game P and a target game Q are included.

A in fig. 5a represents a winning rate prediction error, a ranking prediction accuracy, a winning rate convergence speed, and a ranking convergence speed of the ELO algorithm with respect to the target game P; c in fig. 5b represents the winning rate prediction error, the ranking prediction accuracy, the winning rate convergence speed, and the ranking convergence speed of the Glicko algorithm with respect to the target game P; e in fig. 5c represents the winning rate prediction error, the ranking prediction accuracy, the winning rate convergence speed, and the ranking convergence speed of the TrueSkill algorithm with respect to the target game P.

FIG. 5a is a diagram showing the winning rate prediction error, the ranking prediction accuracy, the winning rate convergence speed, and the ranking convergence speed of the ELO algorithm relative to the target game Q; d in fig. 5b represents the winning rate prediction error, the ranking prediction accuracy, the winning rate convergence speed, and the ranking convergence speed of the Glicko algorithm with respect to the target game Q; f in fig. 5c represents the winning rate prediction error, the ranking prediction accuracy, the winning rate convergence speed, and the ranking convergence speed of the TrueSkill algorithm with respect to the target game Q.

It should be understood that taking a in fig. 5a and C in fig. 5b as examples, only the ranking prediction accuracy and the winning prediction error are considered for the target game P, and the ELO algorithm and the Glicko algorithm are respectively adopted to score, the ranking prediction accuracy of a in fig. 5a is 0.678, and the winning prediction error is 0.03; the rank prediction accuracy for C in fig. 5b is 0.85 and the winning prediction error is 0.007; the Glicko algorithm may be considered to be better adapted with respect to the target game P.

Referring to fig. 6, the structure of a scoring algorithm evaluation device according to an embodiment of the present application is shown.

Based on the scoring algorithm evaluation method provided in the foregoing embodiment, the embodiment of the present application further provides a scoring algorithm evaluation device, and referring to fig. 6, the device 600 includes:

an object creation module 601, configured to create a plurality of virtual objects for operating a target game, and configure initial test scores for the plurality of virtual objects, respectively;

a simulation test module 602, configured to control a plurality of virtual objects to participate in a plurality of rounds of simulation tests based on the target game; in each round of simulation test, based on the current test score of the virtual object, distributing game play for the virtual object, and updating the test score of the virtual object according to the game play result of the virtual object in the game play by adopting a scoring algorithm to be evaluated;

The evaluation index determining module 603 is configured to determine accuracy data and convergence speed data corresponding to the scoring algorithm based on game results of the plurality of virtual objects in the game in each round of simulation test and the test score ranks of the plurality of virtual objects;

and the evaluation module 604 is used for determining the suitability of the scoring algorithm relative to the target game according to the accuracy data and the convergence speed data corresponding to the scoring algorithm.

As one example, the accuracy data includes a winning rate prediction error that is determined by:

the real winning rate determining module is used for determining the real winning rate corresponding to each virtual object according to the game play result of the virtual object in each reference game play in which the virtual object participates; and determining a predicted winning rate corresponding to the virtual object according to the predicted winning rates of the virtual object in each reference game, wherein the predicted winning rates are determined according to the test scores of the virtual object before participating in the reference game; determining an object winning rate prediction error corresponding to the virtual object according to the real winning rate and the predicted winning rate corresponding to the virtual object;

And the object winning rate prediction error determination module is used for determining the winning rate prediction error corresponding to the scoring algorithm according to the object winning rate prediction error corresponding to each virtual object.

As one example, the convergence speed data includes a winning rate convergence speed, which is determined by:

the winning rate prediction error determining module is used for determining a winning rate prediction error corresponding to each round of simulation test; the winning rate prediction error corresponding to the simulation test is determined according to the object winning rate prediction error of each virtual object in the simulation test;

the first target simulation test determining module is used for determining a simulation test with the minimum corresponding winning rate prediction error as a first target simulation test in each round of simulation tests;

and the winning rate convergence speed determining module is used for determining the winning rate convergence speed corresponding to the scoring algorithm according to the winning rate prediction error corresponding to the first target simulation test and the winning rate prediction error corresponding to each round of simulation test before the first target simulation test.

As one example, the accuracy data includes a ranking prediction accuracy, which is determined by:

The prediction ranking sequence determining module is used for determining a prediction ranking sequence according to the current test scores of the virtual objects;

the ranking prediction accuracy determining module is used for determining ranking prediction accuracy corresponding to the scoring algorithm according to the predicted ranking sequence and the real ranking sequence; the real ranking sequence is determined according to the real scores corresponding to the virtual objects, and the real scores are used for representing the real game level of the virtual objects in the target game.

As one example, the convergence speed data includes a ranking convergence speed, which is determined by:

the ranking prediction accuracy determining module is used for determining ranking prediction accuracy corresponding to each round of simulation test; the ranking prediction accuracy corresponding to the simulation test is determined according to the ranking sequence and the real ranking sequence of each virtual character in the simulation test;

the second target simulation test determining module is used for determining the simulation test with the largest ranking prediction accuracy corresponding to each round of simulation test as a second target simulation test;

and the ranking convergence speed determining module is used for determining the ranking convergence speed corresponding to the scoring algorithm according to the ranking prediction accuracy corresponding to the second target simulation test and the ranking prediction accuracy corresponding to each round of simulation test before the second target simulation test.

As an example, the apparatus 600 further comprises:

the updating module is used for controlling the plurality of virtual objects to participate in multiple rounds of simulation tests based on the target game; in each round of simulation test, based on the current test score of the virtual object, distributing game play for the virtual object, and adopting a scoring algorithm and a score updating mechanism to be evaluated, and updating the test score of the virtual object according to the game play result of the virtual object in the game play;

the determining module is used for determining accuracy data and convergence speed data corresponding to a score updating mechanism based on game results of a plurality of virtual objects in game games in each round of simulation test and test score ranking of the plurality of virtual objects;

and the scoring module is used for determining the suitability of the scoring updating mechanism relative to the target game according to the accuracy data and the convergence speed data corresponding to the scoring updating mechanism.

As one example, an update module includes:

the first updating unit is used for updating the test scores of the target virtual objects by adopting a scoring algorithm according to the game play results of the target virtual objects in the game play relative to other virtual objects aiming at the target virtual objects in the game play;

As one example, an update module includes:

the first determining unit is used for determining team scores corresponding to each team in the game play when the game play support takes the team as a unit to participate; the team score corresponding to the team is determined according to the current test scores of each virtual object in the team;

the second determining unit is used for determining team update scores corresponding to each team respectively according to the team scores corresponding to each team in the game and the game result by adopting a scoring algorithm;

and the second updating unit is used for updating the test scores of the virtual objects based on the team update scores corresponding to each team aiming at each virtual object in each team.

As one example, an update module includes:

the third determining unit is used for determining a mixed update score according to the current test score of the target virtual object, the team score corresponding to the opponent team and the game result by adopting a scoring algorithm aiming at the target virtual object in game; the opponent team comprises other virtual objects opposite to the target virtual object in the game opponent, and the team score corresponding to the opponent team is determined according to the current test scores of the other virtual objects opposite to the target virtual object;

And a third updating unit for updating the test score of the virtual object based on the mixed update score.

The scoring algorithm evaluation device provided by the embodiment of the application has the same beneficial effects as the scoring algorithm evaluation method provided by the embodiment, so that the description is omitted.

The embodiment of the application also provides a computer device, which can be a terminal device or a server, and the terminal device and the server provided by the embodiment of the application are introduced from the aspect of hardware materialization.

Referring to fig. 7, fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application. As shown in fig. 7, for convenience of explanation, only the portions related to the embodiments of the present application are shown, and specific technical details are not disclosed, please refer to the method portions of the embodiments of the present application. The terminal may be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (pda), a Point of Sales (POS), a vehicle-mounted computer, and the like, taking the terminal as an example of a computer:

fig. 7 is a block diagram showing a part of the structure of a computer related to a terminal provided by an embodiment of the present application. Referring to fig. 7, a computer includes: radio Frequency (RF) circuitry 1210, memory 1220, input unit 1230 (including touch panel 1231 and other input devices 1232), display unit 1240 (including display panel 1241), sensors 1250, audio circuitry 1260 (which may connect speaker 1261 and microphone 1262), wireless fidelity (wireless fidelity, wiFi) module 1270, processor 1280, and power supply 1290. Those skilled in the art will appreciate that the computer architecture shown in fig. 7 is not limiting and that more or fewer components than shown may be included, or that certain components may be combined, or that different arrangements of components may be provided.

Memory 1220 may be used to store software programs and modules, and processor 1280 may execute the various functional applications and data processing of the computer by executing the software programs and modules stored in memory 1220. The memory 1220 may mainly include a storage program area that may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), and a storage data area; the storage data area may store data created according to the use of the computer (such as audio data, phonebooks, etc.), and the like. In addition, memory 1220 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.

Processor 1280 is a control center of the computer and connects various parts of the entire computer using various interfaces and lines, performing various functions of the computer and processing data by running or executing software programs and/or modules stored in memory 1220, and invoking data stored in memory 1220. In the alternative, processor 1280 may include one or more processing units; preferably, the processor 1280 may integrate an application processor and a modem processor, wherein the application processor primarily handles operating systems, user interfaces, application programs, etc., and the modem processor primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 1280.

In an embodiment of the present application, the processor 1280 included in the terminal further has the following functions:

Optionally, the processor 1280 is further configured to execute steps of any implementation manner of the scoring algorithm evaluation method provided by the embodiment of the present application.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a server 1300 according to an embodiment of the present application. The server 1300 may vary considerably in configuration or performance and may include one or more central processing units (central processing units, CPU) 1322 (e.g., one or more processors) and memory 1332, one or more storage media 1330 (e.g., one or more mass storage devices) storing applications 1342 or data 1344. Wherein the memory 1332 and storage medium 1330 may be transitory or persistent. The program stored on the storage medium 1330 may include one or more modules (not shown), each of which may include a series of instruction operations on a server. Further, the central processor 1322 may be configured to communicate with the storage medium 1330, and execute a series of instruction operations in the storage medium 1330 on the server 1300.

The Server 1300 may also include one or more power supplies 1326, one or more wired or wireless network interfaces 1350, one or more input/output interfaces 1358, and/or one or more operating systems, such as Windows Server ^TM ，Mac OS X ^TM ，Unix ^TM ,Linux ^TM ，FreeBSD ^TM Etc.

The steps performed by the server in the above embodiments may be based on the server structure shown in fig. 8.

Wherein CPU 1322 is configured to perform the following steps:

Optionally, CPU 1322 may also be configured to perform the steps of any one implementation of the scoring algorithm evaluation method provided by embodiments of the present application.

The embodiments of the present application also provide a computer readable storage medium storing a computer program for executing any one of the foregoing implementations of a scoring algorithm evaluation method described in the foregoing embodiments.

Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform any one of the methods of scoring algorithm evaluation described in the foregoing respective embodiments.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media in which a computer program can be stored.

It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "and/or" for describing the association relationship of the association object, the representation may have three relationships, for example, "a and/or B" may represent: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.

The above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A scoring algorithm evaluation method, the method comprising:

creating a plurality of virtual objects for operating a target game, and respectively configuring initial test scores for the plurality of virtual objects;

controlling the virtual objects to participate in a plurality of rounds of simulation tests based on the target game; in each round of simulation test, distributing game play for the virtual object based on the current test score of the virtual object, and updating the test score of the virtual object according to the game play result of the virtual object in the game play by adopting a scoring algorithm to be evaluated;

determining accuracy data and convergence speed data corresponding to the scoring algorithm based on game play results of the plurality of virtual objects in game play in each round of the simulation test and the test score ranking of the plurality of virtual objects;

2. The method of claim 1, wherein the accuracy data comprises a winning rate prediction error determined by:

For each virtual object, determining the real winning rate corresponding to the virtual object according to the game play result of the virtual object in each reference game play in which the virtual object participates; and determining a predicted winning rate corresponding to the virtual object according to the predicted winning rates of the virtual object in each reference game, wherein the predicted winning rates are determined according to the test scores of the virtual object before participating in the reference game; determining an object winning rate prediction error corresponding to the virtual object according to the real winning rate and the predicted winning rate corresponding to the virtual object;

and determining the winning rate prediction error corresponding to the scoring algorithm according to the object winning rate prediction error corresponding to each virtual object.

3. The method according to claim 1 or 2, wherein the convergence speed data comprises a winning rate convergence speed, the winning rate convergence speed being determined by:

determining a winning rate prediction error corresponding to each round of the simulation test; the winning rate prediction error corresponding to the simulation test is determined according to the object winning rate prediction error of each virtual object in the simulation test;

Determining a simulation test with the minimum winning rate prediction error corresponding to each round of simulation test as a first target simulation test;

and determining the winning rate convergence speed corresponding to the scoring algorithm according to the winning rate prediction error corresponding to the first target simulation test and the winning rate prediction error corresponding to each round of simulation test before the first target simulation test.

4. The method of claim 1, wherein the accuracy data comprises a ranking prediction accuracy determined by:

determining a predictive ranking sequence according to the current test scores of the virtual objects;

determining ranking prediction accuracy corresponding to the scoring algorithm according to the predicted ranking sequence and the real ranking sequence; the real ranking sequence is determined according to real scores corresponding to the virtual objects, and the real scores are used for representing real game levels of the virtual objects in the target game.

5. The method of claim 1 or 4, wherein the convergence speed data comprises a ranked convergence speed determined by:

Determining the ranking prediction accuracy corresponding to each round of simulation test; the ranking prediction accuracy corresponding to the simulation test is determined according to the ranking sequence and the real ranking sequence of each virtual character in the simulation test;

in each round of simulation test, determining the simulation test with the highest ranking prediction accuracy as a second target simulation test;

and determining the ranking convergence speed corresponding to the scoring algorithm according to the ranking prediction accuracy corresponding to the second target simulation test and the ranking prediction accuracy corresponding to each round of simulation test before the second target simulation test.

6. The method according to claim 1, wherein the method further comprises:

controlling the virtual objects to participate in a plurality of rounds of simulation tests based on the target game; in each round of simulation test, distributing game play for the virtual object based on the current test score of the virtual object, and updating the test score of the virtual object according to the game play result of the virtual object in the game play by adopting a scoring algorithm and a score updating mechanism to be evaluated;

Determining accuracy data and convergence speed data corresponding to the score updating mechanism based on game results of the plurality of virtual objects in game games in each round of the simulation test and the test score ranking of the plurality of virtual objects;

and determining the suitability of the score updating mechanism relative to the target game according to the accuracy data and the convergence speed data corresponding to the score updating mechanism.

7. The method of claim 6, wherein updating the test score of the virtual object based on the game play result of the virtual object in the game play using a scoring algorithm and a score update mechanism to be evaluated, comprises:

updating the test scores of the target virtual objects in the game play by adopting the scoring algorithm according to the game play results of the target virtual objects in the game play relative to other virtual objects;

when the game comprises at least three virtual objects, if the game is supported to participate in units of objects, the game result is determined according to the ranking of the target virtual objects; and if the game play support takes the team as a unit to participate, determining the play result according to the play result of the team to which the target virtual object belongs.

8. The method of claim 6, wherein the updating the test score of the virtual object based on the game outcome of the virtual object in the game using the scoring algorithm and the score update mechanism to be evaluated comprises:

when the game is supported to participate in units of team, determining team scores corresponding to each team in the game; the team score corresponding to the team is determined according to the current test scores of the virtual objects in the team;

adopting the scoring algorithm, and determining team update scores corresponding to each team according to the team scores corresponding to each team in the game and the game result;

and updating the test scores of the virtual objects based on the team update scores corresponding to each team for each virtual object in each team.

9. The method of claim 6, wherein the updating the test score of the virtual object based on the game outcome of the virtual object in the game using the scoring algorithm and the score update mechanism to be evaluated comprises:

Aiming at a target virtual object in the game, determining a mixed update score according to the current test score of the target virtual object, a team score corresponding to an opponent team and the game result by adopting the scoring algorithm; the opponent team comprises other virtual objects opposite to the target virtual object in the game opponent, and the team score corresponding to the opponent team is determined according to the current test scores of the other virtual objects opposite to the target virtual object;

and updating the test score of the virtual object based on the mixed update score.

10. A scoring algorithm evaluation device, the device comprising:

the simulation test module is used for controlling the virtual objects to participate in multiple rounds of simulation tests based on the target game; in each round of simulation test, distributing game play for the virtual object based on the current test score of the virtual object, and updating the test score of the virtual object according to the game play result of the virtual object in the game play by adopting a scoring algorithm to be evaluated;

The evaluation index determining module is used for determining accuracy data and convergence speed data corresponding to the scoring algorithm based on game results of the plurality of virtual objects in the game in each round of the simulation test and the test score ranking of the plurality of virtual objects;

11. A computer device, the computer device comprising a processor and a memory;

the memory is used for storing a computer program;

the processor is configured to execute the scoring algorithm evaluation method according to any one of claims 1 to 9 according to the computer program.

12. A computer-readable storage medium storing a computer program for executing the scoring algorithm evaluation method according to any one of claims 1 to 9.