CN116459520A

CN116459520A - Intelligent virtual role control method, device, equipment and storage medium

Info

Publication number: CN116459520A
Application number: CN202210029239.6A
Authority: CN
Inventors: 刘望桐; 胡杰
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-01-11
Filing date: 2022-01-11
Publication date: 2023-07-21

Abstract

Embodiments of the present disclosure provide an intelligent virtual character control method, apparatus, device, and computer-readable storage medium. The method provided by the embodiment of the disclosure carries out various subjective judgments based on real-time environment data perceived by the intelligent virtual character, so that a behavior strategy and specific behaviors to be controlled to be executed by the intelligent virtual character are further determined based on probability according to the judging results, and finally determined intelligent virtual character behaviors have randomness instead of the determined behaviors based on the judging results, so that the behaviors of the intelligent virtual character are more consistent with human behaviors, and have higher flexibility and game level, thereby improving the game experience of players. By the method, the structural design and control of the intelligent virtual roles are realized, the intelligent virtual roles designed and realized by the structure have clear logic and clear layers, the division and development of actual work are facilitated, the expandability is high, and the later debugging and maintenance are facilitated.

Description

Intelligent virtual role control method, device, equipment and storage medium

Technical Field

The present disclosure relates to the field of artificial intelligence and electronic games, and more particularly, to an intelligent virtual character control method, apparatus, device, and storage medium.

Background

In the field of electronic games, a great deal of research work is put into the related art of game development in order to bring players with a real virtual game environment, a strong sense of substitution, and a rich entertainment. Since the birth of electronic games, artificial intelligence in games has been one of the focus of research and has received increasing attention. For First Person Shooter (FPS) games, artificial intelligence is more important. While visual rendering and shooting-related physical simulation calculations provide a considerable sense of realism, FPS games focus on player interactions-inter-interactions or inter-attacks with virtual characters in the game, as their primary purpose is to provide real combat to players, and what is essential in combat is the offensiveness between characters. In order to provide a more realistic and interesting gaming experience, virtual characters in a game need to be able to respond more in line with human behavior based on the player's behavior and the virtual environment in the game. Therefore, how to build a virtual character whose behavior is highly realistic and anthropomorphic is one of the key problems in the development of FPS games.

However, since there is no complete and systematic virtual character design structure or methodology, the relevant functional layers of the virtual character design are unclear, and the common virtual character design method generally divides the virtual character design into three parts, namely, behavior, analysis and perception, which lack clear definition, there is a lot of fuzzy content between the parts, and since each virtual character function is not clearly distinguished to which part, nor unified thought and index, the responsibility division of the function is not clear, which is unfavorable for the division and progress of actual development work. In addition, in the virtual character design methods, unique determined behaviors are selected under the determined conditions, so that the virtual character behaviors are stiff and lack of change, further, the game experience of a player is single, and the complexity of an algorithm is exponentially increased along with the increase of the number of the behaviors of the virtual character and the increase of the conditions, so that the debugging and the maintenance are difficult.

Therefore, there is a need for an efficient virtual character design method that enables the design of virtual characters that are highly behaving anthropomorphic and have a high game level.

Disclosure of Invention

In order to solve the problems, the intelligent virtual character is controlled layer by layer, the behavior to be controlled to be executed by the intelligent virtual character is determined based on probability according to the environment detection data of the intelligent virtual character, so that a multi-layer virtual character design method with definite layering and division is established, and the designed intelligent virtual character behavior is highly anthropomorphic and flexible.

Embodiments of the present disclosure provide an intelligent virtual character control method, apparatus, device, and computer-readable storage medium.

The embodiment of the disclosure provides an intelligent virtual role control method, which comprises the following steps: acquiring a first number of environment detection data in the vicinity of the intelligent virtual character; generating a second number of environmental judgment results based on the first number of environmental detection data, the second number being not greater than the first number; determining a first behavior policy from a plurality of behavior policies of the intelligent virtual character based on the second number of environmental decisions, the determined first behavior policy indicating that the intelligent virtual character has a higher probability of selecting to execute a behavior corresponding to the first behavior policy than other behavior policies of the plurality of behavior policies of the intelligent virtual character; and determining a behavior to be controlled to be executed by the intelligent virtual character based on the second number of environmental judgment results and the determined first behavior policy, and execution conditions of at least a part of all behaviors of the intelligent virtual character.

The embodiment of the disclosure provides an intelligent virtual character control device, which comprises: an environment detection module configured to obtain a first amount of environment detection data in proximity to the intelligent virtual character; an environment determination module configured to generate a second number of environment determination results based on the first number of environment detection data, the second number being not greater than the first number; a policy decision module configured to determine a first behavior policy from a plurality of behavior policies of the intelligent virtual character based on the second number of environmental decisions, the determined first behavior policy indicating that the intelligent virtual character has a higher probability of selecting to perform a behavior corresponding to the first behavior policy than other behavior policies of the plurality of behavior policies of the intelligent virtual character; and a behavior decision module configured to determine a behavior to control execution of the intelligent virtual character based on the second number of environmental judgment results and the determined first behavior policy, and an execution condition of at least a part of all behaviors of the intelligent virtual character.

The embodiment of the disclosure provides an intelligent virtual character control device, comprising: one or more processors; and one or more memories, wherein the one or more memories have a computer-executable program stored therein, which when executed by the processor, performs the intelligent virtual character control method as described above.

Embodiments of the present disclosure provide a computer readable storage medium having stored thereon computer executable instructions which, when executed by a processor, are for implementing the intelligent virtual character control method as described above.

Embodiments of the present disclosure provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from a computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs an intelligent virtual character control method according to an embodiment of the present disclosure.

Compared with the existing virtual character design method, the method provided by the embodiment of the disclosure establishes an intelligent virtual character design framework comprising five layers, standardizes definition and design points of each layer, and designs the intelligent virtual character with high anthropomorphic and high game level based on the design framework.

The method provided by the embodiment of the disclosure carries out various subjective judgments based on real-time environment data perceived by the intelligent virtual character, so that a behavior strategy and specific behaviors to be controlled to be executed by the intelligent virtual character are further determined based on probability according to the judging results, and finally determined intelligent virtual character behaviors have randomness instead of the determined behaviors based on the judging results, so that the behaviors of the intelligent virtual character are more consistent with human behaviors, and have higher flexibility and game level, thereby improving the game experience of players. By the method, the structural design and control of the intelligent virtual roles are realized, the intelligent virtual roles designed and realized by the structure have clear logic and clear layers, the division and development of actual work are facilitated, the expandability is high, and the later debugging and maintenance are facilitated.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings that are used in the description of the embodiments will be briefly described below. It should be apparent that the drawings in the following description are only some exemplary embodiments of the present disclosure, and that other drawings may be obtained from these drawings by those of ordinary skill in the art without undue effort.

FIG. 1A is a schematic diagram illustrating a scenario in which a game AI interacts with a game environment, according to an embodiment of the disclosure;

FIG. 1B is a schematic diagram showing a common game AI design method;

FIG. 1C is a schematic diagram showing a behavior decision method of a typical game AI;

FIG. 2A is a flowchart illustrating a method of intelligent virtual character control according to an embodiment of the present disclosure;

fig. 2B is a schematic diagram illustrating a hierarchical structure of an intelligent virtual character control method according to an embodiment of the present disclosure;

FIG. 3A is a schematic flow chart diagram illustrating a process at an observation layer in an intelligent virtual role control method in accordance with an embodiment of the present disclosure;

FIG. 3B is a schematic diagram illustrating example processing in an observation layer according to an embodiment of the disclosure;

fig. 4A is a schematic flowchart showing a process at a judgment layer in the intelligent virtual character control method according to an embodiment of the present disclosure;

FIG. 4B is a schematic diagram illustrating an example process in a fault determination layer according to an embodiment of the present disclosure;

FIG. 5A is a schematic flow chart diagram illustrating a process at a policy decision layer in an intelligent virtual role control method in accordance with an embodiment of the present disclosure;

FIG. 5B is a schematic diagram illustrating example policies and behaviors in a policy decision layer according to an embodiment of the disclosure;

FIG. 5C is a schematic diagram illustrating example policy decisions in a policy decision layer according to an embodiment of the disclosure;

FIG. 6A is a schematic flow chart diagram illustrating a process at a behavior decision layer in an intelligent virtual role control method in accordance with an embodiment of the present disclosure;

FIG. 6B is a schematic diagram illustrating example execution conditions in a behavior decision layer according to an embodiment of the present disclosure;

fig. 7A is a schematic flowchart showing a process at an execution layer in the intelligent virtual character control method according to an embodiment of the present disclosure;

FIG. 7B is a schematic diagram illustrating an example execution action flow in an execution layer according to an embodiment of the disclosure;

fig. 8 is a schematic diagram illustrating a hierarchical structure of an intelligent virtual character control method and related processes according to an embodiment of the present disclosure;

fig. 9A is a diagram showing example parameter configurations of a decision layer, a policy decision layer, and a behavior decision layer of an intelligent virtual character control method according to an embodiment of the present disclosure;

FIG. 9B is a schematic flow chart diagram illustrating performing a lateral attack according to an embodiment of the present disclosure;

FIG. 9C is a schematic diagram illustrating selection probabilities for different behaviors in the same environment according to an embodiment of the present disclosure;

fig. 10A is a schematic diagram illustrating an example division of effort of decision layers, decision layers (policy decision layer and behavior decision layer), and execution layers of an intelligent virtual role control method according to an embodiment of the present disclosure;

FIG. 10B is a schematic flow chart diagram illustrating adding new behavior before and after implementation of the intelligent virtual role control method in accordance with embodiments of the present disclosure;

FIG. 10C is a schematic diagram illustrating the division of work of layers of an intelligent virtual role control method in accordance with embodiments of the present disclosure;

fig. 11 is a schematic diagram illustrating an intelligent virtual character control apparatus according to an embodiment of the present disclosure;

FIG. 12 illustrates a schematic diagram of an intelligent virtual character control apparatus according to an embodiment of the present disclosure;

FIG. 13 illustrates a schematic diagram of an architecture of an exemplary computing device, according to an embodiment of the present disclosure; and

fig. 14 shows a schematic diagram of a storage medium according to an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the present disclosure more apparent, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present disclosure and not all of the embodiments of the present disclosure, and that the present disclosure is not limited by the example embodiments described herein.

In the present specification and drawings, steps and elements having substantially the same or similar are denoted by the same or similar reference numerals, and repeated descriptions of the steps and elements will be omitted. Meanwhile, in the description of the present disclosure, the terms "first," "second," and the like are used merely to distinguish the descriptions, and are not to be construed as indicating or implying relative importance or order.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to be limiting of the invention.

For purposes of describing the present disclosure, the following presents concepts related to the present disclosure.

The intelligent virtual character control method of the present disclosure may be based on artificial intelligence (Artificial Intelligence, AI). Artificial intelligence is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and expand human intelligence, sense the environment, acquire knowledge and use the knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. For example, with respect to an artificial intelligence-based intelligent virtual character control method, it is possible to control an intelligent virtual character established in a digital computer to acquire various real-time data in a game environment in a manner similar to a human perception environment and to make corresponding behaviors, and to determine behaviors to be made by the intelligent virtual character based on the data, thereby making behaviors of non-player characters appear intelligent. By researching the design principles and implementation methods of various intelligent machines, the intelligent virtual character control method has the function of designing intelligent virtual characters with high anthropomorphic behaviors and high game level.

For example, the intelligent virtual character in the intelligent virtual character control method of the present disclosure may refer to game artificial intelligence (game AI). The essence of AI is to process the input information and then output it in a suitable manner similar to human intelligence, which involves three parts, namely receiving the input information, analyzing the information and outputting it in a suitable manner, which is essentially the processing and presentation of the information, and game AI is no exception. Game AI is a series of techniques for rendering the behaviour of a non-player character intelligent in computers and games, which can also be understood as a thinkable non-player character. It is clear to those skilled in the art that there are links and differences between the academic field and the AI in the game field. The two kinds of AI are the same in that they are both human behavior simulated as much as possible, and the AI behavior is made closer to the human behavior by various methods, so that they have in common the study of the human behavior. However, the academic field differs from the game field in terms of AI's criteria, mainly in that the academic field places more emphasis on the internal algorithms and mechanisms of AI, while the game field places more emphasis on the performance of AI. The main reason for this difference is the difference in AI requirements between the game and academic fields, and the player can experience only the external appearance of the AI in the game, so that the game AI is more focused on enhancing the external appearance of the AI, and less focused on the internal algorithm of the AI. In addition, there is another reason that the game field generally requires the least cost and resources to represent the AI in the simplest implementation while the AI needs to simulate the player's behavior as much as possible, so that the player feels the game AI is a human being really present instead of a machine of cool ice, thereby increasing the interaction with the AI from man-machine interaction to man-machine interaction. Meanwhile, the substitution sense of the player can be enhanced, and the interest of the player is attracted. While the AI pursuit algorithm in academic fields strives for perfect optimal solution, the AI pursuit in gaming fields strives for diversification, striving for more realistic game world, which does not require every game AI to possess the highest intelligence, as does the cognitive gap in human society, in-game AI.

In view of the foregoing, embodiments of the present disclosure relate to techniques of artificial intelligence, intelligent virtual character design, and the like, and embodiments of the present disclosure will be further described below with reference to the accompanying drawings.

Fig. 1A is a schematic view illustrating a scenario in which a game AI interacts with a game environment according to an embodiment of the present disclosure. Fig. 1B is a schematic diagram showing a common game AI design method. Fig. 1C is a schematic diagram showing a behavior decision method of a typical game AI.

Most electronic games increase their sense of substitution by giving players virtual worlds and interactions with virtual characters, etc., whereas for virtual characters that are not player-controlled, it is difficult or even impossible for players to interact with them without a degree of human (or animal, etc.) intelligence, resulting in inability to be placed in the virtual worlds. Thus, since the birth of electronic games, artificial intelligence in games has been one of the focus of research and has received increasing attention. As shown in fig. 1A, the game AI effects the retrieval of data and the response of behavior through interaction with a gaming environment, which may include other players, non-player controlled virtual characters, a background environment, and the like. In games such as FPS, making the game environment sufficiently realistic (e.g., the task atmosphere is sufficiently realistic) can increase the sense of substitution of players, which can be achieved by improving interactions and associations between players, and also by improving interactions between players and game AI. The world view of the entire virtual game world requires player acceptance, and the presentation of the game world view also requires presentation by the game AI, regardless of whether the task, dialogue, instant-action animation, or scene scenario deduction, is kept away from the game AI.

However, since there is no more complete and systematic game AI design structure or methodology at present, the relevant function layers of the game AI design are not clear, and the common game AI design method is as shown in fig. 1B, the game AI design is roughly divided into three parts of behavior, analysis and perception, each part is not clearly defined, and there is a lot of fuzzy content in between, and since it is not clearly distinguished which part each virtual character function belongs to, nor is there any unified thought and index, the responsibility division of the functions is not clear, which is unfavorable for the division and progress of actual development work.

Furthermore, since finer AI computation requires more computer computing resources, games typically tend to simplify AI design in order to meet real-time requirements, e.g., implemented using a fixed behavioral response for a fixed game scenario. For example, in order for a game AI to realize a series of actions of stealing and killing a player from behind the player, it is necessary to pass through a predetermined number of shelters fixedly, and to pass through a fixed route from a fixed direction to the back of the player, and in such a case, the player will be familiar with the characteristics of the game AI for a certain period of time, and the behavior of the AI will not be reasonable anymore for a long time, resulting in a reduced sense of substitution of the game.

The main way to make the action decision of the typical game AI is the action tree (BehaviorTree), as shown in fig. 1C, in which the trigger conditions A1, A2, A3, … …, an of the action a, the trigger conditions B1, B2, B3, … …, bn of the action B are listed, and so on. The behavior decision process judges whether the condition of each behavior is satisfied in turn completely according to the order of the behavior tree, and executes the first behavior satisfying all the conditions corresponding to the first behavior. Therefore, in such behavior decisions, various behaviors of the game AI in the game world are designed in advance, and the behaviors of the game AI are operated according to certain rules, which are far away from true intelligent decisions, evaluations and learning. That is, the current game AI is more "artificial setting" than "intelligent", and these AI will choose a unique determined behavior under a certain condition, so that the AI behavior is stiff and lacks variation, which further results in a single game experience for the player, and as the number of behaviors increases and the condition increases, the complexity of the behavior tree will increase exponentially, and it is difficult to debug and maintain.

As described above, the behavior decision part of the game AI is important content for intelligence, and unlike the traditional artificial intelligence concerned decision, the game AI needs to consider the reality of the decision while searching for the optimal solution, and the behavior of the game AI should have a certain ambiguity as the intelligent biology. In certain circumstances, when a game AI is faced with multiple choices, it needs to take decisions that the user feels correct, do the corresponding effective actions, rather than searching for the optimal decisions for the problem rationally. The reasonable behavior decision method can make the interaction between the player and the game AI more realistic, and has important significance for the player experience of the game.

The present disclosure provides an intelligent virtual character control method, which determines a behavior to be controlled to be executed by an intelligent virtual character based on probability according to environment detection data of the intelligent virtual character through layer-by-layer control of the intelligent virtual character, so as to establish a multi-layer virtual character design method with explicit layering and division.

Fig. 2A is a flowchart illustrating an intelligent virtual character control method 200 according to an embodiment of the present disclosure. Fig. 2B is a schematic diagram illustrating a hierarchical structure of an intelligent virtual character control method according to an embodiment of the present disclosure.

The intelligent virtual role control method of the present disclosure may be divided into five layers, namely, an observation layer, a judgment layer, a policy decision layer, a behavior decision layer and an execution layer from bottom to top in fig. 2B.

As shown in fig. 2A, in step 201, a first amount of environment detection data in the vicinity of the smart avatar may be acquired.

The operation of step 201 may be performed in the viewing layer as shown in fig. 2B, according to an embodiment of the present disclosure. Fig. 3A is a schematic flowchart showing a process at an observation layer in the intelligent virtual character control method according to an embodiment of the present disclosure.

Alternatively, in step 201, detection results such as game environments and combat targets in the vicinity of the game AI may be output using a detection algorithm constructed by the game based on the game engine to realize the cognition of the game AI to the in-game objective environment. The observation layer may perform a process of recognition of the in-game objective environment by the game AI, including, for example, but not limited to, visual perception, auditory perception, tactile perception (e.g., hit by an attack, blocked by an obstacle, etc.), spatial awareness (e.g., awareness of a path, obstacle, shelter, etc.), and the like.

Accordingly, objective observation data (i.e., environment detection data) can be obtained therefrom based on real-time processing of objective game environment data by the detection algorithm of the above-described recognition process. It should be understood that these observation data are objective judgments for objective things (game environments), and do not include subjective judgments. For example, the observation data may be a judgment of "distance" rather than "distance", a judgment of "number" rather than "how much", and the like.

Fig. 3B is a schematic diagram illustrating an example process in an observation layer according to an embodiment of the present disclosure.

As shown in fig. 3B, the observation layer may perform tasks such as object detection, path finding, and shelter point searching, thereby enabling determination of such things as "whether an object is visible", "whether a path distance to an object is beyond a feasible range", and "whether a shelter point exists in the vicinity".

Optionally, game AI may use radiation detection to determine whether the target is visible. As shown in the first column in fig. 3B, the calculation and judgment can be made by a plurality of rays shown by thick broken lines in the figure. For example, rays may be issued from various angles to the target to determine whether the rays are occluded by an obstacle, and thus whether the target is visible. Alternatively, the detectable range of the ray should be limited, because the game AI should detect objects similar to humans identifying objects through visual perception, and not "cheat" detect objects that are out of line of sight.

In practice, the viewing layer should balance performance consumption and viewing accuracy. For example, for whether the target is visible, a rough observation may only determine whether the head/chest of the target is visible, while a fine observation may be subdivided into head, chest, hand, thigh, calf, etc. The finer the observation, the more realistic the observation, the more sensitive the game AI, but this also results in greater server-generated performance consumption. Therefore, the observation layer needs to reasonably determine parameters in the detection algorithm based on actual task requirements and available resources.

Alternatively, game AI may learn the spatial structure through a pre-set navigation routing system (e.g., navigation grid (NavMesh)) for path searching. As shown in the second column of fig. 3B, since there may be an obstacle blocking between the game AI and the target, so that the game AI cannot directly approach the target, it is necessary to determine an available path through a path search, as shown by the broken line in the second column of fig. 3B.

Optionally, the game AI can also calculate its nearby shelter points through a pre-build of its pre-configured game space. As shown in the third column in fig. 3B, there may be a plurality of shelter points (shown as oval white points around the square in the figure) near the shelter and facing away from the target within the feasible range of the game AI.

As described above, the method of real players observing the world can be abstracted to guide how the correlation detection algorithm of the observation layer is designed. The algorithm for computing the shelter points in the observation layer is designed, for example, by generalizing the logic of the real player's judgment of the shelter in the scene.

Next, in step 202, a second number of environmental judgment results may be generated based on the first number of environmental detection data, the second number being not greater than the first number.

The operation of step 202 may be performed in a decision layer as shown in fig. 2B, according to an embodiment of the present disclosure.

Fig. 4A is a schematic flowchart showing a process at a judgment layer in the intelligent virtual character control method according to an embodiment of the present disclosure. Alternatively, as shown in fig. 4A, subjective cognitive judgment results may be generated according to the cognitive parameter configuration of the judgment layer using objective observation data output by the observation layer.

According to an embodiment of the present disclosure, step 202 may include: classifying the first number of environment detection data according to a predetermined cognitive configuration parameter set to generate the second number of environment judgment results; wherein the set of cognitive configuration parameters may be set for the intelligent virtual character, and one parameter of the set of cognitive configuration parameters may correspond to one behavior of the intelligent virtual character.

Alternatively, the decision layer may perform subjective cognitive summaries of objective data obtained by the observation layer, including, for example, but not limited to, battle situation (e.g., my/enemy is dominant, nearby is dangerous), enemy goal status (e.g., equipment strength, health status), tactical point decision (e.g., nearby best shelter point, best path to attack goal), etc.

Alternatively, the predetermined set of cognitive configuration parameters may include subjective criteria for all types of environmental data, e.g., for a "path distance" between the game AI and the target obtained by the observation layer, the "distance" of the game AI from the target may be determined based on the distance criteria (e.g., classification threshold, etc.) included in the set of cognitive configuration parameters.

Fig. 4B is a schematic diagram showing an example process in a determination layer according to an embodiment of the present disclosure.

As shown in fig. 4B, the judgment layer may determine the degree of self-safety based on detection of information such as blood volume, equipment and distance of the observation layer on the self and enemy according to the judgment standard set in advance on the self. For example, the degree of self-safety may be based on observation data of the blood volume of the self, equipment, distance from enemies, and the like.

Optionally, the classification, summarization and generalization of the objective observation data can be completed by the judgment layer, so that the number of the output subjective cognitive judgment is significantly smaller than the number of the input objective observation data, and the space complexity of the judgment conditions of the subsequent layers is reduced.

For example, based on a first number of environmental detection data ("objective observation data") acquired in the vicinity of the smart virtual character, for example, the environmental detection data may include blood volume data, equipment data, and straight line distance and path distance between the smart virtual character and the enemy, i.e., the first number is 6, based on the acquired first number of environmental detection data ("objective observation data") in the vicinity of the smart virtual character, and a second number is 2, which is a "subjective cognitive judgment" result such as the degree of self-safety of the smart virtual character and the degree of threat of the enemy, regardless of other types of environmental detection data and environmental judgment results. The intelligent virtual character safety degree can be determined based on the blood volume data, the equipment data and the straight line distance and the path distance between the intelligent virtual character and the enemy of the intelligent virtual character, and the enemy threat degree of the intelligent virtual character can be determined based on the blood volume data, the equipment data and the straight line distance and the path distance between the intelligent virtual character and the enemy of the enemy.

Alternatively, the cognitive configuration parameter set should have sufficient configuration space and parameter types to set different cognitive configuration parameter sets for different game AI's, thereby designing a plurality of game AI's with different characters and different behavioral tendencies. For example, by adjusting the "safe blood volume" parameter in the cognitive configuration parameter set, the determination of whether different game AI's are safe for themselves can be controlled.

Optionally, the design of the cognitive configuration parameter set of the judgment layer can follow the thought of controlling one kind of behavior by one kind of parameter, so that the influence of one parameter on a plurality of behaviors is avoided. For example, the "safe blood volume" parameter may only have an effect on the determination of whether or not it is safe by itself, and not on the determination of whether or not it is disadvantaged by the enemy, which may be controlled by another parameter (e.g., the "enemy safe blood volume" parameter in the cognitive configuration parameter set of the game AI).

As described above, based on the plurality of subjective cognitive judgment results determined in step 202, subsequent decision making and execution processing may be performed.

In step 203, a first behavior policy may be determined from the plurality of behavior policies of the intelligent avatar based on the second number of environmental decisions, the determined first behavior policy indicating that the intelligent avatar has a higher probability of selecting to perform a behavior corresponding to the first behavior policy than other behavior policies of the plurality of behavior policies of the intelligent avatar.

The operation of step 203 may be performed in a policy decision layer as shown in fig. 2B, according to an embodiment of the present disclosure.

Fig. 5A is a schematic flow chart diagram illustrating a process at a policy decision layer in an intelligent virtual role control method according to an embodiment of the present disclosure. Alternatively, as shown in fig. 5A, the subjective cognitive judgment output by the judgment layer may be used to generate a policy decision result according to the policy decision parameter configuration of the policy decision layer.

Alternatively, the policy decision layer may generate the behavior policy (behavior tendency) of the game AI according to the parameter configuration of the policy decision layer of the game AI by using subjective cognitive data output by the decision layer, so that the game AI with different characters as described above can exhibit different behavior tendencies for a longer time.

According to an embodiment of the present disclosure, each of the plurality of behavior policies of the intelligent virtual character may correspond to at least one behavior, and each of all behaviors of the intelligent virtual character may correspond to at least one behavior policy.

Alternatively, the policy decision layer may determine a plurality of explicit and representative action policies available for selection by the game AI, and divide all actions that the game AI may take into account the action policies to which it belongs, which may include the same actions (i.e., the same action may belong to a plurality of policies), since a single action taken by the game AI typically does not reflect its propensity to take action over a period of time in the future, and implementation of a single action policy typically depends on sequential execution of the plurality of actions.

Fig. 5B is a schematic diagram illustrating example policies and behaviors in a policy decision layer according to an embodiment of the disclosure.

As shown in fig. 5B, the action strategies available for game AI selection may include attack and defense, and accordingly, the actions corresponding to attack may include frontal attack, lateral attack, and replacement of the shelter, and the actions corresponding to defense may include returning the shelter, shooting from the shelter, and moving away from the enemy, and the like. It should be understood that the behavior policies and behaviors shown in the figures are used in the present disclosure by way of example only and not limitation, the behavior policies of the present disclosure may also include a variety of behavior policies such as steal, escape, etc., and the behavior policies of the present disclosure may also include various behaviors corresponding to these variety of behavior policies.

According to an embodiment of the present disclosure, step 203 may include: determining the probability of the intelligent virtual character executing each of the plurality of behavior strategies based on the second number of environment judgment results and preset strategy decision configuration parameters, wherein the strategy decision configuration parameters can indicate the correlation between the environment judgment results and the behavior strategies; and randomly selecting one of the plurality of behavior policies as the first behavior policy based on the determined probability that the intelligent avatar performs each of the plurality of behavior policies.

Optionally, the policy decision layer selects a behavior policy to be adopted by the policy decision layer based on probability according to an environment judgment result output by the judgment layer and preset configuration parameters. Wherein the pre-configured policy decision configuration parameters may include the influence coefficient of each type of judgment result of the game AI on the environment on each type of policy decision, for example, the contribution of a specific judgment result on the implementation possibility of a certain type of policy decision, i.e. whether the game AI is more prone to implement the policy decision under the judgment result.

As described above, the action policy determined in the policy decision layer does not directly decide what action the game AI must take at the next time, but affects the distribution and proportion of the game AI actions over a period of time. For example, when determining a behavior policy to select an attack, the game AI does not necessarily perform various behaviors corresponding to the attack, such as shown in fig. 5B, but the probability that the game AI selects to perform a behavior under the attack policy will be significantly raised over a period of time.

As an example, fig. 5C is a schematic diagram illustrating example policy decisions in a policy decision layer according to an embodiment of the disclosure.

In fig. 5C, the current state of the game AI has been judged by the judgment layer (e.g. "whether equipment is dominant", "whether shelter is dominant" and "whether blood volume is dominant" shown on the left side of the figure), and in the case of this subjective judgment, the game AI may determine its respective probabilities of selecting "attack" or "defense" (here assuming that the game AI selects only among these two types of behavior strategies) (shown as 72% and 28% in fig. 5C, respectively), so that one of the behavior strategies is randomly selected using a random number function (wherein "attack" is selected with a probability of 72% and "defense" is selected with a probability of 28%). This process also reflects the similarity of game AI to real human thinking, which makes a choice of behavior strategies based on subjective judgment of itself, but after a specific behavior strategy is decided, the behavior under that behavior strategy may not be chosen to be executed, but the behavior to be executed is determined at the last moment before execution, except that the probability that the behavior to be executed is the behavior under that behavior strategy is higher than the probability of the behavior under other behavior strategies.

Alternatively, the policy decision layer may not directly output the selected policy decision result (i.e., the first behavior policy), but indirectly influence the result of the behavior decision by influencing the parameters, algorithms, weights, etc. of the behavior selection process involved in the behavior decision layer based on its policy decision result.

Alternatively, the policy decision layer may be an open hierarchy, except that the decision-making game AI itself affects its policy decision results, such as team leader (e.g., the master of the cluster game AI), game AI global controller, game AI intensity controller, game AI director, etc., may affect the performance of the game AI by affecting the outcome of the policy decision layer.

According to an embodiment of the present disclosure, the probability of the intelligent avatar determined in the current control to execute each of the plurality of behavior policies is derived based on an update to the probability of the intelligent avatar determined in the previous control to execute each of the respective plurality of behavior policies, the update being based on one or more of the following time intervals: a predetermined time interval; or a time interval between a time when a predetermined update condition is satisfied and a time in a previous control when a probability of the intelligent avatar executing each of a corresponding plurality of behavior strategies is determined. Wherein, according to an embodiment of the present disclosure, the current control and the previous control correspond to a current control of the smart avatar and a last control of the smart avatar, respectively.

Alternatively, the game AI's selection of the behavior policy may be different at each moment in time, as the game environment may change over time requiring the game AI to respond in real time, and the game AI's selection of the response behavior is also randomly selected based on probabilities, which are updated based on the mechanisms described above.

Alternatively, the probability of selection of a behavior strategy may be updated at least at predetermined intervals to ensure the lowest reaction speed of the game AI. In addition, the probability may be updated based on various changes in the scene to ensure the immediacy of the response of the game AI to the environmental changes.

For example, the predetermined time interval may be set to a fixed number of seconds, after which the game AI automatically updates the probability of itself executing various action strategies.

In addition, the behavior trend of the game AI can be updated in real time when a more significant change occurs in the game environment, such as when an enemy suddenly appears in the environment, the blood volume reduction rate of the game AI exceeds a preset threshold value, or when the enemy leaves the field of view for more than 10 seconds when a defending strategy is adopted.

It should be understood that the descriptions of "current control" and "previous control" in the present disclosure correspond to the present-round game AI control and the previous-round game AI control, respectively, and each round may output an operation to be executed in order to control the game AI.

In step 204, a behavior to control execution of the intelligent avatar may be determined based on the second number of environmental decisions and the determined first behavior policy, and execution conditions of at least a portion of all behaviors of the intelligent avatar.

The operation of step 204 may be performed in a behavior decision layer as shown in fig. 2B, according to an embodiment of the present disclosure.

Fig. 6A is a schematic flow chart diagram illustrating a process at a behavior decision layer in an intelligent virtual role control method according to an embodiment of the present disclosure. Alternatively, as shown in fig. 6A, the behavior decision result may be generated according to the behavior decision parameter configuration of the behavior decision layer based on subjective cognitive judgment output by the judgment layer and the policy decision result of the policy decision layer.

Optionally, the behavior decision layer may use the policy decision result output by the policy decision layer and the subjective cognitive result output by the judgment layer to generate, according to the parameter configuration of the behavior decision layer, a behavior decision result of the game AI, so as to determine what behavior the game AI specifically performs. The behavior decision parameter configuration may include all execution conditions related to all behaviors of the game AI and their relationships to various behavior policies and various behaviors under these behavior policies.

According to embodiments of the present disclosure, the execution condition of each of all behaviors of the intelligent avatar may include a necessary execution condition and an optional execution condition. Alternatively, each of all of the behaviors of the game AI may have its corresponding mandatory and optional execution conditions.

Alternatively, the action decision layer may select an action to control the execution of the game AI from among all actions that the game AI may execute, which should cover the execution conditions of these all actions, and define the "necessary execution conditions" and the "optional execution conditions" of each action, thereby determining all actions that the game AI may execute in the current game environment.

Alternatively, the "necessary execution condition" may mean that when the condition of a certain action is satisfied, the game AI must select to execute the action. While "optional execution condition" means that when the condition of a certain action is satisfied, the game AI can select this action, but does not necessarily select this action, but selects from all actions satisfying the "optional execution condition" without any "optional execution condition". For a certain behavior, if the current situation does not satisfy its "optional execution condition" nor its "optional execution condition", then the behavior cannot be executed. This process avoids unreasonable behavior of the game AI, and also ensures that its behavior has a certain rationality while ensuring flexibility and randomness of its behavior.

Optionally, the setting of execution conditions (including "must-execute conditions" and "optional execute conditions") for any of the actions needs to start from "what conditions the action can execute under" to ensure that all possible trigger conditions for the action can be mined out.

According to an embodiment of the present disclosure, step 204 may include: determining at least one execution condition which is met by the intelligent virtual character and at least one action corresponding to the at least one execution condition based on the second number of environment judgment results and the execution conditions of at least one part of all actions of the intelligent virtual character; and under the condition that one or more than one necessary execution condition exists in at least one execution condition met by the determined intelligent virtual character, selecting the necessary execution condition with the highest priority among the one or more than one necessary execution condition based on the preset priority of the one or more than one necessary execution condition, and taking the action corresponding to the necessary execution condition as the action to be controlled to execute the intelligent virtual character.

According to an embodiment of the present disclosure, at least one behavior corresponding to the at least one execution condition may constitute a behavior pool, and a behavior to be executed by the game AI may be selected from the behavior pool.

As described above, the execution condition currently satisfied by the game AI may be determined based on the environmental judgment result of the judgment layer, wherein the judgment for the necessary execution condition may be performed first.

Alternatively, it may be determined whether one or more indispensable execution conditions exist in at least one execution condition satisfied by the intelligent virtual character, and if the indispensable execution conditions do not exist, the following operation may be continued, and if the indispensable execution conditions exist, the behavior corresponding to the indispensable execution condition with the highest priority may be selected for the game AI to execute according to the priority preset for the indispensable execution conditions. The priority level may be set differently according to the character of different game AI, for example, for a game AI in which character is careful, the highest priority level may be set to a condition related to the degree of self-security.

Alternatively, the setting of the necessary execution conditions may be made based on the most basic reflection behavior. Fig. 6B is a schematic diagram illustrating example execution conditions in a behavior decision layer according to an embodiment of the present disclosure.

As shown in fig. 6B, the optional execution condition may be "none" for a particular behavior, i.e., there is no scene that must be executed for that behavior. For example, for "side attack" or "away from target" behavior, there is no requirement for this. While for the "front attack" and "return shelter" behaviors, the indispensable execution conditions may be set to "enemy is in an dying state" and "most recently 1 second is attacked", respectively, because for both conditions, most people tend to select these two behaviors, enemy tends to hit by the winning in a heavily wounded state, and when just attacked tends to find a shelter first and then determine the subsequent behavior, and thus the indispensable execution conditions are set based on the condition reflection similar to human.

Alternatively, these actions may have one or more selectable execution conditions, and in the event that at least one of its selectable execution conditions is met, the corresponding action may be added to the pool of actions without having to prioritize it. For example, in FIG. 6B, when the condition "enemy is in a non-healthy state" is satisfied, game AI may perform a front attack, so the "front attack" behavior may be added to the behavior pool.

Alternatively, the algorithm for selecting the final execution behavior from the behavior pool needs to ensure randomness of the selection, and even in the case that conditions such as external environment are not changed, the behavior result obtained by each selection from the behavior pool should be changed instead of being kept the same.

Step 204 may further include, in accordance with an embodiment of the present disclosure: determining a probability of the intelligent virtual character executing each of the at least one behavior based on the determined first behavior policy and the at least one behavior corresponding to the at least one execution condition, in the case that one or more alternative execution conditions do not exist in the determined at least one execution condition satisfied by the intelligent virtual character; and randomly selecting an action to control the intelligent avatar to perform based on the determined probability of the intelligent avatar performing each of the at least one action.

As described above, after all the optional execution conditions satisfied by the game AI are added to the action pool, an action to be executed may be selected from the action pool, and the selection may be made based on the probability that each action in the action pool is executed.

According to an embodiment of the present disclosure, determining the probability that the intelligent avatar performs each of the at least one behavior based on the determined first behavior policy and the at least one behavior corresponding to the at least one execution condition includes: in the case that one or more behaviors corresponding to the determined first behavior policy exist in at least one behavior corresponding to the at least one execution condition, increasing a probability that the intelligent virtual character executes each of the one or more behaviors.

Alternatively, according to the first behavior policy selected by the policy decision layer, the game AI may be considered to be more prone to perform the behavior under the first behavior policy in the current situation, and thus, the behaviors belonging to the first behavior policy in the behavior pool may have a higher probability of being selected. For example, for a behavior in the behavior pool that belongs to the first behavior policy, its weight may be increased (e.g., doubled) to increase its probability of being selected.

According to an embodiment of the present disclosure, determining the probability that the intelligent avatar performs each of the at least one behavior based on the determined first behavior policy and the at least one behavior corresponding to the at least one execution condition may further include at least one of: increasing the probability of the intelligent avatar executing each of the one or more behaviors in the currently controlled pool of behaviors that do not belong to the previously controlled pool of behaviors; or increasing the probability that the intelligent avatar performs each of the one or more behaviors in the current controlled behavior pool if there are one or more behaviors in the current controlled behavior pool that are different from the behaviors performed by the intelligent avatar controlled by previous controls.

Alternatively, for one or more actions in the pool of actions that are newly occurring in the current control (i.e., not present in the previous control, but newly joining the pool of actions in the current control), these actions may be considered to be responsive to the real-time changes occurring in the current gaming environment, which are closely related to the real-time changes, so selecting these actions may be more reasonably more real-time reactions to the current environment, making the action of the game AI more immediate. Thus, the weight of these actions can be increased, thereby increasing the probability of their selection.

Alternatively, the probability of one or more of the behaviors in the pool of behaviors being selected this time may be increased as different from the behaviors selected in the previous control, because the higher probability behaviors may be continuously selected due to the probability selection mechanism, which may result in a single behavior of the game AI, so increasing the probability of these behaviors may make the behavior of the game AI more flexible. Thus, the weight of these actions can be increased, thereby increasing the probability of their selection.

Step 204 may further include, in accordance with an embodiment of the present disclosure: and executing preset default behaviors under the condition that at least one execution condition met by the intelligent virtual roles does not exist.

Alternatively, when the current environment does not satisfy any execution condition, the game AI may execute a default behavior (e.g., wait in place, etc.), and wait for the control process at the next time.

According to an embodiment of the present disclosure, each of all the behaviors of the intelligent avatar includes a combination of a series of actions. Optionally, the intelligent avatar control method 200 may further comprise step 205, i.e. a sequence of actions comprised by controlling the intelligent avatar to perform the determined behavior.

The operation of step 205 may be performed in an execution layer as shown in fig. 2B, according to an embodiment of the present disclosure.

Fig. 7A is a schematic flowchart showing a process at an execution layer in the intelligent virtual character control method according to an embodiment of the present disclosure. Alternatively, as shown in fig. 7A, a series of actions included in the determined behavior may be performed according to an execution parameter configuration of the execution layer using a behavior decision result output by the behavior decision layer.

Alternatively, the execution layer may relate to any particular behavior that the game AI is capable of exhibiting outward that may be perceived by the player, including, but not limited to, such as moving, steering, changing posture (e.g., standing, squatting, lying on or sideways, etc.), aiming, firing, speaking, etc.

It should be noted that each behavior of game AI should be a "describable" and "unambiguous" behavior. Wherein "describable" can mean that this behavior can be described as a combination of several actions; while "unambiguous" may mean that anyone should be able to learn the specific content of the behavior through the description, and that everyone has little difference in understanding the same behavior.

As an example, fig. 7B is a schematic diagram illustrating an example execution action flow in an execution layer according to an embodiment of the present disclosure.

As shown in FIG. 7B, the behavior "shoot from behind the shelter" may include a sequential combination of three actions, namely a probe-aiming-firing action combination.

According to an embodiment of the present disclosure, during controlling the intelligent avatar to perform the series of actions, at least one of: immediately ending execution of the series of actions in a case where a predetermined ending condition is satisfied, the predetermined ending condition being set to a different condition based on success or failure of the series of actions; immediately ending the execution of the series of actions and switching to the execution of a series of actions comprised by another action in case a predetermined interrupt condition is fulfilled; or immediately ending the execution of the series of actions in case the execution time of the series of actions exceeds the preset execution time and the predetermined end condition or the predetermined interrupt condition is not satisfied.

Alternatively, each behavior may have an explicit ending mechanism, a breaking mechanism, and a timeout mechanism. The ending mechanism may refer to ending the behavior immediately when a predetermined ending condition is satisfied, which may be set differently depending on success and failure of execution of the behavior. The interrupt mechanism may refer to ending the action immediately when a predetermined interrupt condition is met and instead executing another action. The timeout mechanism may refer to a mechanism that, in the presence of a timer for an action, if the execution time of the action exceeds a preset execution time without triggering an ending mechanism/interrupting mechanism, considers that the action is executing a timeout and immediately ends the action.

Alternatively, an action may be switchable, i.e. any action may be immediately interrupted at any time during execution and switched to execute another action. For example, the movement may be stopped immediately at any point during the execution of the "move to a shootable shelter" and the "shoot outside shelter" may be executed instead.

Alternatively, any of the behaviors may be combinable, i.e., a plurality of simple behaviors may be combined into one complex behavior by means of sequential or conditional combinations. For example, the two actions of "move to a shootable shelter" and "shoot from behind the shelter" may be combined in a spliced manner to form a "move to shelter and shoot" action.

As a summary of the above-described processes, fig. 8 is a schematic diagram showing a hierarchical structure of an intelligent virtual character control method and its related processes according to an embodiment of the present disclosure.

As shown in fig. 8, the behavior to be executed by the game AI may be determined based on the environmental data acquired from the game environment in real time.

At the observation layer, objective observation data near the game AI can be output by using the constructed detection algorithm so as to realize objective cognition of the game AI on objective environment in the game.

In the judging layer, objective observation data output by the observation layer can be used for generating a subjective cognition judging result according to the cognition parameter configuration of the judging layer so as to realize subjective cognition of the game AI on the objective environment in the game.

In the policy decision layer, subjective cognitive judgment output by the judgment layer can be used for generating a policy decision result according to the policy decision parameter configuration of the policy decision layer so as to reflect the behavior tendency of the game AI for the real-time game environment.

In the behavior decision layer, a behavior decision result can be generated according to the subjective cognitive judgment output by the judgment layer and the strategy decision result of the strategy decision layer and the behavior decision parameter configuration of the behavior decision layer so as to determine the behavior to be executed by the game AI.

At the execution layer, a series of actions included in the determined behavior may be performed according to an execution parameter configuration of the execution layer using the behavior decision result output by the behavior decision layer.

An example process flow of the intelligent avatar control method of the present disclosure will be further described below with respect to specific embodiments of the present disclosure.

Fig. 9A is a diagram illustrating example parameter configurations of a decision layer, a policy decision layer, and a behavior decision layer of an intelligent avatar control method according to an embodiment of the present disclosure. Fig. 9B is a schematic flow chart diagram illustrating performing a lateral attack according to an embodiment of the present disclosure.

For example, the observation layer may perform the following operations to obtain a plurality of objective observation data:

1) Reading game environment data by a game engine;

2) From the eye position of the game AI character, the head, chest, hands and feet joints of the enemy are subjected to ray detection, and all detection results of the positions are found to be blocked, so that an observation result of 'the enemy is invisible';

3) Calculating the straight line distance from the game AI character to the enemy character to be 15 meters, and obtaining an observation result of 'the enemy distance to be 15 meters';

4) Calculating the path finding distance from the game AI character to the enemy character to be 21 meters, and obtaining an observation result of 'enemy path distance 21 meters';

5) Calculating the blood volume percentage of the game AI role to be 91%, and obtaining an observation result of 'self blood volume to be 0.91'; and

6) The enemy character blood volume percentage was calculated to be 40% to obtain the observation result "enemy blood volume 0.4".

Based on the objective observation data, the judgment layer may perform the following operations to generate a plurality of subjective cognitive judgment results:

1) Reading an output result of the observation layer, parameter configuration of the judgment layer as shown in (a) in fig. 9A, and calculation cache of the judgment layer;

2) According to the observation result that the enemy is invisible, combining the judgment layer to calculate and cache to obtain the enemy leaving the field of view time of 8 seconds;

3) According to the observation results, namely the enemy distance is 15 meters and the enemy path distance is 21 meters, the ratio of the path distance to the enemy position to the straight line distance is 1.4, which is smaller than the "offensive detour coefficient" in the configuration parameters, and the judgment result, namely the target is in the attack range, is obtained;

4) According to the observation result that the self blood volume is 0.91 and is larger than the self healthy blood volume in the configuration parameters, a judgment result that the self blood volume is in a healthy state is obtained; and the judgment result of 'self-in-non-moribund state' is obtained according to the observation result being larger than 'self-moribund blood volume' in the configuration parameters; and

5) According to the observation result, the enemy blood quantity is 0.4 and smaller than the enemy healthy blood quantity in the configuration parameters, the observation result, the enemy is in a non-healthy state, is obtained; and the judgment result of 'the enemy is in a non-moribund state' is obtained according to the observation result being larger than 'the moribund blood volume' in the configuration parameters.

Next, based on the subjective cognitive judgment result, the policy decision layer may perform the following operations to generate a policy decision result:

1) Reading the output result of the judgment layer and the parameter configuration of the policy decision layer as shown in (b) in fig. 9A;

2) The weights of the behavior strategies of attack and defense are initialized to be 1 and 1 respectively;

3) According to the judgment result that the target is in the attack range, reading the parameter configuration that the policy weight influence is influenced when the target is in the attack range obtains the result: attack weight +1, defense weight +1;

4) According to the judging result of 'self health state', reading parameter configuration 'self health time countermeasure weight influence' to obtain a result: attack weight +1;

5) According to the judgment result, reading the parameter configuration of the enemy in the unhealthy state, and obtaining a result of the 'enemy unhealthy time strategy weight influence': attack weight +1;

6) According to the accumulated weight, attack and defense=4:2, the probability of selecting attack is 66%, and the probability of selecting defense is 33%; and

7) And obtaining a random strategy decision result as attack by using a random number function.

Therefore, according to the subjective cognitive judgment result and the policy decision result, the execution layer may execute the following operations to generate a behavior decision result:

1) Reading the output result of the judgment layer, the output result of the strategy decision layer and the parameter configuration of the behavior decision layer as shown in (c) in fig. 9A;

2) According to the results output by the judging layer, the enemy leaving the field of view time is 8 seconds, the target is in the attack range, the target is in the healthy state, the target is in the non-dying state, the enemy is in the non-healthy state and the enemy is in the non-dying state, and the method comprises the following steps of: the optional execution conditions of any behavior are not met, and the optional execution conditions of the front attack, the side attack and the back attack are met, so that the three behaviors are added into a behavior pool;

3) Reading historical behavior decision data to obtain 'front attack and lateral attack' in a last behavior pool, wherein a random selection result is front attack, so that the weight +1 of the shelter is laterally attacked and returned in the behavior decision to obtain front attack: and (3) laterally attacking: the weight ratio of the returned shelter is 1:2:2;

4) According to the output result 'attack' of the policy decision layer, the weight of the front attack and the lateral attack belonging to the classification of the 'attack' can be doubled, so that the front attack is obtained: and (3) laterally attacking: the weight ratio of the returned shelter is 2:4:2;

5) According to the frontal attack: and (3) laterally attacking: the weight ratio of the returned shelter is 2:4:2, the probability of selecting the front attack is 25%, the probability of selecting the side attack is 50%, and the probability of selecting the returned shelter is 25%; and

6) The random result of the behavior decision layer is "side attack" using a random number function.

Thus, according to the behavior decision result of the behavior decision layer, the execution layer may execute a series of actions corresponding to the result as shown in fig. 9B:

1) Reading the output result of the behavior decision layer, namely 'side attack';

2) Reading action sequence data of a lateral attack behavior:

Firstly, acquiring the position of 15 meters on the side surface of an enemy;

then move to the acquired location during which movement is continuously maintained if the enemy is not visible; if the enemy is visible, stopping moving and shooting;

3) A "side attack" action is performed.

Fig. 9C is a schematic diagram illustrating selection probabilities for different behaviors in the same environment according to an embodiment of the present disclosure.

As shown in fig. 9C, in the same game environment, the policy decision layer and the behavior decision layer implement selection of the behavior policy and behavior based on probabilities, respectively, and through the policy selection layer and the behavior decision layer, the behavior is selected from the front attack, the side attack and the return shelter with different probabilities.

The above process ensures that behaviors that do not meet the current conditions are not selected (e.g., the previously-described "far from target" behaviors that are not considered), and that even if the external environment is unchanged, the AI behaviors are reasonably random and exhibit a significant tendency (if the current behavior is in the attack strategy, the probability of the attack strategy being selected is higher). The final decision result is truly flexible and can produce a wide variety of completely different interaction procedures even for two AI's of the same scene, the same configuration.

As described above, the intelligent virtual character control method of the present disclosure realizes the structural design and control of the game AI, and the game AI designed and realized by the structure has clear logic and clear level, and the behavior thereof has randomness, rather than the determined behavior based on the judgment result, so that the behavior of the game AI is more consistent with the human behavior, and has higher instantaneity, flexibility and game level.

It should be understood that the parameters and their value settings included in the various layers described above, as well as the specific processing of the various layers, are used in this disclosure for illustrative purposes only and not limitation, and other modifications that may achieve similar effects or be made without departing from the scope of this disclosure are equally applicable to the intelligent avatar control methods of this disclosure.

Fig. 10A is a schematic diagram illustrating an example division of effort of decision layer, decision layer (policy decision layer and behavior decision layer) and execution layer of an intelligent virtual role control method according to an embodiment of the present disclosure. Fig. 10B is a schematic flow chart diagram illustrating adding new behaviors before and after implementation of the intelligent virtual role control method according to an embodiment of the present disclosure. Fig. 10C is a schematic diagram illustrating the division of work of the layers of the intelligent virtual character control method according to an embodiment of the present disclosure.

As described above, the judgment layer, the decision layer (policy decision layer and behavior decision layer) and the execution layer of the intelligent virtual character control method of the present disclosure can be used to classify game Al, but their roles are not the same.

Alternatively, for the judgment layer, it may make subjective judgment as described above, such as judgment of "what is far". As shown in fig. 10A, the judgment layer may judge "how far away" for the magnitude of distance, "how expensive" and "how strong" for the value of the enemy equipment (where "expensive" refers to how high the equipment is, and "strong" refers to how high the equipment is in combat), and "how much" for the number of enemy, etc.

Alternatively, for the decision layer, it may make selections of behavior strategies and behaviors as described above, such as selecting "do" or "do not. As shown in fig. 10A, the decision layer may make a selection such as "whether to throw a mine", "whether to fire", "whether to wrap" or "whether to withdraw".

Alternatively, for the execution layer, it may make specific selections regarding behavior execution, such as selecting "do not good". As shown in fig. 10A, the execution layer may select a selection such as "throw distance", "throw accuracy", "hit rate", or "shot aiming position".

As described above, by dividing the functions of each layer, a large number of game AI of different types can be flexibly assembled and formed, and the workload of designing different AI is greatly reduced. Further, it is also possible to realize division of areas in the game world based on the game AI thus set, such as a new hand area (game AI gall defence, low shooting hit rate, etc.) and a high hand area (game AI aggressive multi-attack, high shooting hit rate, etc.).

Further, for the case where a new behavior is to be added for game AI, fig. 10B shows different addition processing of "find shelter" for the same new behavior before and after implementing the intelligent virtual character control method of the present disclosure.

As shown in (a) of fig. 10B, the conventional method as described above requires that addition of new behaviors be completed based on condition judgment at each branch of the behavior tree and the added new behaviors be dispersed throughout the behavior tree, since the behavior tree is built according to various conditions existing in the environment and a plurality of different behaviors may be included under the same branch of the behavior tree.

As an alternative to the above method, as shown in (B) of fig. 10B, the method of the present disclosure determines the condition for triggering the behavior from the behavior, so all the conditions related to the same behavior in the corresponding structure are located in the same branch, and in this structure, only one new branch needs to be additionally established for the new behavior and all the conditions corresponding to the behavior need to be mined to fill the branch, so that the addition of the new behavior can be completed. The method of the embodiment of the disclosure has strong expandability and is convenient for later debugging and maintenance.

In fig. 10C, TD (Technical Designer, technical plan) and LD (Level Designer) are two subdivision positions under a game plan position, where TD is responsible for design work of a game function (e.g., game AI) with a certain development difficulty in a game, and LD is responsible for design work of a game map Level.

As shown in fig. 10C, each level of work is respectively responsible for a specific staff along with development progress, and the hierarchical structure designed by the intelligent virtual character control method of the present disclosure realizes structural design and control of the game AI, which is beneficial to division and development of actual work.

Fig. 11 is a schematic diagram illustrating an intelligent virtual character control apparatus 1100 according to an embodiment of the present disclosure.

As shown in fig. 11, the intelligent virtual character control apparatus 1100 may include an environment detection module 1101, an environment judgment module 1102, a policy decision module 1103, and a behavior decision module 1104.

According to an embodiment of the present disclosure, the environment detection module 1101 may be configured to obtain a first amount of environment detection data in the vicinity of the intelligent avatar.

Alternatively, the environment detection module 1101 may perform the operations of the observation layer as described with reference to fig. 2B.

Alternatively, the environment detection module 1101 may output detection results, such as game environments and combat targets, in the vicinity of the game AI using a detection algorithm built by the game based on the game engine to enable awareness of the game AI to the in-game objective environment. The observation layer may perform a process of recognition of the in-game objective environment by the game AI, including, for example, but not limited to, visual perception, auditory perception, tactile perception (e.g., hit by an attack, blocked by an obstacle, etc.), spatial awareness (e.g., awareness of a path, obstacle, shelter, etc.), and the like.

The environment determination module 1102 may be configured to generate a second number of environment determination results based on the first number of environment detection data, the second number being not greater than the first number.

Alternatively, the environment judgment module 1102 may perform the operation of judging a layer as described with reference to fig. 2B.

Alternatively, the environment determination module 1102 may perform subjective cognitive summaries of the objective data obtained by the environment detection module 1101, including, but not limited to, battle situations (e.g., my/enemy is in a predominance, there is a danger nearby), enemy goal states (e.g., equipment strength, health status), tactical point determinations (e.g., best nearby shelter points, best path to attack the goal), and so forth.

Alternatively, the predetermined set of cognitive configuration parameters may include subjective criteria for all types of environmental data, e.g., for a "path distance" between the game AI and the target obtained by the environmental detection module 1101, the "distance" of the game AI from the target may be determined based on the distance criteria (e.g., classification threshold, etc.) included in the set of cognitive configuration parameters.

The policy decision module 1103 may be configured to determine a first behavior policy from a plurality of behavior policies of the intelligent virtual character based on the second number of environmental decisions, the determined first behavior policy indicating that the intelligent virtual character has a higher probability of selecting to execute a behavior corresponding to the first behavior policy than other behavior policies of the plurality of behavior policies of the intelligent virtual character.

Alternatively, the policy decision module 1103 may perform the operations of the policy decision layer as described with reference to fig. 2B.

Alternatively, the policy decision module 1103 may generate the behavior policy (behavior tendency) of the game AI according to the parameter configuration of the policy decision layer of the game AI by using the subjective cognitive data output by the environment decision module 1102, so that the game AI with different characters as described above can exhibit different behavior tendencies for a longer time.

Alternatively, the behavior policy determined by the policy decision module 1103 does not directly decide what behavior the game AI must take at the next time, but affects the distribution and proportion of the game AI behavior over a period of time. For example, when determining a behavior policy to select an attack, the game AI does not necessarily perform various behaviors corresponding to the attack, but the probability that the game AI selects to perform a behavior under the attack policy will be significantly raised over a period of time.

The behavior decision module 1104 may be configured to determine a behavior to control execution of the intelligent virtual character based on the second number of environmental decisions and the determined first behavior policy, and execution conditions of at least a portion of all behaviors of the intelligent virtual character.

Alternatively, the behavior decision module 1104 may perform the operations of the behavior decision layer as described with reference to fig. 2B.

Alternatively, the behavior decision module 1104 may use the policy decision result output by the policy decision module 1103 and the subjective cognitive result output by the environment judgment module 1102 to generate a behavior decision result of the game AI according to the parameter configuration thereof, so as to decide what behavior the game AI specifically performs. The behavior decision parameter configuration may include all execution conditions related to all behaviors of the game AI and their relationships to various behavior policies and various behaviors under these behavior policies.

Alternatively, the algorithm for selecting the final execution behavior needs to ensure randomness of the selection, and even in the case where conditions such as the external environment are not changed, the behavior result obtained by each selection should be changed instead of remaining the same.

According to an embodiment of the present disclosure, the intelligent avatar control device 1100 may further include a behavior execution module 1105, which may be configured to control the intelligent avatar to execute a series of actions included in the determined behavior.

Alternatively, the behavior execution module 1105 may perform operations of the execution layer as described with reference to fig. 2B.

Alternatively, the behavior execution module 1105 may be related to any particular behavior that the game AI is capable of exhibiting outward that may be perceived by the player, including, but not limited to, such as moving, turning, changing posture (e.g., standing, squatting, lying on or sideways, etc.), aiming, firing, speaking, etc.

According to still another aspect of the present disclosure, there is also provided an intelligent virtual character control apparatus. Fig. 12 shows a schematic diagram of an intelligent virtual character control apparatus 2000 according to an embodiment of the present disclosure.

As shown in fig. 12, the intelligent virtual character control apparatus 2000 may include one or more processors 2010, and one or more memories 2020. Wherein said memory 2020 has stored therein computer readable code which, when executed by said one or more processors 2010, can perform the intelligent avatar control method as described above.

The processor in embodiments of the present disclosure may be an integrated circuit chip having signal processing capabilities. The processor may be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, and may be of the X86 architecture or ARM architecture.

In general, the various example embodiments of the disclosure may be implemented in hardware or special purpose circuits, software, firmware, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device. While aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that the blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

For example, a method or apparatus according to embodiments of the present disclosure may also be implemented by means of the architecture of computing device 3000 shown in fig. 13. As shown in fig. 13, computing device 3000 may include a bus 3010, one or more CPUs 3020, a Read Only Memory (ROM) 3030, a Random Access Memory (RAM) 3040, a communication port 3050 connected to a network, an input/output component 3060, a hard disk 3070, and the like. A storage device in the computing device 3000, such as a ROM 3030 or hard disk 3070, may store various data or files for processing and/or communication use of the intelligent avatar control method provided by the present disclosure and program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, the architecture shown in FIG. 12 is merely exemplary, and one or more components of the computing device shown in FIG. 13 may be omitted as may be practical in implementing different devices.

According to yet another aspect of the present disclosure, a computer-readable storage medium is also provided. Fig. 14 shows a schematic diagram 4000 of a storage medium according to the present disclosure.

As shown in fig. 14, the computer storage medium 4020 has stored thereon computer readable instructions 4010. The intelligent virtual character control method according to the embodiments of the present disclosure described with reference to the above drawings may be performed when the computer readable instructions 4010 are executed by a processor. The computer readable storage medium in embodiments of the present disclosure may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate synchronous dynamic random access memory (ddr SDRAM), enhanced Synchronous Dynamic Random Access Memory (ESDRAM), synchronous Link Dynamic Random Access Memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that the memory of the methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory. It should be noted that the memory of the methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

Embodiments of the present disclosure also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from a computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs an intelligent virtual character control method according to an embodiment of the present disclosure.

Compared with the existing virtual character design method, the method provided by the embodiment of the disclosure establishes an AI design framework comprising five layers, standardizes definition and design points of each layer, and designs the AI with high anthropomorphic and high game level based on the AI design framework.

It is noted that the flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises at least one executable instruction for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The exemplary embodiments of the present disclosure described in detail above are illustrative only and are not limiting. Those skilled in the art will understand that various modifications and combinations of these embodiments or features thereof may be made without departing from the principles and spirit of the disclosure, and such modifications should fall within the scope of the disclosure.

Claims

1. An intelligent virtual character control method, comprising:

acquiring a first number of environment detection data in the vicinity of the intelligent virtual character;

generating a second number of environmental judgment results based on the first number of environmental detection data, the second number being not greater than the first number;

determining a first behavior policy from a plurality of behavior policies of the intelligent virtual character based on the second number of environmental decisions, the determined first behavior policy indicating that the intelligent virtual character has a higher probability of selecting to execute a behavior corresponding to the first behavior policy than other behavior policies of the plurality of behavior policies of the intelligent virtual character; and

and determining a behavior to be controlled to be executed by the intelligent virtual character based on the second number of environment judgment results and the determined first behavior strategy and execution conditions of at least a part of all behaviors of the intelligent virtual character.

2. The method of claim 1, wherein determining a first behavior policy from a plurality of behavior policies of the intelligent avatar based on the second number of environmental decisions comprises:

determining the probability of the intelligent virtual character executing each of the plurality of behavior strategies based on the second number of environment judgment results and preset strategy decision configuration parameters, wherein the strategy decision configuration parameters indicate the correlation between the environment judgment results and the behavior strategies; and

randomly selecting one of the plurality of behavior policies as the first behavior policy based on the determined probability that the intelligent virtual character performs each of the plurality of behavior policies;

wherein each of the plurality of behavior policies of the intelligent virtual character corresponds to at least one behavior, and each of all behaviors of the intelligent virtual character corresponds to at least one behavior policy.

3. The method of claim 1, wherein the execution conditions for each of all behaviors of the intelligent avatar include a mandatory execution condition and an optional execution condition;

wherein determining, based on the second number of environmental decisions and the determined first behavior policy, and execution conditions of at least a portion of all behaviors of the intelligent virtual character, a behavior to control execution of the intelligent virtual character includes:

Determining at least one execution condition which is met by the intelligent virtual character and at least one action corresponding to the at least one execution condition based on the second number of environment judgment results and the execution conditions of at least one part of all actions of the intelligent virtual character; and

and under the condition that one or more than one alternative execution condition exists in at least one execution condition met by the determined intelligent virtual character, selecting an alternative execution condition with the highest priority among the one or more than one alternative execution condition based on the preset priority of the one or more than one alternative execution condition, and taking the action corresponding to the alternative execution condition as the action to be controlled to execute the intelligent virtual character.

4. The method of claim 3, wherein determining an action to control execution of the intelligent avatar based on the second number of environmental decisions and the determined first action policy, and execution conditions of at least a portion of all actions of the intelligent avatar, further comprises:

determining a probability of the intelligent virtual character executing each of the at least one behavior based on the determined first behavior policy and the at least one behavior corresponding to the at least one execution condition, in the case that one or more alternative execution conditions do not exist in the determined at least one execution condition satisfied by the intelligent virtual character; and

An action to control execution of the intelligent virtual character is randomly selected based on the determined probability of execution of each of the at least one action by the intelligent virtual character.

5. The method of claim 4, wherein determining the probability that the intelligent avatar performs each of the at least one behavior based on the determined first behavior policy and at least one behavior corresponding to the at least one execution condition comprises:

in the case that one or more behaviors corresponding to the determined first behavior policy exist in at least one behavior corresponding to the at least one execution condition, increasing a probability that the intelligent virtual character executes each of the one or more behaviors.

6. The method of claim 5, wherein the at least one behavior corresponding to the at least one execution condition constitutes a pool of behaviors, and determining the probability that the intelligent virtual character will perform each of the at least one behavior based on the determined first behavior policy and the at least one behavior corresponding to the at least one execution condition further comprises at least one of:

increasing the probability of the intelligent avatar executing each of the one or more behaviors in the currently controlled pool of behaviors that do not belong to the previously controlled pool of behaviors; or (b)

In the case where there are one or more behaviors in the currently controlled behavior pool that are different from the behaviors determined by the previous control that control the intelligent avatar to perform, the probability that the intelligent avatar performs each of the one or more behaviors is increased.

7. The method of claim 2 or 6, wherein the probability of the intelligent avatar determined in the current control to execute each of the plurality of behavior policies is based on an update to the probability of the intelligent avatar determined in the previous control to execute each of the respective plurality of behavior policies, the update being based on one or more of the following time intervals:

a predetermined time interval; or (b)

A time interval between a time when a predetermined update condition is satisfied and a time in previous control when a probability of the intelligent avatar executing each of a corresponding plurality of behavior policies is determined;

wherein the current control and the previous control correspond to a current control of the intelligent virtual character and a last control of the intelligent virtual character, respectively.

8. The method of claim 3, wherein determining an action to control execution of the intelligent avatar based on the second number of environmental decisions and the determined first action policy, and execution conditions of at least a portion of all actions of the intelligent avatar, further comprises:

And executing preset default behaviors under the condition that at least one execution condition met by the intelligent virtual roles does not exist.

9. The method of claim 1, wherein generating a second number of environmental decisions based on the first number of environmental detection data comprises:

classifying the first number of environment detection data according to a predetermined cognitive configuration parameter set to generate the second number of environment judgment results;

wherein the set of cognitive configuration parameters is set for the intelligent virtual character, one parameter of the set of cognitive configuration parameters corresponding to one behavior of the intelligent virtual character.

10. The method of claim 1, wherein each of all of the behaviors of the intelligent avatar comprises a combination of a series of actions, the method further comprising:

controlling the intelligent virtual character to execute a series of actions included in the determined behavior;

wherein during control of the intelligent avatar to perform the series of actions, at least one of:

immediately ending execution of the series of actions in a case where a predetermined ending condition is satisfied, the predetermined ending condition being set to a different condition based on success or failure of the series of actions;

Immediately ending the execution of the series of actions and switching to the execution of a series of actions comprised by another action in case a predetermined interrupt condition is fulfilled; or (b)

In a case where the execution time of the series of actions exceeds the execution time set in advance and the predetermined end condition or the predetermined interrupt condition is not satisfied, the execution of the series of actions is immediately ended.

11. An intelligent virtual character control apparatus, comprising:

an environment detection module configured to obtain a first amount of environment detection data in proximity to the intelligent virtual character;

an environment determination module configured to generate a second number of environment determination results based on the first number of environment detection data, the second number being not greater than the first number;

a policy decision module configured to determine a first behavior policy from a plurality of behavior policies of the intelligent virtual character based on the second number of environmental decisions, the determined first behavior policy indicating that the intelligent virtual character has a higher probability of selecting to perform a behavior corresponding to the first behavior policy than other behavior policies of the plurality of behavior policies of the intelligent virtual character; and

And a behavior decision module configured to determine a behavior to control execution of the intelligent virtual character based on the second number of environmental judgment results and the determined first behavior policy, and an execution condition of at least a part of all behaviors of the intelligent virtual character.

12. The apparatus of claim 11, wherein determining a first behavior policy from a plurality of behavior policies of the intelligent avatar based on the second number of environmental decisions comprises:

13. The apparatus of claim 11, wherein the execution condition of each of all behaviors of the intelligent avatar includes a mandatory execution condition and an optional execution condition;

14. The apparatus of claim 13, wherein determining an action to control execution of the intelligent avatar based on the second number of environmental decisions and the determined first action policy, and execution conditions of at least a portion of all actions of the intelligent avatar, further comprises:

15. The apparatus of claim 11, wherein each of all of the behaviors of the intelligent avatar comprises a combination of a series of actions, the apparatus further comprising:

a behavior execution module configured to control the intelligent virtual character to execute a series of actions included in the determined behavior;

16. An intelligent virtual character control apparatus, comprising:

one or more processors; and

one or more memories in which a computer executable program is stored which, when executed by the processor, performs the method of any of claims 1-10.

17. A computer program product comprising computer instructions which, when executed by a processor, cause a computer device to perform the method of any of claims 1-10.

18. A computer readable storage medium having stored thereon computer executable instructions for implementing the method of any of claims 1-10 when executed by a processor.