GB2482153A

GB2482153A - Virtual or smart environment behaviour profile generation

Info

Publication number: GB2482153A
Application number: GB1012243.0A
Authority: GB
Inventors: Marc Iain Davies
Original assignee: Individual
Current assignee: Individual
Priority date: 2010-07-21
Filing date: 2010-07-21
Publication date: 2012-01-25
Anticipated expiration: 2030-07-21
Also published as: GB2482153B; GB201012243D0

Abstract

Generating artificial behaviour profiles for use in a virtual reality 11 or smart environment 12 to approximate human behaviour includes monitoring and recording actions and decisions 20 of a human subject. A human subject behaviour profile 21 comprising a list of the recorded actions and corresponding decisions is generated and evolutionary operations 30 are performed on the human subject behaviour profile 21 using an algorithm-based methodology. An artificial behaviour profile 48 is then generated from the algorithm-based methodology, which can be used for example in combat games to control fighting characters, crowd environments such as shopping malls or in real world smart environments.

Description

ArtficaI Behaviour This invention relates to the generation of artificial behaviour profiles based on and approximating human behaviour.

The invention has been developed primarily for use in virtual reality environments to overcome limitations in the realism of computer controlled characters in computer games, but the invention also finds use in non-virtual applications. Such non-virtual applications include but are not limited to intelligent buildings, smart devices, home-security systems and robotics.

Artificial inteUgence is a term used to define the ability for a machine to perform cognitive tasks. An artificial intelligence system generay requires io three elements, the ability to store knowledge, the ability to apply the stored knowledge to solve problems, and the ability to learn -acquire new knowledge through experience.

Virtual reality computer games often feature numerous characters which are automatically controlled by a pre-defined computer program. These artificial computer-based characters, referred to hereinafter as non-player characters, can have a variety of roles, such as providing information to advance a storyline, acting as a retailer from whom items can be purchased, or simply existing in the background to make the virtual world seem more realistic.

A non-player character need not necessarily be a virtual human figure and may comprise any active virtual entity such as a vehicle in a racing game, animals, aliens or robots.

The level of artificial intelligence afforded to a non-player character is often dependent on the purpose the non-player character serves in the game and the importance of the non-player character. In most cases the level of complexity of the artifidal intelligence will be proportional to the level of contact it has with human players.

While it is clearly desirable to have a game appear as realistic as possible to a player, there are operational drawbacks in creating such diversity, resulting in a compromise between realism and operability. In particular, the game must be: i) operational on the hardware platform for which it is designed, such as on games consoles or on desktop computers; ii) a suitable size so as to fit onto the storage medium used by that platform, for example a CD/DVD!Blu-.Ray discs or cartridges; and iii) developed within the scope of a given timeframe and financial budget.

As modern computer games are also expected to be visually impressive, with high definition graphics and large virtual worlds, the quality of the artificial inteUigence of non-player characters can suffer as part of a resources trade-off.

To address the above problems, the non-player characters generated in some games function using a common mechanism operating with a looped architecture. Whilst overcoming the quality drawbacks, this approach often results in repetitive functionality which is easily noticeable by a human player, for example, where an enemy soldier non-player character fights in exactly the same way as every other enemy soldier throughout the game. Such limitations can often make a game predictable and boring to a player as their sense of realism is reduced. This is especially the case if non-player characters are enemies to be fought, as once the player identifies the pattern of actions used by the artificial intelUgence, it is often possible to defeat every similar enemy in exactly the same way, eliminating the challenge and the sense of achievement in completing the game.

Another method utilised in computer games to enable non-player character actions to appear more realistic is to allow them to cheat. For example, in order to allow a non-player character to challenge a human player effectively in a game where the non-p'ayer character needs to be competitive, it cou'd be allocated extra resources or better attributes. An example of this is io where the game includes a combat scenario and the non-player character is provided with improved weapons or support units, increased speed or the ability to inflict higher damage levels. This method suffers the same limitations of repeatability, again leading to predictability. Even with extra resources or better attributes, the artificial intelligence of the non-player character remains dependant on the ability to strategise and counter moves made by human players. If this ability is acking, players can quicky discover how to overcome any advantage given to the non-player character. This strategy can often be repeated by the player in the future to overcome any other similar enemy non-player character.

Artificial intelligence may also be utilised in non-virtual applications such as in the research and development of smart devices and intelligent buildings.

Smart devices are devices which are considered intelligent due to their ability to interact with humans and other devices, such as mobile phones, actuators and sensors. An intelligent building is essentially a smart environment where smart devices are employed to improve a user's existence, essentiaUy to make life easier. Research and development of smart environments requires human test subjects to interact with the smart devices in the environment, with many projects involving smart environments often running continuously for months or years. This is not only time consuming but the cost of employing test subjects is not economical. For the purposes of clarification, all non-virtual applications described herein will be referred to hereinafter as "smart environments". These also include but are not limited to home-security systems and robotics.

It is a principal aim of the present invention to provide a method of io generating artificial behaviours which approximate human behaviour and which can be used to simulate non-player characters in virtual reality environments and humans in real ife applications, such as in smart environments.

According to this invention, there is provided a method of generating artificial behaviour profiles for use in at least one of a virtual reality or a smart is environment to approximate human behaviour, comprising the following steps: a) monitoring and recording a selection of actions and decisions of a human subject in at least one of a virtual reality and a smart environment; b) generating a human subject behaviour profile comprising a list of the recorded actions and corresponding decisions; c) performing evolutionary operations on the human subject behaviour profile using an algorithm-based methodology; and d) generating an artificial behaviour profile from the algorithm-based methodology.

The generation of artificial behaviour prothes based on the actions and decisions of human subjects ensures that the actions and decisions of an artificial behaviour profile are natural and possess a human-like persona.

The method monitors how individual users interact with the system to which the method is incorporated and generates a human behaviour profile.

The algorithm-based methodology selects a human behaviour profile and may use this as a benchmark for evaluating and ranking the artificial behaviour profiles which are generated. Typically one human subject behaviour profile is selected and used to evolve new artificial behaviour profiles, though artificial io behaviour profiles may be generated using more than one human subject behaviour profile. Ideally, the method is customisable to enable the production of any number of artificial behaviour profiles according to the needs of the application.

Preferably, each action and decision in the list comprises a sequence of is instructions for interpretation and execution by a processor. In this way, a processor may simply process the instructions in order to operate in the desired fashion.

The evolutionary operations of the algorithm-based methodology may be carried out by any suitable means but are ideally performed by a genetic program. Genetic programs are based on biological processes and comprise fragments of program code often represented in tree-like structures. Genetic programs create further generations of programs using the principles of genetics. Accordingly, the evolutionary operations performed by the genetic program may comprise an iterative process whereby the human subject behaviour profile or a resultant artificial behaviour profile is modified by one or more of the operations of crossover, mutation, and reproduction.

Preferably, the human subject behaviour profile is stored In a mimicry profile storage medium. The method may run constanily so that new human 6 subject behaviour profiles are created whenever possible and are added to the mimicry profile storage medium for selection by the genetic program when requked.

The artificial behaviour profile may be stored in an artificial profile storage medium, if required by the particular application. In this way, the artIficial profile storage medium may be optional depending on the application in whIch the method Is used.

in a preferred embodiment of the invention one or more artificial behaviour profiles stored In the artificial profile storage medium are assigned to an artificial computer-based character In a virtual reality game. In thIs 16 applicatIon, the method may operate discretely In the background as part of the program of a computer game, allowing the generation of artificial behaviour profiles which approximate human behaviour. ThIs method Is versatile and may be customised to suit video games from a variety of different genres such as role-playing games (RPG), first-person shooter (FPS) and turn-basedlreal-time strategy games.

While the method of the present invention finds particular use with computer games of all types It would be most effective with online games and In particular with massive multi-user online (MMO) environments due to the potential'y large numbers of human players, from which behaviour profiles can be obtained using the variety of human subject behaviour profiles.

The use of the method for computer games may be especially beneficial for scenes in a game involving large crowds of non-player characters, for example; a shopping mall with many people. In such an event, to make the scene appear realistic, it is preferable that the non-player characters react with the environment as a human would. In this way the method of the present invention allows the generation of artificial behaviour profUes which may be assigned to a non-p'ayer character in order to behave in a human-like fashion.

The artificial behaviour profile, when generated, may be automatically assigned to an artificial computer-based character in a virtual reality game if required. The artificial behaviour profile assigned to the artificial computer-based character may be automatically updated when an alternative artificial behaviour profile with advanced behaviour characteristics becomes avaUable.

n an alternative embodiment of the invention, the method may be incorporated in a smart environment whereby one or more artificial behaviour profiles may be used to interact and control devices in the smart environment.

An artificial behaviour profile may be selected from the artifidal profile storage medium or may automatically be used to interact and control devices in a smart envftonment as soon as the artificial behaviour profile is generated. This application could be particularly useful in providing an alternative to using human test subjects in the research and development of intelligent buildings and devices. In this method, once at least one human subject has interacted with the smart environment, the method can generate many artificia' behaviour prothes based on the actions and decisions made by the human subject. The artificial profiles may then be loaded into the computer systems to which the smart environment is connected and control the environment as if it were being used by a real human subject. This method would overcome the research issues such as cost of employing and the availability of suitable test subjects for long term experiments.

n another embodiment, the human subject behaviour profile may comprise a combination of the recorded actions and corresponding decisions of a human subject in both a virtual reality and a smart environment. In such a io mixed reality smart environment one or more physical smart environment is combined with one or more virtual reality environment. The two environments need not be similar in appearance, it is possible for content to exist in one but not the other. In such an embodiment one or more generated artificial behaviour profile may be assigned to a non-player character in a virtual reality is game and may interact and control devices in a smart environment. Some smart devices may exist in both the smart environment and the virtual reality environment and are linked such that if either has their state changed, the action would be invoked on both the smart and virtual reality environments. In this way, it may be possible for both player characters and non-player characters in a virtual reality environment to interact with real devices in the physical word and vice versa.

According to a second embodiment of this invention there is provided a product comprising a sequence of instructions for interpretation and execution by a processor to generate artificial behaviour profiles for use in at least one of a virtua' reaty and a smart environment to approximate human behaviour, according to the methods previously described.

By way of examp'e on'y, one specific method for generating artificial behaviour proffles and appcations ncorporating such methods wifi now be described in detaN, reference being made to the accompanying drawings in which:-Figure 1 is a schematic flow-chart diagram iustrating the main steps of the method of the present nvention; Figure 2 is a schematic flow-chart diagram i'lustrating the steps of the io mimicry system; Figure 3 is a schematic flow-chart diagram iUustrating the steps of the genetic program; Figure 4 is a schematic flow-chart diagram iustrating further steps of the genetic program for ranking artificial profiles; is Figure 5 is a schematic flow-chart diagram illustrating an artificial profile structure; Figure 6 is a schematic flow-chart diagram i'lustrating the integration of the method of this invention with a computer game; Figure 7 s a schematic flow-chart diagram ifiustrating the use of the method of Figure 6 for an artificial behaviour profile for a background non-player character; Figure 8 s a schematic flow-chart diagram iUustrating the use of the method of Figure 6 for an artificia' behaviour profi'e for an enemy non-p'ayer character; Figure 9 is a schematic flow-chart diagram illustrating the integration of the method of this invention with a non-virtual smart environment; Figure 10 is a schematic flow-chart diagram illustrating the integration of the method of this invention with a virtual smart environment; and Figure 11 is a schematic flow-chart diagram illustrating the integration of the method of this invention with a mixed reality smart and virtual environments; Referring initially to Figure 1, the method of the invention generally comprises a series of main steps 10 to be sequentially performed by a processor. The method may be applied to a range of environments, induding virtual environments such as computer games Ii, smart environments 12, mixed smart and virtual environments 13, and virtual smart environments 14, as is discussed in more detail below.

The first step in the method is the mimicry system 20 shown in detail in Figure 2. The mimicry system 20 monitors how individual human subjects is interact with the application 11, 12, 13, 14, 15, to which the method is applied.

The system may operate continuously or when triggered to do so by the requirements of the application. Once started 19, as shown in Figure 2, the mimicry system creates a human behaviour profile 21 and the action and decision inputs of the user to the application are monitored. The system achieves this by iteratively performing the steps of watching for user inputs 22 and noting whether a user input has been detected 23. If a user input has been detected 23, a reference, such as a timestamp, light level, temperature or enemy type is added 24 to the human behaviour profile 21 along with the input 25.

The mimicry system 20 may be customisabe so that the human behaviour profi'es are limited to a set length or timefrarne. Once a user input is added to the human behaviour profile the method assesses whether a full human behaviour profile has been created and whether the exit criteria 26, has been met. Once the exit criteria 26, has been met, the human behaviour profile is added to the mimicry profile storage medium 27 and the mimicry system process ends 28. The human behaviour profile is composed of discrete computer programs comprised of functions and terminals, each program defining a particular user action or decision.

The next main step in the method is the genetic program 30 (Figure 3) which uses the human subject behaviour profiles to evolve similar artificial behaviour profiles. The genetic program 30 refers to the mimicry profile storage medium 27 for a list of possible actions and decisions that can be performed during a specific time period.

The key steps of the genetic program 30 can be seen in Figure 3. At the start 29 of the genetic program 30, a mimicry profile is selected from the mimicry profile storage medium 27 to be used as a benchmark 31 for eva'uating and ranking the artificial behaviour profiles that are generated. The steps performed by the genetic program 30 are typical of most genetic programs and include the generation of an initial random population of artificial behaviour profiles 32 composed of the terminals and functions of the discrete computer programs. The population of artificial behaviour profiles are then ranked 33 to determine the best in view of the benchmark and this process is illustrated in detail in Figure 4.

In a first iteration, each of the artificial behaviour profiles of the population are analysed 35 agaInst the benchmark 31. FIrstly the raw fitness Is determined which is the sum of the differences between each rule in the artificial behaviour profile and its counterpart rule In the benchmark human 6 behaviour profile. The difference is dependent on the context of the reference used in the mimicry system and the artificial behaviour profile rules. For example, for a timestamp reference the difference will be the time between the two rules occurring and for references using specific categories, the difference wilibethenumberoftlmesthereferenceoftworulesmatch. Thedifference valuesarecompensatedforatstep3luntllalioftheruleshavebeenaccessed 38. The standardised fitness values 39 and the adjusted fitness values 40 are then calculated.

Once each of the artificial behaviour profiles of the population have been analysed the artificial behaviour profiles with fewer rules than the benchmark is are removed 41. The adjusted fitness values of the population are then added 42 and the normaiised fitness of the population calcuiated 43 in order to rank the population 44.

Following the ranking process 33 of the genetic program 30, the method assesses whether a predefined exit criteria has been satisfied 47. If It has, the best artificial behaviour profile Is Identified 48 and Is added to the artificial profllestoragemedium49. lftheexltcriteria47isnotsatlsfledarandom genetic operator is applied 50 to the artificial behaviour profile until the limit of the population has been reached 54. The random genetic operator may be one of cloning 51, crossover 52 or mutation 53. Once the population lkft is reached the new population of artificial behaviour prothes are ranked 33 and the process continues unth the exit criteria 47 is satisfied and a suitable artificial behaviour profile generated.

An example of an artificial behaviour profile structure can be seen in Figure 5. The following represent the functions of the profile: The reference may be a timestamp (i.e. day, hour, minute, Reference, 57 second) or some other identifier such as light level. The reference 57 in this example has an arity of two.

This represents an action behaviour, without moving to the Action -1, 58 target device. The action-i function 58 in this example has an arity of two.

This represents an action behaviour, moving to the target Action -2, 59 device. The action-2 function 59 in this example has an arity of three.

This represents a preference behaviour (decision), without Preference-i, 60 moving to the target device. The preference-i function 60 in this example has an arity of three.

This represents a preference behaviour (decision), moving Preference-2, 61 to the target device. The preference-2 function 61 in this example has an arity of three.

The corresponding terminals of the artificial profile structure of Figure 5 are defined as follows: Continue action 63 Continue with the last invoked behaviour.

Perform an action chosen from the range of possibilities Perform action 64 available for the selected device.

More to the selected device. Moving to the device can Move to device 65 present additional possible actions that can be performed.

Switch the selected device on. Preference behaviours Switch device on 66 ensure a device is on before changing settings.

Change a setting chosen from the range of possibilities Change settings 67 available for the selected device.

Figure 6 shows the method of Figure 1 incorporated into a computer game 11. The computer game 11 may be a typical computer game including a virtual world and containing populations of objects and non-player characters 70. Human subjects play the game by controlling a player character 71 using a user interface 72. The user interface 72 defines the actions performed and the decisions made by the human subject controlling the player character 71 This could include features such as selecting options from an in-game menu, movements made during a battle or answering questions posed to the player by io a non-player character 70. The method 10 generates an artificial behaviour profile 48 which is assigned to a non-player character 70. Non-player characters can be assigned one or more artificial profiles 48. Artificial profiles may be generated spontaneously for each non-player character and updated whenever necessary. Alternatively, non-player characters can be assigned one or more artificial behaviour profiles from the artificial profi'e storage medium 49, which contains a list of the best artificial behaviour profiles evolved by the genetic program.

The artificial behaviour profiles generated for use as non-player characters in computer games may take various forms and are wholly dependant on the game and what is required for the particular scenarios in the game. Two examples are shown in Figure 7 and Figure 8 respectively.

Figure 7 demonstrates the use of the invention in a market scenario of a computer game and Figure 8 illustrates the use of the invention in an enemy battle scene. In the market scenario, the process begins when a player character 71 enters the market 73. The method assesses 74 whether there are any non-player characters 70 in the market. If there are not, the method creates a new human behaviour profile by monitoring a human payers actions unti' the p'ayer has left the game and the human behaviour profi'e is stored is in the mimicry profile storage medium 27. In the battle scene, the process begins when the player encounters an enemy non-player character 76. The method assesses whether a similar enemy has already been encountered 77.

If not the method monitors the actions of the player 75 and assigns each action an appropriate reference, such as attack or defence 78. The actions are monitored until the enemy is defeated and then the human behaviour profile is added to the mimicry profile storage medium 27.

n each of the examples of Figure 7 and 8, if there are non-player characters 70 in the market or if the player has encountered a similar enemy, the method either selects an artificial behaviour profile from the artificial profile -16-storage medium 49 and assigns the profile 79 to the non-player character, or selects a human subject behaviour prothe from the mimicry profile storage medium 27 and starts the genetic program 30 to generate a suitable artificial behaviour profile.

Integration of the method 10 in a smart environment 12 can be seen in Figure 9. The smart environment 12 is a real world environment containing a range of physical smart and non-smart devices 83. In a smart environment users can interact with the devices in the world directly, (for example turning on a lamp), indirectly (for example using a remote control or computer menu io screen) or through a combination of both. The structure of the user interface 81 for a smart environment depends on the complexity and purpose of the world. In most cases the primary interface would consist of people directly using physical smart devices. However, there may also be secondary interface options such as a menu that appears on a computer, touch-screen television or is mobile phone. To capture all user interaction within a smart environment, regardless of the method used to interface with devices, in addition to the official user interface 81, the mimicry step 20 looks for changes to device settings made using device wrappers 82 that allow them to be controlled by a computer. Device wrappers are pieces of program code which allow computers to control the smart devices 83, (i.e. change the settings). Various programming architectures can be used, (e.g. Universal Plug and Play, (UPnP)) to create device wrappers 82 for the smart devices 83. Therefore, even if the user interface 81 is not used to interact with one or more smart devices 83 in the environment, any setting changes made will still be recorded.

A similar method to that used to integrate the method with a computer game can be seen in Figure 10. This method however also incorporates virtual smart environments. A virtual smart environment 14 is a virtual world which contains sets of objects, including intelligent devices and computer game characters (induding both player and non-player characters). In this method, the device wrappers 82 can also access the graphical display of the virtual world. This is necessary to allow the graphic of an intelligent device in the virtual world to be updated to reflect state changes, for example turning a light on or off.

A mixed reality smart environment is achieved by combining one or more physical smart environments with one or more virtual smart environments as shown in Figure 11. In a mixed reality smart environment it is possible for characters in the virtual world, (both player characters and non-player characters), to interact with real devices in the physical world and vice versa.

The two environments need not be similar in appearance, (it is possible for content to exist in one world but not in the other). It is possible that some smart devices exist in both worlds, (i.e. a virtual counterpart to a physical object).

Counterparts would share a set of device wrappers so if either has their state changed, (switched on/off, etc.) the action would be invoked on both the physical and virtual device.

When a real person enters the smart environment the method creates a new human subject behaviour profile. As the person moves around the environment, whenever they use a smart device using the user interface(s), the method adds a record to the active human subject behaviour profile. After a predefined exit criteria s met, (i.e. the person leaves the environment, a maximum number of actions are added to the active mimicry profile, or a set time period passes), the method stores the complete human subject behaviour profile in the mimicry profile storage medium 27.

If the method is instructed to use a previously generated artificial behaviour profile it will select one (per non-player character) from the artificial profile storage medium 49 and invoke the stored behaviours from the rule list on the environment. If stored artificial behaviour profiles are not used then the genetic program 30 starts, selecting a human subject behaviour profile from the io mimicry profile storage medium 27 to use as a benchmark, then evolving variations of its behaviours into new artificial behaviour profiles. When the exit conditions of the genetic program 30 are satisfied, (e.g. at least one profile with a fitness above a certain predefined value is generated), it stops and the best profiles are stored in the artificial profile storage medium 49. Each non-player is character in the environment is assigned one of the best profiles generated by the genetic program 30. The behaviour rules for each profile assigned to an non-player character are invoked on the smart environment devices, (i.e. automatically changing the state of smart devices by following the rules contained in the profile).

Claims

CLAIMS1. A method of generating artificial behaviour profiles for use in at least one of a virtual reaty or a smart envftonment to approximate human behaviour, comprising the following steps: a) monitoring and recording a selection of actions and decisions of a human subject in at least one of a virtua' reality and a smart environment; b) generating a human subject behaviour profile comprising a st of the recorded actions and corresponding decisions; C) performing evolutionary operations on the human subject behaviour io profile using an algorithm-based methodology; and d) generating an artificial behaviour profile from the algorithm-based methodology.
2. A method as claimed in any of claim 1, wherein each action and decision in the list comprises a sequence of instructions for interpretation and execution by a processor.
3. A method as claimed in claim 2, wherein the evolutionary operations of the algorithm-based methodology are performed by a genetic program.
4. A method as claimed in daim 3, wherein the evolutionary operations performed by the genetic program comprise an iterative process whereby the human subject behaviour profile or a resultant artificial behaviour profi'e is modified by one or more of the operations of crossover, mutation, and reproduction.

-20 -
5. A method as claimed in any of the preceding claims, wherein the human subject behaviour profile is stored in a mimicry profile storage medium.
6. A method as claimed in any of the preceding claims, wherein the artificial behaviour profile is stored in an artificial profile storage medium.
7. A method as daimed in claim 6, wherein one or more artificial behaviour profiles stored in the artificial profile storage medium are assigned to an artificial computer-based character in a virtual reality game.
8. A method as claimed in any of claims 1 to 6, wherein the behaviour profile when generated is automatically assigned to an artificial computer-based io character in a virtual reality game.
9. A method as claimed in claims 7 or 8, wherein the artificial behaviour profile assigned to the artificial computer-based character is automatically updated when an alternative artificial behaviour profile with advanced behaviour characteristics becomes available.
10. A method as claimed in claim 6, wherein one or more artificial behaviour profiles in the artificial profile storage medium are used to interact and control devices in a smart environment.
11. A method as claimed in any of claims 1 to 6, wherein the artificial behaviour profile when generated is automatically used to interact and control devices in a smart environment.
12. A method as claimed in any of claims 1 to 6, wherein the human subject behaviour profile comprises a combination of the recorded actions and corresponding decisions of a human subject in both a virtual reality and a smart environment.

-21 -
1 3. A method as claimed in claim 12, wherein one or more generated artificial behaviour profiles are assigned to a non-player character in a virtual reality game and are used to interact and control devices in a smart environment.
14. A method as claimed in claim 1 and substantially as hereinbefore described with reference to and as il'ustrated in the accompanying drawings.
15. A product comprising a sequence of instructions for interpretation and execution by a processor to generate artificial behaviour profiles for use in at least one of a virtual reality and a smart environment to approximate human behaviour, according to a method as claimed in any of claims I to 14.
16. A product as claimed in claim I 5 and substantial'y as hereinbefore described with reference to and as il'ustrated in the accompanying drawings.