JP7413628B2 - Abstraction interface for gamification of machine learning algorithms - Google Patents

Description

TECHNICAL FIELD This disclosure generally relates to processing machine learning (ML) algorithms with user interaction. More specifically, the present disclosure relates to abstraction interfaces for gamification of machine learning algorithms.

Many researchers are turning to games and game-like activities and approaching research through crowdsourced research or citizen science (CS), also known as human computing (HC). CS research technology allows researchers to access a vast potential network of volunteer citizen scientists interested in supporting scientific research. CS games such as Foldit® turn scientific tasks into games. Many CS games have been proven to successfully generate interest, provide valuable data analysis, and generate creative solutions to scientific problems. Despite the potential of these CS games, CS games have not found success and engagement on the level of commercial video games.

Embodiments of the present disclosure provide an abstracted interface for interactive machine learning, gamification of machine learning algorithms, and/or constructivist augmented machine learning.

In one embodiment, a method of interactive machine learning is provided. This method is based on the steps of identifying solutions to a given problem and determining that the identified solutions are possible solutions to the given problem. processing possible solutions by executing; selecting a processed solution based on a plurality of processed solutions at least in part from different iterations of the process; and selecting a processed solution based on a termination condition. and determining whether to terminate the process using the selected solution to the given problem. Executing the process includes receiving input to the interactive application for use in performing at least a portion of the steps via a user interface for the interactive application.

In another embodiment, a system for interactive machine learning is provided. The system includes a processor and a communication interface operably connected to the processor. The processor executes a machine learning application using the possible solutions based on identifying solutions to the given problem and determining that the identified solutions are possible solutions to the given problem. selecting a processed solution based on a plurality of processed solutions at least in part from different iterations of the process; and selecting a processed solution based on a termination condition. determining whether to terminate the process using the selected solution to the problem. The communication interface is configured to receive input to the interactive application for use in performing at least a portion of the steps via a user interface for the interactive application.

In another embodiment, a non-transitory computer-readable medium for interactive machine learning is provided. The computer-readable medium, when executed by a processor of the system, directs the system to the steps of: identifying a solution to a given problem; and determining that the identified solution is a possible solution to the given problem. processing the possible solutions by running a machine learning application using the possible solutions and selecting the processed solution based on at least a portion of the processed solutions from different iterations of the process; and determining whether to terminate the process using the selected solution for a given problem based on a termination condition. The computer-readable medium includes program code that, when executed by a processor of the system, causes the system to receive input to an interactive application for use in performing at least a portion of the steps, via a user interface for the interactive application. further including.

1 illustrates an example of a networked system in which various embodiments of the present disclosure may be implemented. 1 illustrates an example of a server that may implement various embodiments of the present disclosure. 1 illustrates an example of a client device on which various embodiments of the present disclosure may be implemented. 2 illustrates a process for implementing an abstract algorithm interface that enables gamification of machine learning algorithms, according to various embodiments of the present disclosure. 2 illustrates an example implementation of a machine learning algorithm within a model of a gameplay loop according to various embodiments of the present disclosure. 4 illustrates a process for dynamically scaling gameplay and using algorithmic performance to adjust game levels in accordance with various embodiments of the present disclosure. 2 illustrates an example graph of machine learning algorithm performance over time used to dynamically scale gameplay and adjust game levels in accordance with one or more embodiments of the present disclosure.

For a more complete understanding of the present disclosure and its advantages, refer to the following description in conjunction with the accompanying drawings, in which like reference numbers represent like parts.

Other technical features will be readily apparent to those skilled in the art from the following drawings, description, and claims.

1-7 discussed below and the various embodiments used to explain the principles of the present disclosure in this patent document are for illustrative purposes only and are not intended to limit the scope of the present disclosure. It should never be construed. Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably arranged system or device.

Various embodiments of the present disclosure demonstrate that having researchers attempt to become experts in video game design and development, or creating customized games for each scientific problem, can be costly. Recognize that there is. Various embodiments of the present disclosure also recognize that despite the ability of some CS games to solve scientific problems, these games lack the engagement of mainstream video games. At low engagement, the usefulness of custom-generated CS games does not reach its full potential.

Various embodiments of the present disclosure provide solutions for connecting scientific research to existing and/or successful commercial video games. Embodiments of the present disclosure provide a mechanism for researchers to utilize mainstream video games in a more general format. In particular, various embodiments of the present disclosure discuss how video games (or more generally interactive entertainment), machine learning, and problems (e.g., machine learning problems, scientific problems related to research and development) Provides a framework for interfaces.

Embodiments of the present disclosure also utilize crowdsourcing or CS techniques to improve machine learning and provide interfaces for interacting with machine learning. A specific example of current disclosure would be to give large online communities, like the gaming community, the time they already spend playing games to help them deal with important issues that help create a better world. Provide motivation and empowerment to take advantage of.

As used herein, the use of the terms "video game" or "game" is intended as an example, and embodiments of the present disclosure may be implemented in any type of interactive application. Also, as used herein, problems are handled or solved using machine learning or computer processing, including improving or modifying machine learning models themselves, as well as scientific problems related to research and development. It can point to any type of problem.

FIG. 1 depicts an example networked system 100 in which various embodiments of the present disclosure may be implemented. The embodiment of networked system 100 shown in FIG. 1 is for illustrative purposes only. Other embodiments of networked system 100 may be used without departing from the scope of this disclosure.

As shown in FIG. 1, system 100 includes a network 101 that facilitates communication between various components within system 100. For example, network 101 may communicate Internet Protocol (IP) packets or other information between network addresses. Network 101 may include all or a portion of one or more local area networks, metropolitan area networks, wide area networks, virtual private networks, global networks such as the Internet, or other communication systems or systems located at one or more locations. can include systems.

Network 101 facilitates communication between various servers 102-104 and various client devices 106-114. Each of servers 102-104 may be any suitable electronic computing or processing device capable of providing computing services, including software, for one or more client devices 106-114. Each of servers 102 - 104 may include, for example, one or more processing units, one or more memories for storing instructions and data, and one or more network interfaces to facilitate communication over network 101 . can. For example, as discussed in more detail below, server 102 provides an abstracted interface for gaming of machine learning algorithms implemented using one or more client devices 106-114. Server 103 may be a software or game development classified server that integrates abstraction interfaces to utilize crowdsourcing techniques to improve machine learning algorithms, as discussed in more detail below. . Server 104 may be a server associated with a researcher, developer, or other grid computing consumer who has a problem that needs to be solved and/or a project or task that needs to be processed using an abstracted interface. You can. Additionally, in some embodiments, any of the servers 102-104 may receive, store, and/or analyze metrics regarding machine learning algorithm performance to evaluate algorithm performance. Alternatively, the results of machine learning algorithm solutions can be provided to researchers, as described in more detail below.

Each client device 106 - 114 represents any suitable electronic computing or processing device that interacts with at least one server or other computing device over network 101 . In this example, client devices 106-114 include a desktop computer 106, a mobile phone or smart phone 108, a tablet computer 110, a laptop computer 112, a video game console 114, a set-top box and/or a television, and the like. However, any other or additional client devices may be used in networked system 100. For example, one of the client devices 106-114 of the system 100 can be any Internet or network-enabled device or Internet of Things (IoT) device (e.g., a Smart TV, a refrigerator, a Raspberry PI, etc.). Can be done. As discussed below, in various embodiments, client devices 106-114 participate, under the coordination of server 102, to form a volunteer computing grid (perhaps with other computing devices) and perform machine learning. Implement abstraction interfaces for algorithm gamification. For example, as described herein, client devices 106-114 may run various video games or interactive applications that include the ability to run one or more machine learning algorithms on abstract matter. . Further, in various embodiments, any of the client devices 106-114 may generate, transmit, and/or analyze metrics regarding machine learning algorithm performance to evaluate algorithm performance and/or , as described in more detail below, can provide researchers with the results of machine learning algorithm solutions. In some embodiments, individual client devices 106 - 1144 communicate directly or indirectly with, e.g., a centralized server, e.g., using peer-to-peer, ad hoc, and/or mesh-based networks. They can communicate with each other or with servers to implement abstracted interfaces. Further examples of platforms for computing grids for collaborative processing of computing tasks include Platform for Collaborative Processing of Computing, filed June 5, 2018, which is incorporated herein by reference in its entirety. No. 16/000,589 entitled ``Tasks''.

Although FIG. 1 shows an example of a networked system 100, various modifications can be made to FIG. For example, system 100 may include any number of each of the components in any suitable configuration, and each of servers 102 - 104 and client devices 106 - 114 may include any number of components that are part of system 100 . It can represent a server and/or a client device. Computing and communication systems generally come in a wide variety of configurations, and FIG. 1 does not limit the scope of this disclosure to any particular configuration. Although FIG. 1 depicts one operating environment in which the various features disclosed herein may be used, these features may be used in any other suitable system.

2 and 3 illustrate example computing devices in a networked system according to various embodiments of the present disclosure. Specifically, FIG. 2 depicts an example server 200 and FIG. 3 depicts an example client device 300. In this illustrative example, server 200 represents any one of servers 102-104 of FIG. 1, and client device 300 represents one or more of client devices 106-114 of FIG. be able to.

As shown in FIG. 2, server 200 includes a bus system 205 that supports communications between processor 210, storage 215, communication interface 220, and input/output (I/O) units 225. Processor 210 executes instructions that can be loaded into memory 230. Processor 210 may include any suitable number and type of processors or other devices in any suitable configuration. Examples of types of processors 210 include microprocessors, microcontrollers, digital signal processors, field programmable gate arrays, application specific integrated circuits, and discrete circuits.

Memory 230 and persistent storage 235 may include any structure that can temporarily or permanently store and facilitate retrieval of information (such as data, program code, and/or other suitable information). or a plurality of storage devices 215. Memory 230 may represent random access memory or any other suitable volatile or non-volatile storage. Persistent storage 235 may include one or more components or devices that support long-term storage of data, such as read-only memory, hard drives, flash memory, or optical disks. For example, persistent storage 235 may store one or more databases of data, client applications, tasks awaiting shipment to cells within the volunteer computing grid, and/or processing of results from the volunteer computing grid, etc. Can be done. In various embodiments, persistent storage 235 stores a video game having interfaced machine learning algorithms or video game components for integrating machine learning algorithms within the game for distribution over network 101. can do.

Communication interface 220 supports communication with other systems or devices. For example, communication interface 220 may include a network interface card or a wireless transceiver that facilitates communication over network 101. Communication interface 220 may support communication via any suitable physical or wireless communication link. I/O unit 225 enables data input and output. For example, I/O unit 225 may provide connections for user input via a keyboard, mouse, keypad, touch screen, or other suitable input device. I/O unit 225 may also send output to a display, printer, or other suitable output device.

Although FIG. 2 shows an example of the server 200, various modifications can be made to FIG. 2. For example, various components of FIG. 2 can be combined, further subdivided, or omitted, and additional components can be added depending on particular needs. As a particular example, although depicted as one system, server 200 may include multiple server systems that may be remotely located. In another example, different server systems may have some or more processing, storage, and/or communication resources to provide an abstraction interface for gamification of machine learning algorithms in accordance with various embodiments of the present disclosure. can provide everything.

FIG. 3 illustrates an example client device 300 according to an embodiment of the present disclosure. The embodiment of client device 300 shown in FIG. 3 is for illustration only, and client devices 106-114 of FIG. 1 may have the same or similar configuration. However, client devices may be provided in a wide variety of configurations, and FIG. 3 does not limit the scope of this disclosure to any particular implementation of an electronic device. As shown in FIG. 3, client device 300 includes a communications interface 305, a processor 310, an input/output (I/O) interface 315, an input 325, a display 320, and a memory 330. Memory 330 includes an operating system 332 and one or more client applications 334.

Communication interface or circuit 305 supports communication with other systems or devices. For example, communication interface 305 may include a network interface card or a wireless transceiver that facilitates communication over network 101. Communication interface 305 may support communication via any suitable physical or wireless communication link. For embodiments that utilize wireless communications, the communication interface 305 connects one or more antennas using various wireless communication protocols (e.g., Bluetooth, Wi-Fi, cellular, LTE communication protocols, etc.). Incoming RF signals can be received via the RF signal.

Processor 310 may include one or more processors or other processing devices and execute an OS 332 stored in memory 330 to control overall operation of client device 300. Processor 310 may also execute client applications 334 that reside in memory 330, such as one or more client applications for processing and storage of tasks, as part of a volunteer computing grid. In various embodiments, the processor 310 runs a client application 334 residing in memory 330, such as one containing an abstracted machine learning algorithm for providing a solution to a problem, as described in more detail below. Run more video games. Processor 310 may include any suitable number and type of processors or other devices in any suitable configuration. Examples of types of processor 310 include microprocessors, microcontrollers, graphics processing units (GPUs), digital signal processors, field programmable gate arrays, application specific integrated circuits, and discrete circuits.

Processor 310 may move data into or out of memory 330 as required by executing processes. Processor 310 is also coupled to an I/O interface 315, which provides client device 300 with the ability to connect to other devices such as a laptop computer or handheld computer. I/O interface 315 provides a communication path between accessories and processor 310.

Processor 310 is also coupled to input 325 and display 320. An operator of client device 300 may use input 325 to enter data into client device 300 . For example, input 325 may be a touch screen, buttons, keyboard, trackball, mouse, stylus, electronic pen, video game controller, etc. Display 320 may be a liquid crystal display, a light emitting diode display, or other display capable of rendering text and/or at least limited graphics, such as from a website. Memory 330 is coupled to processor 310. A portion of memory 330 may include random access memory and another portion of memory 330 may include flash memory or other read-only memory.

Although FIG. 3 shows an example of the client device 300, various modifications can be made to FIG. 3. For example, various components of FIG. 3 may be combined, further subdivided, or omitted, and additional components may be added depending on particular needs. As a particular example, processor 310 may be divided into multiple processors, such as one or more central processing units and one or more graphics processing units (GPUs). In another example, display 320 may be connected as an external device to or not be part of client device 300, such as a video game console or desktop computer, for example.

As discussed herein, various embodiments of the invention utilize components of system 100 to provide a framework for an interface between video games, machine learning, and solutions to problems. Abstraction interfaces according to various embodiments connect machine learning algorithms that are executed within a game, allow players to interact with the algorithms, and make algorithm connections independent of specific algorithms or techniques. For example, a machine learning algorithm can be abstracted within a game, and a machine learning algorithm can be created using another piece of the algorithm that is input through the game interface, or a completely new algorithm (e.g., for different scientific problems for different researchers). ) and still fit within the same game.

Various embodiments recognize that traditional game developers already know what the solution to their video game is when programming the game. However, in machine learning, various embodiments allow researchers to have no idea what the correct solution to their problem is, or even if a correct solution exists. I realize that there are many. Accordingly, various embodiments of the present disclosure enable users to play already popular and/or engaging games. These games solve problems where reaching the end goal is known, keeping it fun and also integrating machine learning algorithms that may not have a final solution or a guarantee of a good solution. Accordingly, various embodiments of the present disclosure provide an abstracted interface for playing games that engage users with little or no algorithmic invasiveness while preserving game fun and engagement. .

Various embodiments of the present disclosure further recognize and exploit the understanding that positive results for a solution can arise simply by requiring the ability to characterize how an algorithm performs. Solutions provided by integrated machine learning algorithms may still be insufficient even after in-game processing. However, if an algorithmic solution can be characterized, various embodiments of the present disclosure can compare the algorithmic characterization to determine whether the solution provided by the algorithm is an improvement. As long as there is at least a small amount of improvement, the ability of abstracted interfaces to perform within highly popular games can significantly improve the overall amount of improvement in machine learning algorithms and the final solution to the problem being studied. increase

In some embodiments, citizen science or crowd-based research is utilized in accordance with one or more aspects of the machine learning algorithm to provide further improvements to the solutions provided by the machine learning algorithm. For example, by analyzing CS projects, CS games, and game design techniques, embodiments of the present disclosure provide a structure for researchers to break down problems into a consistent format that can be included within a video game. . An accompanying application programming interface (API) provides a common interface that allows game developers and the modding community (eg, developers) to access CS data and accompanying support functionality. This structure allows developers to apply their design expertise to more naturally integrate CS into their games, which provides more engagement and expands the user base. The abstracted interfaces of various embodiments allow scientists to focus on their domain knowledge and expertise, while leveraging the skills of game developers to generate new interfaces and improve gameplay. Both provide a means for working on integrations and can leverage the added creativity of crowd-sourced community solutions for games that support the modding community. Additionally, the integrated solution allows CS researchers to utilize CS research after the game and/or interface is released while new data and parameters are being uploaded.

FIG. 4 illustrates a process 400 for implementing an abstract algorithm interface that enables gamification of machine learning algorithms, according to various embodiments of the present disclosure. For example, the process illustrated in FIG. 4 may be executed by client device 300 of FIG. 3, server 200 of FIG. 2, and/or any of client devices 106-114 of FIG. 1 (collectively or individually referred to as a "system"). It can be implemented.

As discussed herein, embodiments of the present disclosure divide machine learning algorithms into five steps, discussed below, and process solutions to problems within a video game if the problems can follow the same format. can thus provide an interface between video games, machine learning, and problems. As illustrated, process 400 is an iterative process in which a machine learning algorithm processes multiple proposed solutions in various cycles. The process begins with the system suggesting possible solutions (step 405). For example, in step 405, the system is running a video game. Within the video game, the system also processes machine learning algorithms to provide solutions to scientific problems that are broken down into a format for proper processing by the in-game machine learning algorithm. In step 405, the system suggests or generates any potential solutions to the problem, whether possible or not. In embodiments that incorporate CS, the system can suggest what the criteria are for potential solutions to the research problem and request input from users of potential solutions. For example, solutions may be suggested by the user or identified by the system based on user input, as discussed in more detail below. In various embodiments, multiple solutions may be proposed at step 405 and evaluated at each iteration of process 400.

The system then verifies whether the proposed solution is possible (step 410). For example, in step 410, the system determines whether the proposed solution is likely to be a valid solution to the problem. For example, if there are limitations to established measurements, algorithms, or equations that help determine the structure of the output, or if the solution is in a particular format (e.g., a Bayesian network that needs to be acyclic) If required, the proposed solution may not be possible. If the proposed solution (or all of the solutions proposed in step 405) is impossible or invalid, the system returns to step 405 and proposes other solutions.

After verifying that the proposed solution is possible, the system processes the verified solution and generates metrics (step 415). For example, in step 415, the system tests or evaluates the solution by running a machine learning algorithm using the currently proposed solution. The processing also includes characterizing the performance of the machine learning algorithm by determining how well the algorithm performed using one or more metrics. For example, and without limitation, these algorithm characterization metrics may include execution time, global/local scores, total solutions searched, percent of valid solutions, and trend on learning curve. These are characteristics of machine learning algorithms that the system uses to see if the algorithm improves. As discussed in more detail below, testing or evaluation of a solution may include parameters of the solution that may be suggested or selected by the user (e.g., identified by the system based on user input) or selected or suggested by the system. It can include modification of values as well as combinations thereof.

As mentioned above, machine learning algorithms can be implemented within a game as long as the solution can be characterized, regardless of whether the improvement is significant or whether the solution is ultimately a useful outcome. The processing of machine learning algorithms in games is useful if there is any improvement, as determined based on a comparison of characterization metrics. The abstraction interface of the present disclosure allows machine learning algorithms to be executed within popular and interesting video games, so that even minor improvements can be made by the massive scale of video games. It will be significantly expanded.

Execution time is the total time (or other metric of computational resources such as cycles) used by a machine learning algorithm to process a solution. For example, even if an algorithm provides only a small improvement per cycle and has a short running time, this algorithm may be better suited to an algorithm that provides a larger improvement in the solution per cycle but has a much longer running time. may be more preferable than The global score characterizes how the current solution ranks against all previous solutions processed for a particular scientific problem. For example, solutions are processed by the system, as are other client devices within system 100 that are also running machine learning algorithms for the particular scientific problem being processed. The local score characterizes how the current solution ranks against all previous solutions processed by a particular client device 300.

The solution searched is a metric for the number of possible solutions in the total solution space that the techniques used by the machine learning algorithm can explore.

For example, using techniques used by machine learning algorithms, the solutions explored may be determined by how many solutions, or what percentage of the proposed solutions in step 405, out of the total possible number of solutions in the solution space. Quantify how many solutions (e.g., the number of solutions) explored by a machine learning algorithm. For example, taking advantage of the increased engagement provided by this disclosure, machine learning algorithms that explore a larger percentage of the total solution may be better despite lower performance elsewhere. Furthermore, this metric can be updated over time as additional solutions are suggested in additional iterations of the process.

Percentage of valid solutions is a metric for how many percentages of proposed or suggested solutions for a machine learning algorithm are valid, e.g. only two-thirds of the total solutions If effective, an algorithm with more total solutions can be worse than another algorithm with fewer solutions but a higher percentage of effectiveness. Additionally, this is a metric that is updated over time as additional solutions are verified in additional iterations of the process. The trend on the learning curve is a metric of how an algorithm performs over time. A learning curve is a diagram that illustrates how an algorithm performs over time, using, for example, a global score metric or a local score metric. The performance of the algorithm may not be very good at the start, but may rise more quickly on the learning curve, and thus may be preferable to an algorithm that starts with a better solution but shows little improvement from cycle to cycle. do not have.

As discussed herein, insofar as an algorithm processing a solution to a scientific problem can be characterized using, for example, one or more of the example metrics discussed above, the performance of the algorithm can be measured using If this measured performance improves, regardless of the degree of improvement, the algorithm has been successfully abstracted into gameplay so that it is ultimately useful for solving scientific problems.

The system then selects a processed solution (step 420). For example, in step 420, the system reviews the results of the n processed solutions, e.g., using the metrics generated for the processed solutions, and determines which solutions to use for further testing. Decide or choose. For example, often the solution proposed in step 405 is based on an initial state seeded from a previously selected solution from step 420. Accordingly, step 405 may further include the system proposing or proposing a new solution using the selected solution as a starting point. For example, as part of step 420, the system may select a solution based on criteria such as whether the solution is worth exploring further, whether it has the highest score (local or global), etc.

The system then terminates the process based on the termination conditions (step 425). For example, at step 425, the system may determine whether processing of the machine learning algorithm satisfies one or more termination conditions. If the termination condition is not met, the system returns to step 405 and returns to another iteration of process 400. If the termination condition is met, the system stops processing the machine learning algorithm. For example, termination condition(s) may be for the machine learning algorithm to process for a certain amount of time, for a certain number of iterations, for a certain score on any of the metrics, for the machine learning algorithm to process for a certain number of iterations, to process a certain score on any of the metrics, may include that the solution does not improve for a certain number of trials or for a certain period of time (e.g., the algorithm has converged).

Additionally, in various embodiments, the present disclosure provides a framework that allows users to interact with machine learning algorithms via video games. By abstracting gameplay into algorithms and abstracting the algorithms that run within the game, various embodiments of this disclosure can mix and match elements of machine learning and user interaction to create one or more of the algorithms. Multiple steps can be replaced or augmented, or one or more steps can be augmented. For example, a user may select possible solutions, evaluate and rank solutions, and/or select or decide which solution is the best or next used solution. can.

This disclosure provides at least four example scenarios for using user interaction to replace or augment one or more steps of an algorithm or to augment one or more steps. As a first example, a user can generate training data that is used to evaluate processed solutions. For example, for a machine learning algorithm to identify faces from photographs, a user may select photographs of faces to generate training data used to evaluate algorithm performance. The second example is supervised learning. For example, in the case of a machine learning algorithm for selecting specific objects in a photo, the user can indicate whether the machine learning algorithm performed correctly.

A third example is mutation. When a machine learning algorithm remains in a local search space or "local minimum" (where the solution does not improve much, if at all, over time), the algorithm applies mutation techniques to reduce parameters and variables. Escape from the local search space by changing it randomly. In one or more embodiments, the system may request input from the user to suggest mutations or different solutions in an attempt to move the algorithm away from the local minimum. For example, a client device may evaluate 1000 solutions using machine learning and provide one or more of the solutions to the user for the user to manipulate to make them better. The user manipulation solution is then fed back to the machine learning algorithm to continue the process as the system attempts to escape from the local search space. This allows the system to apply user-selected mutations to the machine learning algorithm, as well as mutations generated using heuristics or machine learning. As one particular example, for the problem of trying to program a spacecraft to fly through an asteroid field, each sensor on the spacecraft is considered an input to a machine learning algorithm, and each sensor , for example, with weight values in the form of a matrix for a neural network. The machine learning algorithm calculates sensor weighting values based on the program's relative importance among the various sensors (e.g., the type and severity of responses the spacecraft should take based on the various sensor inputs). (including the type of response and the severity of the response). User-selected mutations can provide input regarding the weight sensors of which sensors are more or less important (eg, relative importance between long-range or short-range sensors). Therefore, rerunning the algorithm with user-selected mutations can move the solution out of the local search space or local minimum.

A fourth example is the human search heuristic. In addition to machine learning algorithm processing, data from actual monitor user gameplay is recorded and provided to the machine learning algorithm to improve the solution (or one of them) from what the user attempts to train the machine learning algorithm more quickly. section) can be "learned" or selected. In other words, machine learning algorithms can evaluate accumulated user data and identify decision patterns that attempt to learn why user decisions are made. For example, the system evaluates a user's search and data optimization and then trains a neural network to search for data in the same or similar way that the user did to optimize the user's search and data optimization. programming a search heuristic to determine whether the results are better or lead to a better solution than if the neural network had performed the process without examining the data generated from the include.

As a result, embodiments of the present disclosure further provide mechanisms for artificial intelligence (AI) and users to collaborate in a collaborative environment to solve problems. In addition to the ability for machine learning algorithms to simply use idle CPU resources and process specific components in the background, embodiments of the present disclosure can also improve machine learning by incorporating input from user feedback and gameplay. Further improvements are provided to the processing of the algorithm and the user contribution techniques described above are used to further improve the effectiveness of the machine learning algorithm. For example, if a user is selecting a solution, the machine learning algorithm can have the game visually display the solution to the user for the user to decide whether that solution should be used in the next cycle. . Accordingly, embodiments of the present disclosure provide a framework that allows users to interact with machine learning algorithms via video games.

Therefore, by focusing on improving rather than identifying the optimal solution and basing the game interaction part on the algorithm to find a better solution that is not necessarily the best solution, embodiments of the present disclosure abstract machine learning algorithms. This allows machine learning algorithms to be applied to more games than would otherwise be applicable. By focusing on and utilizing the abstraction techniques disclosed herein, embodiments of the present disclosure can, for example, create a game as if no algorithms were used in the game or deviations from normal gameplay. It makes it possible to maintain high engagement with the game with little to no play. By abstracting the elements of machine learning algorithms in this manner, embodiments of the present disclosure enable the inclusion of machine learning algorithms within the components of a game. This allows researchers and developers to use the abstraction interfaces for machine learning algorithms disclosed herein to create machine learning tools within video games that are mass-appealing and much more widely accepted. be able to handle their own problems through. Further, the abstracted interfaces of this disclosure may use any type of machine learning algorithm or technique (i.e., independent of any particular type of algorithm); for example, machine learning algorithms may include genetic algorithms, Particle swarm optimization, neural network techniques, etc. can be used.

In various embodiments, the system provides a universal interface that allows game developers to integrate various CS projects as well as the ability to access a distributed volunteer grid computing network. For researchers to utilize this resource, they simply need to analyze and decompose the project into a format supported by the interface. Game developers integrate APIs into their games, design engagement systems that support given problem types, and access result data. This system, once converted to a supported format, is reusable to apply to other similar problems. Basically, any type of CS project can be analyzed if the project can be decomposed into the following general analysis structure.

First, the system identifies a problem to analyze or solve. When analyzing a project, the first step is to identify the type of work required to process the data. For example, is it a data generation issue (e.g. a user is creating new data for some purpose)? Is the problem a user analysis problem (e.g., are users categorizing or otherwise performing microtasks)? Is the problem a user-computed problem (eg, the user's players are applying more complex problem-solving skills)? This problem would more traditionally be done using machine learning algorithms or approaches, but could it be a problem with a potential interactive component (hybrid project)?

Second, the system captures and transforms the data. For example, the system determines what format the data is in, whether the data is divided into smaller, more manageable micro-chunks of data to make it easier to access, whether the data is uniform, and whether the data is unique to some parts of the data. Determine whether something exists. For example, the system may determine that some of the data has already been processed and can serve as a base sample (eg, as training data or as the basis for a game tutorial). The data may need to be placed in a parsable format, such as a traditional comma separated value (CSV) file, or the CSV file may need to be used to index into the image data.

Third, the system identifies supporting heuristics, simulation studies, or fitness criteria. For example, the system determines whether there are established heuristics that have proven successful in past analyzes of this or similar studies. These heuristics can provide guidelines to help the development community understand the process and design associated with gameplay loops, allowing developers to create activities that mimic some of the heuristic approaches. Some of the choices may be informed. Developers can map the normalized heuristic output to various gameplay elements (e.g. object colors, statistical multipliers) and provide unique ways for users to visualize and interact with the problem/data. You can also do it. Similarly, the system determines whether there are established equations, algorithms, tools, or other equations that can make the simulation more accurate. Examples of this type of information include the Rosetta structure prediction method. In situations where user calculations are a factor, this type of information helps give the data value. These simulation tools may also be used for validation, either in the validation step or the selection step, and may also be useful in creating fitness scores or metrics as part of the process step. . Group scoring or consensus can also serve as a validation tool that uses statistical analysis techniques to provide numerical guidance. Any type of fitness in which a process step outputs a value or metric that represents the relative quality of a given user-generated result, in order to provide feedback to the user or gain a relative sense of the quality of the user's work. A function exists. This fitness function can be derived from the type of problem being solved (eg, a clustering problem) or something specific to the field of study. To work well in games, these fitness functions need to be as fast, efficient, and consistent as the frame rate, and real-time feedback can be critical to successful engagement.

There are several factors to consider to easily integrate scientific research into gameplay and aesthetics/themes. Some principles for integrating games to provide a balance between gameplay and scientific research value are described below. First, the game is designed for wider application. If a problem can be abstracted so that other problems of the same type can also be addressed (e.g. any problem that uses a clustering algorithm can use the same game mechanics), then gameplay Opportunities are provided for mixing data to add diversity. Unless scientific understanding is fundamental to the success of the output, game design elements are designed to achieve desired results without specific reference to science.

Second, game integration identifies and enhances the desired output. To enhance the research value of the output, fitness functions and other measures of success influence the feedback given to the player. The fitness function provides numbers, but the feedback is also modified to match the aesthetic and theme of the game. Outputs are compared to the described measures of success, as best results may vary depending on research needs. Other desired behaviors such as accuracy (quality), overall contribution (quantity), and efficiency (speed) factor them into game rewards and/or feedback. The recipe for success is not necessarily related to output, but is balanced so that the most important factors have sufficient weight. Improper weighting can reinforce undesirable behaviors or result in users "gaming" the system, both of which can reduce the quality of the results.

Third, research activities are integrated into the game loop. Since the gameplay loop represents the core activity that motivates the player, connecting research activities to the loop facilitates seamless integration. Both the player's actions and the resulting output are factors in integrating the activities. Research activities are broken down into component actions that are used to complete the output. These actions often imply specific gameplay elements, mechanics, or systems. Comparing the formation of outputs resulting from research activities with desired results can suggest integration techniques, for example any existing game mechanism that mimics scientific activities. By integrating mechanics into gameplay, scientific elements become more naturally incorporated. Similarly, leveraging game mechanics proven in other games can help reduce the number of variables that need to be balanced, allowing the focus to be on integration rather than engaging mechanics themselves.

Fourth, games are designed for unusual outcomes. Unlike gaming systems, which can be tailored to the needs of the game, research output can sometimes provide extreme and unexpected results. Depending on the data, it is possible to allow the player to experience complete success, failure, or never achieve it. Therefore, the game needs to maintain engagement with these results, which are factored into game feedback or other gameplay elements.

Fifth, the game balances complexity and accuracy. Players can balance numerous factors, but the complexity of gameplay, the complexity of human computation, and the expected speed and accuracy pose challenges that extend beyond the average player's cognitive threshold. There are points that can be created. To balance this out, increases in complexity in one element can be balanced by decreases in complexity in other elements. If the problem has a need for high precision or requires complex human calculations, a fast-paced, real-time, high-skill game is unlikely to produce the desired results with a large number of players. Slow-paced or turn-based mechanics allow players to consider the problem over time, which may better suit their needs. As a result, simpler problems can typically sustain more complex mechanics and timing if they better suit the desired gameplay.

Sixth, game integration considers the aesthetic and/or thematic integration of the game. Research activities involving non-visual data often work on any subject. Those that involve visual data, such as elements found in classification type problems, may require some creative consideration to fit the elements into the overall game.

To illustrate the advantages of embodiments of the present disclosure, the following examples relating to drug research are discussed. This implementation is intended as an illustrative example and not a limitation to the various embodiments that may be implemented in accordance with the principles and teachings of this disclosure. In searching for chemotherapy co-medication properties for multi-drug resistant cancers, the problem identified for solution was one of quantity. Combination drugs are drugs that increase the effectiveness of existing drug-resistant cancer cells. Combination drug research is relatively new, so there is little data that allows researchers to identify good candidates among existing drugs. Even with some elimination/reduction, over 5 million potential concomitant medications would need to be evaluated in some way. In an early study, scientists purchased 71 drugs and tested them as potential co-medications. Twenty-three species showed some success, while the remaining 48 did not. Each drug has 30 discrimination variables. In the simplest situation, researchers would like a way to determine the commonalities of the 23 successful drugs, to help create a system for identifying new drugs for testing, and conversely to determine what the commonalities of the remaining drugs are. Revealed what aspects made them unsuccessful.

Following the CS model, the first question focused on identifying the problem. After examining the problem, dataset, and desired information, we look for commonalities between the groups presented as a clustering problem. After determining the problem parameters, the data was analyzed to determine how to present the information. Each drug was already present in a table that recorded 30 variables. To use the data in a clustering problem, the data was normalized so that everything could be represented on a consistent scale, in this case 0 to 1. We added data points to record the relative success or failure of each drug as a combination drug and converted the data to a CSV file.

When looking for heuristics, simulation studies, and suitability criteria, the need for a clustering problem was identified. There are many clustering algorithms that can form a basic algorithm. In this case, a centroid-based algorithm such as K-means was chosen as a starting point. Basically, a centroid-based algorithm moves the centroid looking for the best score and starts a heuristic. Center of gravity motion also suggests an interaction area for the player to interface with and functionally replaces that aspect of the algorithm.

The other driving factor in the clustering algorithm is to determine the fitness score, which guides the movement of the center of gravity and whether it is moving towards or away from an improved score. This will help you identify what you are doing. This feature is also important because it allows players to evaluate their success. In this situation, the goal is to try to see what the two groups separated, which is the basis for the fitness score, and the weighted average is how many co-medications one group has at work. and how many were in other groups. Since there are two basic groups, we started with two centroids, but the fitness score is designed such that any number of centroids can be added in future iterations.

Analyzing any accuracy guidelines requires the user to have sufficiently accurate data to know whether the user is improving their score, but if the data provided by the player is Accuracy was not as important as it is expected to provide training data for a separate neural net designed to analyze and identify potential drug combinations. If more refinement was proven to be necessary, the researchers planned to run a separate conventional centroid-based algorithm based on the player's results to refine the scoring.

Important information to know for game development is how the clustering algorithm worked when implementing problems, data, and fitness scores in game development to allow game developers to create interfaces. The objective was to understand what was being done and what was being attempted to be achieved with the data set. Finally, the questions in the analytical structure were analyzed to check whether the data needed to design the interface to the game was identified. In order to propose a new solution, the user or algorithm needs to be able to provide new centroid locations. To provide a valid solution for our parameters, the centroid location needed to be within the normalized space determined when we formatted the data. The processing step in this situation was to apply a fitness function and obtain a relative score representing how well the groups were separated. In the application of player-generated data as training information for neural networks, the resulting solution was effective and therefore no filtering was necessary at that stage. If further refinement was found to be necessary, a separate K-means algorithm was run on the resulting centroids. Finally, we defined the exit parameter as open.

Two examples of gamification of the algorithms described above are provided. The first example depicts a player manipulating a radio dial to influence the center of gravity position, and the resulting score shifted the waveform closer or farther by matching the target wave. The second is a medical mini-game where the data positions represent body parts and the player can adjust the center of gravity position to improve various functions. Either or both of these mini-games can be integrated into various aspects of popular and engaging video games.

Various embodiments provide constructively enhanced machine learning. Constructivist learning generally refers to scenarios in which teachers can bring past knowledge into a collaborative partnership with students to guide students' experiential learning processes. In these embodiments, the system introduces user knowledge directly into the network, which mimics the instructor-to-student collaboration found in constructivist learning theory. As discussed herein, these embodiments of the present disclosure augment machine learning by receiving user-generated input. For example, in each of the examples discussed above, machine learning algorithms are expanded based on interactions, whether for mutations, training data, search heuristics, supervised learning, etc. For example, in some embodiments, the system uses clustering techniques on the convolution kernels to reduce the problem space and identify key convolution kernels. The system then replaces the kernel with the user-built kernel. For example, the system allows for algorithm guidance by augmenting inputs and directly modifying hidden layers, weights, and connections throughout the training process. The system can also take advantage of patterns and optimization opportunities identified by the user during training and subsequently modify the machine learning model to take advantage of the user's intuition. In these embodiments, the system can improve model accuracy, compress model size, or allow model refinement in the absence of large data sets.

In one example of constructively augmented machine learning, machine learning is augmented by using human identified patterns to accelerate machine learning. For example, by receiving a large set of user game inputs (e.g. for object identification in an image) for a proposed solution, the system averages the game inputs to calculate a probability distribution. be able to. The system can then narrow the search space for the machine learning algorithm. In this way, the system uses user input as an alpha filter that is applied as one of many filters applied by the machine learning algorithm when processing a problem to identify a solution.

Another example of constructive augmentative machine learning can track and model user input during games or other interactive applications. The system can use the model of user input to modify the values of the machine learning algorithm based on performance.

In another example of constructive augmentation machine learning, user input can be used to compress machine learning algorithms. For example, the system can use clustering to find an average filter for compression. However, such compression may over-represent some solutions that are assumed to be different, for example due to noise present in the solutions, while under-representing discrete solutions within the compressed dataset. . Using the abstraction interface of these embodiments of the present disclosure, the system can submit clusters for classification and compression by a user (eg, in an interactive environment). For example, if a user is better off recognizing (and ignoring) noise (e.g. in an image) in order to correlate similar solutions, machine learning algorithms may not be able to correlate due to the noise. There is. On the other hand, users may not have the basis for the need to achieve the compression that exists in machine learning algorithms when correlating solutions that are not sufficiently similar. In these embodiments, the system uses user compression inputs along with machine learning inputs, weights each input based on what the associated classifier does better, and uses one or both of the inputs to , the size of the data set can be compressed or reduced to help further speed up computations and reduce storage requirements.

In another example of constructive augmentative machine learning, user input can be used to improve or modify the machine learning model itself, where the improvement of the machine learning model is considered a given problem. For example, a machine learning model can run a clustering algorithm on the convolution to determine the average centroid. These centroids may be presented to the user, eg, as an image or set of images, eg, in an interactive application, for the user to select or suggest solutions for improving the machine learning model. The user selected or proposed solution may be the formation of one or more parameters or determined centroids to modify or update the machine learning model. The system can then run a machine learning algorithm (e.g., over multiple iterations) based on the modified or updated model and evaluate the model's performance to determine whether improvements have occurred. . The later iterations may include the system receiving additional suggestions for solutions for improving the machine learning model after the previous iterations on user solution suggestions. In this way, by using user-suggested solutions to improve machine learning models, it is possible to reduce the amount of data required or used to adequately train a neural network. It could be possible. Additionally, using user-suggested solutions to improve machine learning models can allow for a reduction in the amount of time for computing or training neural networks due to user-suggested modifications. .

FIG. 5 illustrates an example implementation of a machine learning algorithm within a model of a gameplay loop 500 according to various embodiments of the present disclosure. For example, the gameplay loop 500 shown in FIG. 5 may be executed within a game executed by client device 300 of FIG. It can be implemented. The illustration of gameplay loop 500 is intended as an example and not a limitation to the various embodiments that may be implemented in accordance with this disclosure.

Gameplay loop 500 is a model or representation of common activities that may occur during gameplay and is relevant or usable for incorporating machine learning algorithms. For example, during the load (collect) section 510, the system may obtain datasets and loaded questions while the user is collecting resources during the game. During the analysis section 515 (the construction, upgrade, and modification section of the gameplay loop 500), the system is processing or evaluating solutions during the game that the user is building, upgrading, and/or modifying. For example, portions of process 400 may be sub-loops within analysis section 515 where new solutions are proposed, validated, and processed. In one example, analysis section 515 may include modifying the proposed or determined solution, eg, with respect to constructivist augmented machine learning as discussed above. Thus, the system can utilize a deep learning network's modification kernel for machine learning models that are modified by user interaction. Clustering may provide a possible solution. The system can then use user interaction to modify the clusters and then proceed with modifying the machine learning model.

Next, once the processed solution has been processed, the system tests the solution (test section 520) to determine how well the selected solution was, and then compares the solution to the previous solution. (section 525) to determine whether the solution was better than the previous solution, and then select that solution (selection section 505) and begin loop 500 again. As part of the testing section 520, the system may generate metrics during user engagement or combat during the game. As part of the comparison section 525, the system may also encourage user participation and/or improved results with in-game rewards, including, for example, continued gameplay, virtual items, rankings, and the like.

FIG. 6 illustrates a process for dynamically scaling gameplay and using algorithmic performance to adjust game levels, according to various embodiments of the present disclosure. For example, the process illustrated in FIG. 6 may be performed by client device 300 of FIG. 3, server 200 of FIG. 2, and/or any of client devices 106-114 of FIG. 1 (collectively or individually referred to as a "system"). It can be implemented.

In these embodiments, the system monitors and uses learning curves for algorithm performance to adjust game levels and dynamically scale gameplay (step 605). For example, in step 605, the system may plot performance metrics, such as global scores, over time to generate and monitor algorithm learning curves. An example of a scoring metric for algorithm performance or learning curve is illustratively shown in FIG. As shown, a large ramp in algorithm performance initially occurs at 705, after which the algorithm performance flattens out at a plateau 710 after some time. The ramp-up period is typically the beginning of a level. As soon as the score reaches a plateau, the game reaches the end of the level. The system detects the occurrence of this plateau (step 610). For example, in step 610, the system may use one or more thresholds or an average of consecutive scores as a parameter to determine when algorithm performance plateaus relative to a previous ramp-up. This plateau can be considered a minimum.

Next, the system monitors when algorithm performance exceeds a plateau (step 615). For example, in step 615, the system is looking for mutations from the local minimum of plateau 710 for the user, finds a better solution, increases the algorithm performance again in 715, and starts the learning process again. . As part of step 615, the system may use one or more thresholds, which may be relative to score variation within plateau 710, in determining that algorithm performance has exceeded plateau 710. . Upon detecting algorithm performance above a plateau, the system increments the game level (step 620) and the process is repeated for each level of the game. For example, increasing algorithm performance after a plateau is considered level 2 (or next level) of the game.

As the level changes, the gameplay character improves in ability as an incentive to escape from the previous local minimum and improve algorithm performance. Each level is more difficult to reach, meaning additional computational time and/or algorithmic interaction is required to reach the next plateau 720 and breakout at 725. For example, users may want to leave the computer running longer in order to donate additional resources (e.g., performing collection steps in a game) and additional incentives to reach the next level. Incentives are given.

In these embodiments, the difficulty of the game is scaled according to the level of the character, not necessarily the performance of the algorithm. The difficulty of the game is scaled based on the character's abilities based on the level obtained as a result of monitoring the learning curve. For example, in a first-person shooter (FPS) game, difficulty is the number of enemies at a certain level. That number can be increased based on the character's ability to keep gameplay challenging and engaged. A character's strength, or character's abilities, depend on the value of the achieved level, which is tied to a learning curve. In this way, gameplay reflects algorithm performance and users are provided with incentives and engagement to further improve algorithm performance.

Figures 4 and 6 are example processes for implementing abstract algorithm interfaces that enable gamification of machine learning algorithms and use algorithms to dynamically scale gameplay and adjust game levels. 4 and 6, various changes can be made to FIGS. 4 and 6. For example, although shown as a series of steps, the various steps in each figure can overlap, occur in parallel, occur in a different order, or occur multiple times. In other examples, steps may be omitted or replaced with other steps.

It may be advantageous to provide definitions of certain words and phrases used throughout this patent document. "Coupled" and similar terms refer to any direct or indirect communication between two or more elements, whether or not the elements are in physical contact with each other. The terms "sending," "receiving," and "communication" encompass any direct or indirect communication, both direct and similar communications. The term "comprising" means including, but not limited to. The term "or" includes and/or. The term "related" includes similar terms, and means having some kind of relationship, such as including, interacting with, combining, communicating with, cooperating with, proximate to, abutting, intervening with, juxtaposing with, or the like. The term "at least one" when used with a list of items indicates that one or more different combinations of items included in the list may be used and only one item may be required. show. For example, "at least one of A, B, and C" includes any combination of A, B, C, A and B, A and C, B and C, and A, B, and C.

Additionally, the various functions described below may be implemented or supported by one or more computer programs, each formed from computer-readable program code and embodied on a computer-readable medium. The terms "application" and "program" refer to one or more computer programs, software components, sets of instructions, procedures, functions, objects, classes, instances, associated data, or implementation in suitable computer-readable program code. refers to those parts that have been The phrase "computer readable program code" includes any type of computer code, including source code, object code, and executable code. The phrase "computer-readable medium" means a computer-readable medium, such as read-only memory (ROM), random access memory (RAM), hard disk drive, compact disc (CD), digital video disc (DVD), or any other type of memory. Includes any type of media that can be accessed. Non-transitory computer-readable media include media that can store data permanently and media that can store data and later overwrite it, such as rewritable optical disks or erasable memory devices. .

Definitions of other specific terms are provided throughout this patent document. Those skilled in the art should understand that in many, if not most, cases, such definitions apply to previous uses as well as future uses of the words and phrases so defined.

Although this disclosure has been described using exemplary embodiments, various changes and modifications may suggest to those skilled in the art. This disclosure is intended to cover such changes and modifications as come within the scope of the appended claims.

One embodiment provides a method for interactive machine learning. This method is based on the steps of identifying solutions to a given problem and determining that the identified solutions are possible solutions to the given problem. processing possible solutions by executing; selecting a processed solution based on a plurality of processed solutions at least in part from different iterations of the process; and selecting a processed solution based on a termination condition. and determining whether to terminate the process using the selected solution to the given problem. Executing the process includes receiving input to the interactive application for use in performing at least a portion of the steps via a user interface for the interactive application.

Another embodiment provides a system for interactive machine learning. The system includes a processor and a communication interface operably connected to the processor. The processor executes a machine learning application using the possible solutions based on identifying solutions to the given problem and determining that the identified solutions are possible solutions to the given problem. selecting a processed solution based on a plurality of processed solutions at least in part from different iterations of the process; and selecting a processed solution based on a termination condition. determining whether to terminate the process using the selected solution to the problem. The communication interface is configured to receive input to the interactive application for use in performing at least a portion of the steps via a user interface for the interactive application.

Another embodiment provides a non-transitory computer-readable medium for interactive machine learning. The computer-readable medium, when executed by a processor of the system, causes the system to perform the steps of: identifying a solution to a given problem; and determining that the identified solution is a possible solution to the given problem. processing possible solutions by running a machine learning application using the possible solutions; and determining, based on a termination condition, whether to terminate the process using the selected solution for a given problem. Contains program code. The computer-readable medium includes program code that, when executed by a processor of the system, receives input to the interactive application for use in performing at least a portion of the steps, via a user interface for the interactive application, to the system; further including.

In any of the above examples and embodiments, the process generates metrics for the processed solution processed at each iteration, and based on one or more of the generated metrics, the process generates metrics for the processed solution processed at each iteration. further comprising characterizing the performance of the machine learning application using the method. Selecting the processed solution also includes selecting the processed solution from the plurality of processed solutions based on performance of a machine learning application using the selected solution.

In any of the above examples and embodiments, the process includes receiving input for use in performing at least a portion of one of the steps, and the receiving is based on the received input. , identifying modifications to parameters of a machine learning application, identifying a solution to a given problem, identifying the parameter modifications as at least part of the identified solution, and processing possible solutions for an iteration of the process; modifying the machine learning application based on the parameter modification and running the modified machine learning application in one or more iterations.

In any of the above examples and embodiments, the parameter modifications are based on the received input being a set of parameter modifications generated by the machine learning application in a previous iteration of the process, i.e., in a previous iteration at a time prior to that iteration. Identified based on being selected from.

In any of the above examples and embodiments, receiving input for use in performing at least a portion of one of the steps includes, via a user interface, for interaction presented in an interactive application. possible solutions by receiving the input as a parameter input, identifying a solution to a given problem, identifying the parameter input as at least part of the identified solution, and running a machine learning application using the possible solutions and using parameter input from the interaction presented in the interactive application as at least a modification of parameters for the machine learning application.

In any of the above examples and embodiments, receiving input for use in performing at least a portion of the steps may include, via a user interface, relating to interaction presented in the interactive application. receiving input as attempted solutions for at least a portion of a given problem, wherein a plurality of attempted solutions are received over multiple iterations of the process and selecting a solution to be processed; The method includes evaluating a plurality of attempted solutions and selecting a solution to be processed based on the evaluation of the attempted solutions.

In any of the above examples and embodiments, receiving input for use in performing at least a portion of one of the steps may include, via a user interface, implementing a machine learning application using a possible solution. The processing includes receiving input as an indication of whether the output from the execution is correct, and processing the possible solutions includes evaluating the possible solutions based on the received indication.

In any of the above examples and embodiments, the interactive application may be a video game, and the received input used to perform at least a portion of the one step of the plurality of steps is from gameplay of the video game. Received.

Nothing described in this application should be construed to imply that any particular element, step, or feature is essential for falling within the scope of a claim. The scope of patented subject matter is defined solely by the claims. Furthermore, none of the claims fall within 35 U.S.C. unless the exact words "means for" are followed by a participle. S. C. §112(f) is not intended.

[Cross reference to related applications and priority claims]
This application claims priority to U.S. Provisional Patent Application No. 62/648,198, filed March 26, 2018. The contents of the above patent documents are incorporated herein by reference.

Claims (13)

The processor
identifying a solution to a given problem;
processing the possible solutions by running a machine learning application that uses the possible solutions based on determining that the identified solutions are possible solutions to the given problem; ,
selecting a solution to be processed ;
determining whether to terminate the process with the selected solution for a given problem based on a termination condition;
Iteratively executes a process that includes
The processor,
generating metrics for the processed solution processed in each iteration;
characterizing the performance of the machine learning application using the processed solution based on one or more of the generated metrics;
The process further comprises:
Executing the process includes receiving, via a user interface for the interactive application, input to the interactive application for use in performing at least a portion of one of the steps. fruit ,
Selecting the solution to be processed includes selecting the solution to be processed from a plurality of solutions to be processed based on performance of the machine learning application using the selected solution.
Interactive machine learning methods. Receiving input for use in performing at least a portion of one of the steps comprises:
identifying modifications to parameters of the machine learning application based on the received input;
including that
Identifying a solution to a given problem is
identifying parameter modifications as at least part of the identified solution for the iteration of the process;
including that
Possible processing solution is
modifying the machine learning application based on the parameter modification;
running the modified machine learning application in one or more iterations;
including
The interactive machine learning method according to claim 1. the parameter modifications identified based on the input received are selected from a set of parameter modifications generated by the machine learning application in a previous iteration of the process;
the previous iteration is an iteration temporally earlier than the current iteration;
The interactive machine learning method according to claim 2 . Receiving input for use in performing at least a portion of one of the steps comprises:
receiving, via the user interface, the input as a parameter input for an interaction presented in the interactive application;
including that
Identifying a solution to a given problem is
identifying the parameter input as at least part of the identified solution;
including that
Processing the possible solutions by running the machine learning application using the possible solutions includes:
using the parameter input from the interaction presented in the interactive application as at least one modification of a parameter for the machine learning application;
including
The interactive machine learning method according to claim 1. Receiving input for use in performing at least a portion of one of the steps comprises:
receiving input via the user interface as a solution to be attempted for at least a portion of a given problem regarding the interaction presented in the interactive application;
including that
multiple attempted solutions are received over multiple iterations of the process;
Selecting said solution to be processed is
evaluate said multiple attempted solutions;
selecting a solution to be processed based on an evaluation of the attempted solutions;
The interactive machine learning method according to claim 1. Receiving input for use in performing at least a portion of one of the steps comprises:
receiving input via the user interface as an indication of whether the output from running the machine learning application using the possible solution is correct;
including that
Possible processing solution is
evaluating the possible solutions based on the received indicators;
including
The interactive machine learning method according to claim 1. identifying a solution to a given problem;
processing the possible solutions by running a machine learning application that uses the possible solutions based on determining that the identified solutions are possible solutions to the given problem; ,
selecting a solution to be processed ;
determining whether to terminate the process with the selected solution for a given problem based on a termination condition;
a processor (210) configured to iteratively execute a process comprising;
a communications interface (220) operably connected to the processor;
including;
The processor,
generating metrics for the processed solution processed in each iteration;
characterizing the performance of the machine learning application using the processed solution based on one or more of the generated metrics;
further configured to perform said process, further comprising:
the communication interface is configured to receive input to the interactive application for use in performing at least a portion of one of the steps via a user interface for the interactive application ;
selecting said solution to be processed, said processor selecting said solution to be processed from a plurality of processed solutions based on performance of said machine learning application using said selected solution; It is configured,
Interactive machine learning system (200). The processor,
identifying modifications to parameters of the machine learning application based on the received input;
It is configured as follows,
In order to identify a solution to the given problem, the processor:
identifying parameter modifications as at least part of the identified solution for the iteration of the process;
It is configured as follows,
In order to process the possible solutions, the processor:
modifying the machine learning application based on the parameter modification;
running the modified machine learning application in one or more iterations;
It is configured as follows.
The interactive machine learning system according to claim 7 . the parameter modifications identified based on the input received are selected from a set of parameter modifications generated by the machine learning application in a previous iteration of the process;
the previous iteration is an iteration temporally earlier than the current iteration;
The interactive machine learning system according to claim 8 . The communication interface further comprises: receiving input for use in performing at least a portion of one of the steps;
receiving, via the user interface, the input as a parameter input for an interaction presented in the interactive application;
It is configured as follows,
In order to identify a solution to the given problem, the processor further:
identifying the parameter input as at least part of the identified solution;
It is configured as follows,
In order to process the possible solutions by running the machine learning application using the possible solutions, the processor further comprises:
using the parameter input from the interaction presented in the interactive application as at least one modification of a parameter for the machine learning application;
It is configured as follows.
The interactive machine learning system according to claim 7 . The communication interface further comprises: receiving input for use in performing at least a portion of one of the steps;
receiving input via the user interface as a solution to be attempted for at least a portion of a given problem regarding the interaction presented in the interactive application;
It is configured as follows,
multiple attempted solutions are received over multiple iterations of the process;
In order to select the solution to be processed, the processor further:
evaluate said multiple attempted solutions;
selecting a solution to be processed based on an evaluation of the attempted solutions;
It is configured as follows.
The interactive machine learning system according to claim 7 . The communication interface further comprises: receiving input for use in performing at least a portion of one of the steps;
receiving input via the user interface as an indication of whether the output from running the machine learning application using the possible solution is correct;
It is configured as follows,
In order to process the possible solutions, the processor further:
evaluating the possible solutions based on the received indicators;
It is configured as follows.
The interactive machine learning system according to claim 7 . identifying a solution to a given problem;
processing the possible solutions by running a machine learning application that uses the possible solutions based on determining that the identified solutions are possible solutions to the given problem; ,
selecting a solution to be processed ;
determining whether to terminate the process with the selected solution for a given problem based on a termination condition;
Iteratively executes a process that includes
The process is
generating metrics for the processed solution processed in each iteration;
characterizing the performance of the machine learning application using the processed solution based on one or more of the generated metrics;
further including,
receiving input to the interactive application for use in performing at least a portion of the one of the steps via a user interface for the interactive application;
selecting the solution to be processed from a plurality of solutions to be processed based on performance of the machine learning application using the selected solution;
A non-transitory computer-readable medium (215) for interactive machine learning comprising program code that, when executed by a processor (210) of a system (200), causes said system to perform.