CN112742029B

CN112742029B - Method for simulating operation, method for testing game and related device

Info

Publication number: CN112742029B
Application number: CN202010295276.2A
Authority: CN
Inventors: 王洁梅; 周大军
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-04-15
Filing date: 2020-04-15
Publication date: 2023-04-18
Anticipated expiration: 2040-04-15
Also published as: CN112742029A

Abstract

The application discloses a method for simulating operation, a method for testing games and a related device, which are used in the field of artificial intelligence. The method comprises the following steps: receiving a game image sent by a client; if the game image meets the image recognition condition, acquiring an image recognition result of the game image through the object detection model; determining a target operation position according to an image recognition result; and sending the target operation position to the client so that the client carries out simulation operation according to the target operation position. According to the method and the device, the category information and the position information of the game elements can be identified from the game image intercepted by the client, and the corresponding operation strategy is generated based on the information, so that the operation strategy has stronger intention, the controllability of the operation is enhanced, and the operation of a real player can be better simulated.

Description

Method for simulating operation, method for testing game and related device

Technical Field

The present application relates to the field of artificial intelligence, and in particular, to a method for simulating operations, a method for testing a game, and a related device.

Background

Fishing games are a classic type of game that can be played in either the casino, personal Computer (PC) or mobile phone. The fishing game comprises a green-and-flowered fish school, a fishing net paved on the sky, a dazzling special gold coin effect and the like, and players can explore an interesting seabed world as much as possible.

The fishing game is internally provided with an automatic attack function, and after a player reaches a certain level, the player can click an automatic attack button to automatically operate, for example, a fishing net is released every 2 seconds, and fish schools in a game scene are randomly captured through the fishing net, so that the aim of automatic attack is fulfilled.

However, the currently provided "automatic attack" function has a large limitation, and can only launch an attack according to a fixed time or a fixed direction, so that the controllability of the operation is weak, and it is difficult to simulate the operation of a real player.

Disclosure of Invention

The embodiment of the application provides a method for simulating operation, a method for testing games and a related device, which are used for identifying the category information and the position information of game elements from a game image intercepted by a client and generating a corresponding operation strategy based on the information, so that the operation strategy has stronger intention, the controllability of operation is enhanced, and the operation of a real player can be better simulated.

In view of the above, one aspect of the present application provides a method for simulating an operation, including:

receiving a game image sent by a client;

if the game image meets the image recognition condition, acquiring an image recognition result of the game image through the object detection model, wherein the image recognition result comprises category information of at least two detection images and position information of the at least two detection images;

determining a target operation position according to the image recognition result, wherein the target operation position comprises at least one piece of position information to be operated;

and sending the target operation position to the client so that the client carries out simulation operation according to the target operation position.

Another aspect of the present application provides a method for game testing, including:

receiving a game image to be tested sent by a client, wherein the game image to be tested belongs to a game image corresponding to the state in the game;

acquiring an image recognition result of a game image to be tested through an object detection model, wherein the image recognition result comprises category information of at least two detection images and position information of the at least two detection images;

sending the target operation position to the client so that the client carries out simulation operation according to the target operation position;

receiving a simulation operation result sent by a client;

and generating a game test report according to the simulation operation result.

Another aspect of the present application provides a simulated operation device including:

the receiving module is used for receiving the game images sent by the client;

the acquisition module is used for acquiring an image identification result of the game image received by the receiving module through the object detection model if the game image meets the image identification condition, wherein the image identification result comprises category information of at least two detection images and position information of the at least two detection images;

the determining module is used for determining a target operation position according to the image recognition result acquired by the acquiring module, wherein the target operation position comprises at least one piece of position information to be operated;

and the sending module is used for sending the target operation position determined by the determining module to the client so that the client can perform simulation operation according to the target operation position.

In one possible design, in one implementation of another aspect of an embodiment of the present application,

the acquisition module is also used for acquiring the running state information corresponding to the game image after the receiving module receives the game image sent by the client;

the determining module is also used for determining that the game image meets the image recognition condition if the running state information is the game-in-game state;

the determining module is further used for determining that the game image does not meet the image recognition condition if the running state information is the pre-game state;

and the determining module is also used for determining that the game image does not meet the image recognition condition if the running state information is the game-play ending state.

In one possible design, in another implementation of another aspect of an embodiment of the present application,

the determining module is specifically used for acquiring M first detection images from the at least two detection images according to the category information of the at least two detection images, wherein the first detection images belong to an attacked category, and M is an integer greater than or equal to 1;

randomly determining a target detection image from the M first detection images;

and determining the position information corresponding to the target detection image as the position information to be operated.

the determining module is specifically used for acquiring M first detection images and M second detection images from the at least two detection images according to the category information of the at least two detection images, wherein the first detection images belong to an attacked category, the second detection images belong to a task category, and M is an integer greater than or equal to 1;

determining a matching result according to the template image to be matched and the second detection image, wherein the template image to be matched belongs to any one template image in a template image set, and the template image set comprises at least one template image;

if the matching result is that the matching is successful, determining target type information corresponding to the second detection image;

and determining the position information to be operated according to the target type information and the M first detection images.

the determining module is specifically used for acquiring at least one similar result from the second detection image in a sliding window mode by adopting the template image to be matched;

determining a maximum similarity result according to the at least one similarity result;

if the maximum similarity result is larger than or equal to the similarity threshold, determining that the matching result is successful;

and if the maximum similarity result is smaller than the similarity threshold, determining that the matching result is matching failure.

the determining module is specifically configured to, if the category information of the M first detection images includes target type information, acquire a target detection image from the M first detection images, where the target detection image corresponds to the target type information;

the determining module is specifically configured to randomly determine the to-be-operated position information from a preset range if the category information of the M first detection images does not include the target type information.

the determining module is specifically used for acquiring a third detection image from the at least two detection images according to the category information of the at least two detection images, wherein the third detection image belongs to the auxiliary function category;

determining a matching result according to the template image to be matched and the third detection image, wherein the template image to be matched belongs to any one template image in a template image set, and the template image set comprises at least one template image;

and if the matching result is that the matching is successful, acquiring the position information to be operated corresponding to the third detection image.

and the sending module is specifically used for sending the position information to be operated to the client so that the client triggers an attack operation to the direction corresponding to the position information to be operated.

and the sending module is specifically used for sending the position information to be operated to the client so that the client triggers the click operation according to the position information to be operated.

Another aspect of the present application provides a game testing apparatus, comprising:

the receiving module is used for receiving the game images to be tested sent by the client, wherein the game images to be tested belong to the game images corresponding to the game-playing state;

the acquisition module is used for acquiring an image recognition result of the game image to be tested, which is received by the receiving module, through the object detection model, wherein the image recognition result comprises category information of at least two detection images and position information of the at least two detection images;

the sending module is used for sending the target operation position determined by the determining module to the client so that the client carries out simulation operation according to the target operation position;

the receiving module is also used for receiving a simulation operation result sent by the client;

and the generating module is used for generating a game test report according to the simulation operation result received by the receiving module.

Another aspect of the present application provides a server, including: a memory, a transceiver, a processor, and a bus system;

wherein the memory is used for storing programs;

the processor is used for executing the program in the memory, and the processor is used for executing the method of the aspects according to the instructions in the program code;

the bus system is used for connecting the memory and the processor so as to enable the memory and the processor to communicate.

Another aspect of the present application provides a computer-readable storage medium having stored therein instructions, which, when executed on a computer, cause the computer to perform the method of the above-described aspects.

According to the technical scheme, the embodiment of the application has the following advantages:

the embodiment of the application provides a method for simulating operation, which comprises the steps of firstly receiving a game image sent by a client, obtaining an image recognition result of the game image through an object detection model when the game image meets an image recognition condition, determining a target operation position according to the image recognition result, wherein the target operation position comprises at least one piece of position information to be operated, and finally sending the target operation position to the client so that the client can carry out simulation operation according to the target operation position. By the method, the category information and the position information of the game elements are identified from the game image intercepted by the client, and the corresponding operation strategy is generated based on the information, so that the operation strategy has stronger intention, the controllability of the operation is enhanced, and the operation of a real player can be better simulated.

Drawings

FIG. 1 is a block diagram of an exemplary emulated operating system;

FIG. 2 is a schematic flow chart of a simulation method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an artificial intelligence operation flow implemented based on a fishing game in the embodiment of the present application;

FIG. 4 is a diagram illustrating an embodiment of the present application for implementing multi-module invocation based on simulation operations;

FIG. 5 is a schematic diagram of an embodiment of a method for simulating operations in an embodiment of the present application;

FIG. 6 is a schematic view of a game interface of the fishing game according to the embodiment of the present application;

FIG. 7 is a schematic flow chart illustrating the process of obtaining the image recognition result based on the game image in the embodiment of the present application;

FIG. 8 is a schematic diagram of an embodiment of an image of a sample annotation game in an embodiment of the present application;

fig. 9 is a schematic structural diagram of an end-to-end network in an embodiment of the present application;

FIG. 10 is a diagram of an embodiment of identifying information of a second detection image based on a task scene in an embodiment of the application;

FIG. 11 is a schematic diagram of an embodiment of obtaining similar results in the form of a sliding window in the embodiment of the present application;

FIG. 12 is a schematic diagram of an embodiment of randomly determining information of a location to be operated in an embodiment of the present application;

FIG. 13 is a diagram illustrating an embodiment of identifying a third inspection image based on a task scenario according to an embodiment of the application;

FIG. 14 is a schematic diagram of an embodiment of a method for testing a game in the embodiment of the present application;

FIG. 15 is a schematic diagram of an embodiment of a simulation operating device according to an embodiment of the present application;

FIG. 16 is a schematic view of an embodiment of a game testing apparatus according to the embodiment of the present application;

fig. 17 is a schematic diagram of an embodiment of a server in the embodiment of the present application.

Detailed Description

The embodiment of the application provides an operation simulation method, a game testing method and a related device, which are used for identifying the category information and the position information of game elements from a game image intercepted by a client and generating a corresponding operation strategy based on the information, so that the operation strategy has stronger intention, the controllability of the operation is enhanced, and the operation of a real player can be better simulated.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the application described herein may be implemented, for example, in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "corresponding" and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It should be understood that the simulation operation method provided by the present application can be applied to a scene of intelligent operation, for example, when a user cannot play a game temporarily, the simulation operation method provided by the present application can be used for simulation operation. The game testing method provided by the application can be applied to a scene of automatic testing, for example, developers update the playing method of the game A, and if the reasonability of the playing method is tested manually, the great labor cost is consumed, so that the game testing method provided by the application can be adopted, an intelligent agent simulates the operation of a real human being, and the automatic testing is carried out based on the testing requirement configured in advance.

Specifically, the types of games referred to herein include, but are not limited to, fishing games, running games, gunfight games, and the like. For example, in a fishing game, a player can release a fishing net in a certain direction, the fishing net is used for capturing fish swarms or props and the like, the capturing success rate can be adjusted according to the game difficulty, and the fishing net can be automatically released by the simulation operation method provided by the application. For example, in a running cool game, a player can control a character to run along a certain route, and during the running process, a monster or a prop may be touched. For example, in a gun battle game, a player can control a character to shoot in a certain direction, possibly shoot an enemy character, and possibly shoot a bias, and the simulation operation method provided by the application can automatically shoot the game character in a certain direction.

Referring to fig. 1, fig. 1 is a schematic diagram of an architecture of a simulation operating system in an embodiment of the present application, where as shown in the figure, the simulation operating system includes a game server (or a test server) and a terminal device, and a description will be given below of a simulation operating method and a game test method, respectively, with reference to fig. 1 and a specific scenario.

For ease of understanding, the present application will be described with reference to a fishing game in which a player controls the orientation of a turret from which fishing nets, shells or other fishing props are launched that attack targets such as "shoals," "turtles" or "treasure boxes" in the game scene. An automatic attack function is usually set in a fishing game, and after a player reaches a certain level, a button of the automatic attack function can be clicked, so that automatic operation is carried out. In a simulation operation scene, the terminal equipment is used for collecting relevant data of a fishing game, after a user triggers an automatic attack key, the terminal equipment feeds back a game screenshot acquired in real time to the game server, and the game server identifies game elements based on the game screenshot. It should be noted that the frequency of sending the game screenshots may be 10 frames per second, or 20 frames per second, or may be adjusted in real time according to the device performance and the network status, for example, for a terminal device with better device performance, 20 game screenshots may be sent to the game server per second, and for a terminal device with poorer device performance, 10 game screenshots may be sent to the game server per second. For another example, the terminal device may send 30 game shots per second to the game server if the network quality is good, and only 5 game shots per second to the game server if the network quality is poor.

The game server may input each frame of the game screenshot to the object detection model, and the object detection model outputs an image recognition result of each frame of the game screenshot. Optionally, the game server may also extract a part of the image frames from the game screenshots sent by the terminal device for subsequent processing according to the processing capability, for example, the terminal device sends 40 game screenshots per second, and the game server may extract 20 game screenshots therein at a rate of 20 frames per second, input the 20 game screenshots into the object detection model, and output an image recognition result of each game screenshot by the object detection model.

The image recognition result comprises the category (such as 'fish school', 'turtle', 'gun platform', and 'prop') of the game element and the position in the game interface, and the game server can determine the simulation operation strategy (such as performing simulation click operation on the direction in which the 'turtle' is located) based on the image recognition result. The game server feeds back the simulation operation strategy to the terminal equipment, and the terminal equipment carries out simulation operation to realize automatic attack.

In the game testing scenario, the tester may preset an attack target according to the testing purpose, for example, the developer adds a new play of "grab the turtle and win the jackpot" to the original fishing game, and the game tester sets the attack target as "turtle" based on the new play. For testing authenticity, the game server may collect a large number of game shots, which are typically derived from visual data generated by a large number of real players during the game play. When the game server collects a certain number (for example, 100 ten thousand) of game screenshots, the simulation operation may be performed on the game server side or the test server side.

Assuming that the game server locally stores object detection models, after game screenshots fed back by the terminal device are collected, the game screenshots are input into the object detection models, and the object detection models output image recognition results of each game screenshot. The image recognition result includes the category (such as "fish school", "turtle", "gun platform", and "prop" etc.) and the position in the game interface of the game element, and the game server can determine a simulation operation strategy (such as a simulation click operation on the direction in which the "turtle" is located) based on the image recognition result. The game server feeds back the simulation operation strategy to the terminal equipment (which can be one terminal equipment or a plurality of terminal equipments simultaneously perform simulation operation) used by the tester, and the terminal equipment performs simulation operation to realize automatic attack. After a period of testing, the terminal device reports the simulation operation results to the game server (or the test server) uniformly, and the game server (or the test server) performs statistics to generate a game test report.

The developer can adjust the contents of game level, game task, game elements and the like according to the contents of the game test report until the expected effect of development is achieved.

It should be noted that the server in fig. 1 may be one server or a server cluster composed of multiple servers, or a cloud computing center, and the server may specifically be a game server or a test server. The terminal device may be a tablet computer, a notebook computer, a palm computer, a mobile phone, a Personal Computer (PC), a smart television, a voice interaction device, and the like shown in fig. 1.

The terminal device and the server can communicate with each other through a wireless network, a wired network or a removable storage medium. Wherein the wireless network described above uses standard communication techniques and/or protocols. The wireless Network is typically the Internet, but can be any Network including, but not limited to, bluetooth, a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a mobile, private Network, or any combination of virtual private networks. In some embodiments, custom or dedicated data communication techniques may be used in place of or in addition to the data communication techniques described above. The removable storage medium may be a Universal Serial Bus (USB) flash drive, a removable hard drive or other removable storage medium, and the like.

Although only five terminal devices and one server are shown in fig. 1, it should be understood that the example in fig. 1 is only used for understanding the present solution, and the number of the specific terminal devices and the number of the servers should be flexibly determined according to actual situations.

The scheme adopted by the embodiment of the application is realized based on an Artificial Intelligence (AI) technology, and some basic concepts in the field of artificial intelligence are introduced below. AI is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making. The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

In the process of simulation operation and game testing, an object detection model is required to be used, wherein the object detection model is obtained based on Machine Learning (ML) training, the artificial intelligence technology is developed and researched in various directions along with the research and progress of the artificial intelligence technology, and the Machine Learning is a multi-field cross subject and relates to multi-subjects such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach to make computers have intelligence, and is applied in various fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formal education learning.

Computer Vision technology (CV) is involved in identifying elements in the game shots using an object detection model. The computer vision technology is the science of researching how to make a machine see in various research directions of the artificial intelligence technology, and further refers to the science of using a camera and a computer to replace human eyes to carry out machine vision such as identification, tracking and measurement on a target and further carrying out graphic processing, so that the computer processing becomes an image which is more suitable for human eyes to observe or is transmitted to an instrument to detect. As a scientific discipline, computer vision research-related theories and techniques attempt to build artificial intelligence systems that can capture information from images or multidimensional data. Computer vision techniques typically include image processing, image Recognition, image semantic understanding, image retrieval, optical Character Recognition (OCR), video processing, video semantic understanding, video content/behavior Recognition, three-dimensional object reconstruction, 3D techniques, virtual reality, augmented reality, synchronized positioning and mapping, and the like.

Taking the fishing game as an example, how to simulate a real player to perform operations in the fishing game will be described, referring to fig. 2, fig. 2 is a schematic flow chart of a simulation operation method in an embodiment of the present application, and as shown in the figure, a model training flow based on worker intelligence provided in the present application may be divided into four parts, namely, acquiring a game image, recognizing a game image, determining a target operation position, and performing a simulation operation. The following will describe functions and flows of the respective parts, specifically:

in step A1, the terminal device intercepts the game image in real time, for example, the intercepted game image may be an image in a game recorded video, or may be a picture in the process of a game. The terminal device uploads the collected game images to a server, wherein the server can be a game server or other servers, and the specific type of the server is not limited;

in step A2, the server first needs to determine whether the game image uploaded by the terminal device satisfies an image recognition condition, and if the game image satisfies the image recognition condition, the server inputs the game image into a trained object detection model, and outputs a corresponding image recognition result through the object detection model, where the image recognition result includes category information and position information of game elements in the game image. On the contrary, if the game image does not meet the image recognition condition, directly discarding the game image;

in step A3, the server determines the target operation position according to the image recognition result and by combining with the actual task, for example, if the task is "capture 10 turtles", the server needs to find the position of the "turtles" according to the image recognition result, and the position is the target operation position. For another example, if the task is "capture 1 shark", the server needs to find the location of the "shark" according to the image recognition result, and the location is the target operation location;

in step A4, the server sends the target operation position to the terminal device, and the terminal device performs a simulation operation based on the target operation position, for example, a simulation click on the position of the "turtle" or a simulation click on the position of the "shark".

Further, for convenience of understanding, please refer to fig. 3, fig. 3 is a schematic diagram of an artificial intelligence operation flow implemented based on a fishing game in the embodiment of the present application, as shown in the figure, a fishing game installed on a mobile phone is taken as an example, in the figure, B1 is a fishing game operated on a mobile phone, B2 is a game image captured from the fishing game, and B3 is an artificial intelligence services (AISvr) module. The AISVr module is deployed in a server, is connected with the terminal equipment, and is used for processing game images, determining a simulation operation strategy and feeding back the simulation operation strategy to the terminal equipment.

Specifically, the AISVr module establishes communication connection with the terminal device, and then obtains a game image from the terminal device, wherein the game image can be an image captured in a game recording video or an image captured in real time during game playing, the AISVr module determines an image recognition result by calling an object detection model, and then determines a target operation position according to the image recognition result, and then the AISVr module sends the determined target operation position to the terminal device, so that the terminal device can perform simulation operation according to the target operation position, for example, forward fishing net launching, prop using prop, gun platform upgrading, reward receiving and other operations are performed, and thus the operation of a real player can be better simulated. The method flow of the simulation operation provided in the embodiment of the present application is also applicable to a game test scenario, and details are not repeated here.

With reference to fig. 4, fig. 4 is a schematic diagram illustrating an implementation of multi-module calling based on simulation operation in this embodiment of the present application, as shown in the figure, the modules related to this embodiment of the present application are functional modules, that is, each module is used to implement a corresponding function, and communication can be performed between the modules, which will be described in detail below:

the modules referred to in fig. 4 include a data source module C1, an input/output (I/O) module C2, a service Manager (MC) module C3, a User Interface (UI) module C4, a game recognition (game reg) module C5, and an AI module C6. Wherein, the GameReg module C5 is mainly used for identifying the category and the position of each game element in the game image, the AI module C6 is mainly used for generating a simulation operation strategy, the data source module C1 is used for providing the game image, and for example, the data source module C1 is used for extracting the game image in the fishing game. The input/output module C2 communicates with the data source module C1, and distributes the game image of the data source module C1 to the MC module C3. The MC module C3 is mainly used for data forwarding, game state monitoring, and service management.

Then, the MC module C3 forwards the game image to the UI module C4, and the UI module C4 detects a current game running state of the game image (for example, a game state before the game play, a game state in the game play, or a game state at the end of the game play) based on the game image. The UI module C4 feeds back the current game running state to the MC module C3, and outputs a game interface image to the MC module C3 in real time. The MC module C3 feeds back the game interface image to the IO module C2, and the IO module C2 forwards the game interface image to the data source module C1, so that the game interface image is displayed. Wherein, the game interface image has no action of simulation operation.

When the game running state corresponding to the game image is the game state in the game play, the MC module C3 sends the game image to the GameReg module C5, the GameReg module C5 obtains an image recognition result based on the game image and sends the image recognition result to the AI module C6, the AI module C6 determines a target operation position and sends the target operation position to the MC module C3, the MCC3 feeds the target operation position back to the data source module C1 through the IO module C2, and the data source module C1 performs simulation operation according to the target operation position. The above process only describes the flow of one simulation operation, and in practical application, the above flow should be executed circularly until the game is finished, or the user stops the automatic game, or the game test is finished.

In the following, with reference to fig. 5, fig. 5 is a schematic diagram of an embodiment of a method for simulating an operation in an embodiment of the present application, and as shown in the drawing, an embodiment of the method for simulating an operation in an embodiment of the present application includes:

101. the server receives a game image sent by the client;

in this embodiment, the server receives a game image sent by the client. The game image can be an image captured in real time when the game is played, an image captured in a video recorded in the game, or a synthesized image. For convenience of understanding, taking the fishing game as an example for description, please refer to fig. 6, fig. 6 is a schematic diagram of a game interface based on the fishing game according to an embodiment of the present application, as shown in the drawing, fig. 6 (a) illustrates a game interface corresponding to a fishing game in which a plurality of game players participate, the game interface includes 4 turrets, which can be played by 4 players at the same time, and the game interface may further include received bonus information and mission information, for example, the bonus information is "click to receive 2000 coins", and the mission information is "please capture 15 turtles in the next 3 minutes".

Fig. 6 (B) illustrates a game interface for a fishing game involving a single player, the game interface having only one turret at which the player controls to launch an attack on a game element in the game interface. Similarly, the game interface may also include picked-up bonus information as well as mission information. It should be understood that the example in fig. 6 is only used for understanding the present solution, and the specific game interface should be flexibly determined in combination with the actual situation. Based on the description in fig. 6, the game image is a screenshot of the game interface.

It should be noted that, the client specifically refers to an application installed on the terminal device, for convenience of introduction, the client is taken as a game client for example, and the game client may be a client of a fishing game, and it is understood that the client may also be other types of application clients, and the clients are all installed on the terminal device.

102. If the game image meets the image recognition condition, the server obtains an image recognition result of the game image through the object detection model, wherein the image recognition result comprises category information of at least two detection images and position information of the at least two detection images;

in this embodiment, the server usually receives many game images, for example, 10 ten thousand game images, however, some of the game images may not satisfy the image recognition condition, and if the image recognition condition is not satisfied, the corresponding game images are directly removed. Finally, the server screens out game images satisfying the image recognition condition, for example, 8 ten thousand game images satisfying the image recognition condition are selected from the hundred thousand game images. Then, the server inputs the game images meeting the image recognition condition to the trained object detection model, and the object detection model outputs the image recognition result corresponding to each game image, wherein the image recognition result comprises the category information of at least two detection images and the position information of at least two detection images.

For easy understanding, please refer to fig. 6 (B), which is a game image captured in real time, that is, the game image includes a fortunes, sharks, and turtles, and if the task information is "please capture 15 turtles in the next 3 minutes", the object to be detected is "turtles". The game image is input to a trained object detection model, and a detection image A of which the type information is "gun platform" in the game image, and the position information of the detection image A are output through the object detection model. Similarly, a detection image B of the type information "turtle" in the game image, and position information of the detection image B are output by the object detection model. A detection image C of "shark" as type information in the game image, and position information of the detection image C are output by the object detection model. The position information may be expressed as (x, y, w, h) or (x, y), x representing the upper left x coordinate of the detection image, y representing the upper left y coordinate of the detection image, w representing the image width of the detection image, and h representing the image height of the detection image. It is understood that, in practical applications, the x and y in the position information may also be coordinates of a lower left corner, an upper right corner, a lower right corner or a center point of the detected image.

The image recognition condition may be to determine whether the current game image is in a game running state, for example, when it is recognized that the game image corresponds to a game running state of "a state in game play", it may be determined that the game image satisfies the image recognition condition. The image recognition condition may be to determine whether the game type of the current game image is a target game type, and if it is determined that the game image belongs to an image in a fishing game, it indicates that the game image satisfies the image recognition condition, assuming that the target game type is a fishing game type.

Specifically, for a game image satisfying the image recognition condition, the server may input the game image to the object detection model, thereby obtaining an image recognition result of the game image. In this embodiment, the object detection model may adopt a Convolutional Neural Network (CNN) structure, or an end-to-end network (yoly look once, YOLO) V3 network structure, or a network with other structures, which is not limited herein.

For convenience of description, the fishing game is taken as an example for description, please refer to fig. 7, fig. 7 is a schematic flow chart of the image recognition result obtained based on the game image in the embodiment of the present application, as shown in the figure, specifically:

in step D1, the server obtains a game image, that is, a game image shown in fig. 7 (a), where the game image may be an image captured by the terminal device in real time while the game is in progress, an image captured in a recorded video of the game, or a historical game image stored in advance in a local area of the server.

In step D2, the trained object detection model is called, the game image is detected by using the object detection model, and a corresponding image recognition result is output. The training process of the object detection model will be described below.

Firstly, a large number of game sample images need to be collected, the server establishes a communication connection with at least one terminal device, and assuming that an android system (android system) is used by the terminal device, the server acquires a fishing game image running on the terminal device in real time through an Android Debug Bridge (ADB) command, and performs manual annotation on the acquired fishing game image, where the game sample image includes category information and position information corresponding to a detection image in the fishing game image, that is, as shown in fig. 7 (a), the game sample image is referred to fig. 8 for ease of understanding, fig. 8 is an embodiment of annotating the game sample image in the embodiment of the present application, and as shown in fig. 8 (a), the game sample image including a plurality of gun platforms, that is, the game sample image corresponding to a multiplayer game. The game sample image includes labeled information, for example, labeled category information of "turtle", "gun platform", and "shark". Fig. 8 (B) shows a game sample image of a single gun barrel, that is, a game sample image corresponding to a single-player game. The game sample image also includes labeled information, for example, labeled category information is "turtle" and "fort".

In addition, it is necessary to label the position information, for example, the position information obtained by labeling the "turtle" shown in fig. 8 (a) is (30, 50), and the position information obtained by labeling the "turtle" shown in fig. 8 (B) is (110, 45). The information marking process can be realized by adopting a manual marking mode.

Then, training an object detection model to be trained according to the obtained game sample image, so as to obtain the trained object detection model, specifically, adopting an end-to-end network (you only look once, YOLO) V3 network training object detection model, and modifying a network description file and category information corresponding to the YOLOV3 network, for example, if the network description file is 'capture turtle 15', the corresponding category information is 'turtle'. For another example, the category information includes "turtle" and "fort", and the number of categories is 2.

Therefore, the marked game sample image can be used for training the object detection model, when the training condition is met, the model parameters of the object detection model can be obtained, and the model parameters are stored in a network weight file form. It is understood that in actual training, the corresponding training times are generally not less than 10000 iterations. Referring to fig. 9, fig. 9 is a schematic diagram of a structure of an end-to-end network in an embodiment of the present application, where as shown in the figure, "× 1" indicates that the structure is repeated 1 time, "× 2" indicates that the result is repeated 2 times, "× 4" indicates that the result is repeated 4 times, and "× 8" indicates that the result is repeated 8 times. In the YOLOV3 network structure, a convolution (convolution) layer and a residual (residual) layer are included, and the YOLOV3 adopts three feature maps with different scales to detect objects, so that more fine-grained features can be detected. The three final output scales of the network are respectively 1/32,1/16 and 1/8, wherein after several convolution operations, a prediction result of 1/32 is obtained, the downsampling multiple is higher, and the characteristic image receptive field is larger at the moment and is suitable for detecting an object (such as 'shark') with larger size in the game image (or the game sample image). The 1/32 prediction is up-sampled and then concatenated (concat) with the 1/16 prediction after several convolution operations, so that the object detection model can detect the medium-sized object (such as a "turtle"). Finally, 1/16 of the prediction results are up-sampled and then concat operation is carried out with 1/8 of the prediction results after several convolution operations, so that the object detection model can detect small-sized objects (such as 'small fish groups').

In step D3, the server calls the trained object detection model, and the object detection model outputs the image recognition result of the game image, specifically, taking the object detection model as a YOLOV3 network structure as an example, first, an initialization interface function of the object detection model needs to be called, then, the object detection model is loaded, and finally, the game image is input by calling the prediction interface function, and the content output by the object detection model is the image recognition result, that is, the image recognition result is shown in (B) of fig. 7.

103. The server determines a target operation position according to the image recognition result, wherein the target operation position comprises at least one piece of position information to be operated;

in this embodiment, the server determines a target operation position according to the image recognition result, where the target operation position includes at least one piece of to-be-operated position information. Specifically, the image recognition result includes category information of at least two detection images and position information corresponding to each detection image, for example, the two category information included in the image recognition result are "fort" and "turtle", respectively, the position information corresponding to the "fort" is (80, 80), and the position information corresponding to the "turtle" is (85, 40), so that one to-be-operated position information included in the target operation position may be the position of the "fort", that is, (80, 80), or the one to-be-operated position information included in the target operation position may be the position of the "turtle", that is, (85, 40).

104. And the server sends the target operation position to the client so that the client carries out simulation operation according to the target operation position.

In this embodiment, the server sends the target operation position to the client, and the client performs the simulation operation according to the target operation position. Specifically, after the client acquires the target operation position, based on at least one to-be-operated position information included in the target operation position, a simulated click is performed on the corresponding position, and if the current task is to capture a turtle, but the current interface does not detect the turtle, a random click operation can be performed, that is, an attacked object is identified and then an attack is directly performed. In order to avoid the click invalidation due to the random click, a range of the random click may be set, for example, the clickable range is 60 percent of the area in the game image, or the width of the clickable range is 0.25 × the width of the game image, and the height of the clickable range is 0.75 × the height of the game image. In practical application, the attack can also be performed only on the target object, for example, if the current task information is "capture turtle", the attack is performed only on the position information (85, 40) corresponding to the "turtle".

Optionally, on the basis of the foregoing embodiments corresponding to fig. 5, in an optional embodiment of the simulation operation method provided in this embodiment of the application, after the server receives the game image sent by the client, the method may further include:

the server acquires running state information corresponding to the game image;

if the running state information is the game-in-game state, the server determines that the game image meets the image recognition condition;

if the running state information is the pre-game state, the server determines that the game image does not meet the image identification condition;

if the running state information is a match-up ending state, the server determines that the game image does not satisfy the image recognition condition.

In this embodiment, after receiving the game image sent by the client, the server may first obtain the running state information corresponding to the game image. If the running state information is the game-in-game state, the server determines that the game image meets the image recognition condition, and then the server inputs the game image meeting the image recognition condition into the object detection model, and the object detection model performs subsequent prediction. However, if the running state information is the pre-match state or the match-up end state, the server determines that the game images do not satisfy the image recognition condition, and then the server does not predict the game images any more, and can directly reject the game images which do not satisfy the image recognition condition, or store the game images which do not satisfy the image recognition condition for training models under other tasks.

Specifically, the server may identify the corresponding operation state information of the game image by using an image identification technology, where the image identification technology is capable of identifying a key image in the game image, the key image refers to an image with a key feature, and since each game image has a key feature, for example, there are words such as "start" in the game interface before the game is paired, words such as "end" in the game interface after the game is paired, and the game interface during the game is paired has images such as items or fish.

In addition, the server may also obtain the running state information of the game image through the identifier reported by the client, for example, a "start" button is displayed in the game interface of the game match-up, when the player triggers the "start" button, the client immediately feeds back an identifier of the game start to the server, the server receives the identifier of the game start at frame 60, and then the game images collected from frame 60 are all images meeting the image recognition condition, that is, the game images are all images in the game match-up process. The method comprises the steps that a quit button is displayed on a game interface in the game match process, when a player triggers the quit button, a client immediately feeds back a game ending identifier to a server, the server receives the game ending identifier in the 6000 th frame, and therefore the collected game images are all images which do not meet image recognition conditions after the 6000 th frame, namely the game images are images after the game match is ended.

Secondly, in the embodiment of the application, a mode for selecting game images based on image recognition conditions is provided, through the mode, the game images capable of being subjected to subsequent processing can be screened from a large number of game images, and for the game images in a non-game-play state, subsequent prediction is not needed, so that computing resources are saved, and prediction efficiency is improved.

Optionally, on the basis of the foregoing respective embodiments corresponding to fig. 5, in another optional embodiment of the simulation operation method provided in this embodiment of the present application, the determining, by the server, the target operation position according to the image recognition result may include:

the server acquires M first detection images from the at least two detection images according to the category information of the at least two detection images, wherein the first detection images belong to an attacked category, and M is an integer greater than or equal to 1;

the server randomly determines a target detection image from the M first detection images;

and the server determines the position information corresponding to the target detection image as the position information to be operated.

In this embodiment, after the server acquires the category information of at least two detection images, the server acquires M first detection images according to the category information. It is understood that, taking the fishing game as an example, the detected image may be an image corresponding to the category information "fort", or an image corresponding to the category information "shark", or an image corresponding to the category information "gold coin", wherein, of these category information, "shark" can be captured, and neither the "fort" nor the "gold coin" can be operated in an attack manner. Then, the detection image corresponding to the shark whose category information is "shark" is determined as a first detection image, and the first detection image belongs to the attacked category. In practical applications, the attacked category includes, but is not limited to, a category whose category information is "shark", a category whose category information is "turtle", and a category whose category information is "fish school". And the server randomly determines a target detection image from the M first detection images and then determines the position information corresponding to the target detection image as the position information to be operated.

Specifically, for convenience of understanding, please refer to the game interface shown in fig. 6 (a), which is a game image obtained after screenshot, where the game image includes 4 detection images with category information of "fort stage", 1 detection image with category information of "shark", 1 detection image with category information of "turtle", and 5 detection images with category information of "small fish", that is, 11 detection images in total, where "shark", "turtle", and "small fish" all belong to the attacked category, that is, 7 first detection images can be obtained (that is, M =7 at this time), and therefore, a target detection image can be randomly determined from the 7 first detection images. If the first detection image corresponding to the shark is determined as the target detection image, the position information corresponding to the target detection image containing the shark is determined as the position information to be operated, and if the first detection image corresponding to a certain sea turtle is determined as the target detection image, the position information corresponding to the target detection image containing the sea turtle is determined as the position information to be operated.

It can be understood that in the fishing game, the game scores corresponding to different categories of information are different, and are generally related to the difficulty of game setting, for example, the probability of occurrence of a "turtle" as category information is higher, which can result in 200 points when capturing a "turtle", and the probability of occurrence of a "shark" as category information is lower, which can result in 1000 points when capturing a "shark". Based on this, when generating the simulation operation strategy, the object with larger score can be attacked preferentially.

Secondly, in the embodiment of the application, a method for determining the position information to be operated in a task-free scene is provided, through the method, when an attack object is not specified, an attacked target is randomly selected, and then the client automatically attacks the target. Therefore, the game player can be attacked purposefully without waiting when no task is available, so that the game yield is improved, and the rationality of automatic attack is enhanced.

the method comprises the steps that a server obtains M first detection images and M second detection images from at least two detection images according to category information of the at least two detection images, wherein the first detection images belong to an attacked category, the second detection images belong to a task category, and M is an integer larger than or equal to 1;

the server determines a matching result according to the template image to be matched and the second detection image, wherein the template image to be matched belongs to any one template image in a template image set, and the template image set comprises at least one template image;

if the matching result is that the matching is successful, the server determines target type information corresponding to the second detection image;

and the server determines the position information to be operated according to the target type information and the M first detection images.

In this embodiment, after acquiring the category information of at least two detection images, the server acquires M first detection images and M second detection images from the at least two detection images according to the category information. It is understood that, taking the fishing game as an example, the detected image may be an image corresponding to the category information "fort", or an image corresponding to the category information "shark", or an image corresponding to the category information "turtle", wherein, of these category information, "shark" and "turtle" can be captured, but the "fort" cannot be operated in an attack manner. Then, the detected image corresponding to the class information "shark" is determined as the first detected image, and the detected image corresponding to the class information "turtle" is also determined as the first detected image, that is, the first detected image belongs to the attacked class. In addition, a second inspection image may be detected in the task bar, and task information may appear in the task bar, such as "please capture 15 turtles in the next 3 minutes", while an image related to the task information, i.e., the second inspection image, is displayed. And the server matches the second detection image so as to determine the target type information corresponding to the second detection image. And finally, determining the information of the position to be operated according to the target type information and the M first detection images.

For convenience of understanding, referring to fig. 10, fig. 10 is a schematic view illustrating an embodiment of identifying a second detection image based on a task scene in the embodiment of the present application, as shown in fig. 10 (a), a game image shown in fig. 10 includes a second detection image E1 belonging to a task category, a detection image E2 having category information of "turtle" and a detection image E3 having category information of "platform", where the detection image E3 having category information of "platform" belongs to an attacked category, and thus 1 first detection image can be obtained. Then, the server obtains a template image to be matched from the template image set, where the template image to be matched refers to any one template image in the template image set, for example, the template image to be matched is shown as a "shark" template image in (B) in fig. 10, the task content is "3 pieces of captured sharks", the template image to be matched is shown as a "turtle" template image in (C) in fig. 10, the task content is "15 pieces of captured turtles", the template image to be matched is shown as a "fort platform" template image in (D) in fig. 10, and the task content is "upgrade platform to level 10". And matching each template image with the second detection image respectively to obtain corresponding matching results respectively.

Two possible matching approaches are described below:

the first mode is that the template image to be matched is assumed to be a shark template image, the shark template image is matched with the second detection image, the matching result is a score A, and the score A is assumed to be larger than a preset threshold similarity threshold value, which indicates that the second detection image is successfully matched with the shark template image, namely the target type information of the second detection image is shark. And the second detection image is not matched by using other template images, so that the computing resource consumed by matching is saved.

The second way is that, assuming that the template image to be matched is a shark template image, the shark template image is matched with the second detection image, and the matching result is a score A. And if the template image to be matched is the turtle template image, matching the turtle template image with the second detection image to obtain a matching result, namely a score B. And if the template image to be matched is a 'gun platform' template image, matching the 'gun platform' template image with the second detection image to obtain a matching result which is a score C. Then, the maximum value is selected by comparing the score a, the score B and the score C, for example, if the score B is the maximum value, it is determined whether the score B is greater than or equal to a score threshold, if the score B is greater than or equal to the score threshold, the target type information of the second detection image is "turtle", and if the score B is less than the score threshold, it is determined that the current game is a no-mission scene.

Optionally, in practical applications, the template image may include not only picture information but also text information. And if the text information of the template image to be matched is 'capturing 3 sharks', and the second detection image extracted from the task bar comprises 'capturing 15 chelonian' text information, the matching is failed. Similarly, if the text information of the template image to be matched is "capture the turtle 15", and the text information extracted from the task bar in the second detection image includes "capture the turtle 15", the matching is successful. Therefore, the target type information corresponding to the second detection image can be determined to be the turtle, and then the corresponding simulation operation strategy is generated according to the target type information and the M first detection images.

Secondly, in the embodiment of the application, a method for determining the position information to be operated in a task scene is provided, and by the method, a target can be attacked in a directive manner according to a specific task type, the diversity of playing methods is continuously improved, and the task can be completed as far as possible within a limited time, so that the game income is improved, and the reasonability of the game is enhanced.

Optionally, on the basis of the foregoing embodiments corresponding to fig. 5, in another optional embodiment of the method for simulating an operation provided in the embodiments of the present application, the determining, by the server, a matching result according to the template image to be matched and the second detection image may include:

the server acquires at least one similar result from the second detection image in a sliding window mode by adopting the template image to be matched;

the server determines the maximum similar result according to the at least one similar result;

if the maximum similarity result is larger than or equal to the similarity threshold, the server determines that the matching result is successful;

and if the maximum similarity result is smaller than the similarity threshold, the server determines that the matching result is matching failure.

In this embodiment, the server obtains at least one similar result from the second detection image in the form of a sliding window by using the template image to be matched, and usually, a plurality of similar results are taken out, where the similar results may be represented in a score form from 0 to 1, or in a score form from 0 to 10, and the score form represented from 0 to 1 is taken as an example in this application. And the server acquires a maximum similarity result according to the similarity results, compares the maximum similarity result with a similarity threshold value, and determines that the matching result is successful if the maximum similarity result is greater than or equal to the similarity threshold value. And if the maximum similarity result is smaller than the similarity threshold, the server determines that the matching result is a matching failure. In this embodiment, the description is given by taking the similarity threshold as 0.7 as an example, and in practical application, other similarity thresholds may be set, for example, the similarity threshold is 0.6 or the similarity threshold is 0.8. And under the condition that the template image to be matched and the second detection image are failed to be matched, the server can continue to adopt other template images to be matched with the second detection image.

Specifically, the sliding window sampling manner is to slide from left to right with a fixed step size (for example, step size is 1 or step size is 2) starting from the top left corner of the second detection image, and take out one window sub-image each time the window sub-image is slid. And after the transverse result in the second detection image is taken, sliding downwards for a certain step length, then continuously sliding from left to right with a fixed step length, and taking out one window sub-image in each sliding process until all the window sub-images in the second detection image are taken out. Alternatively, the second detection image may be slid from top to bottom in a fixed step (for example, step 1 or step 2) from the top left corner, and one window sub-image is taken out each time the sliding is performed. And after the longitudinal window sub-images in the second detection image are taken, sliding to the right for a certain step length, then continuing sliding from top to bottom with the fixed step length, and taking out one window sub-image each time until all the window sub-images in the second detection image are taken out.

The similarity of the template image to be matched and the window sub-image can be sequentially calculated in a sliding window mode. The size of the window sub-image is the same as that of the template image to be matched, for example, the size of the template image to be matched with the category information of the "turtle" is 30 × 50, that is, the size of the window sub-image is 30 × 50, and for the template image to be matched with the category information of the "gun platform", the size of the template image to be matched is 30 × 30, that is, the size of the window sub-image is also 30 × 30.

For convenience of understanding, the template image to be matched of the "turtle" is taken as an example for explanation, please refer to fig. 11, fig. 11 is a schematic view of an embodiment of the present application that obtains a similar result in the form of a sliding window, and as shown in the figure, it is assumed that the template image to be matched of the "turtle" is 30 × 50, and the window sub-image is also 30 × 50. As shown in fig. 11 (a), the window sub-image F1 may slide from the top left corner of the second detection image F2 along the X direction by a certain step length from left to right, slide to the boundary, slide by a certain step length along the Y direction, and then continue to slide along the X direction by a certain step length from left to right.

Assuming that a total of 9 window sub-images are extracted from the second detection image, then a formula is used to calculate the similarity between each window sub-image and the template image to be matched, each window sub-image corresponds to a similarity result, the similarity results are stored in a result matrix, and the result matrix shown in (B) in fig. 11 can be obtained by taking 9 similarity results as an example, and based on the result matrix, the similarity results are "0.77", "0.75", "0.33", "0.89", "0.99", "0.80", "0.55", "0.78", and "0.56", respectively. And then finding a maximum similarity result in the result matrix, wherein the maximum similarity result represents the best matching result between the second detection image and the template image to be matched. If the partial image of the second detection image is more similar to the template image to be matched, the corresponding similarity result is also larger, and therefore, the maximum value of the similarity results in the result matrix is the maximum similarity result (the best matching value). Based on this, the maximum similarity result can be determined to be "0.99" from the 9 similarity results. And after the maximum similarity result is obtained, comparing the maximum similarity result with a similarity threshold, and if the maximum similarity result is greater than or equal to the similarity threshold, determining that the matching result is successful, namely determining that the second detection image comprises the same type information as the template image to be matched. If the maximum similarity result is smaller than the similarity threshold, the matching result is determined to be a matching failure, the second detection image is considered not to include the category information indicated by the template image to be matched, and then other template images in the template image set can be continuously used for matching with the second detection image.

Four ways for calculating the similarity result will be described below, including a square error matching method, a standard square error matching method, a correlation matching method, and a standard correlation matching method, respectively, and taking the square error matching method as an example, the following formula is adopted to calculate the similarity result:

wherein, R (x, y) represents a similar result, T represents a template image to be matched, I represents a second detection image, x represents the abscissa of the second detection image, y represents the ordinate of the second detection image, x 'represents the abscissa of the template image to be matched, and y' represents the ordinate data of the template image to be matched.

Taking the standard square error matching method as an example, the following formula is adopted to calculate the similarity result:

The correlation difference matching method adopts the product of each pixel in the template image to be matched and the second detection image as the matching result, the larger the result value is, the higher the matching degree is, and 0 represents the worst matching result. Taking the correlation matching method as an example, the following formula is adopted to calculate the similarity result:

Taking the standard correlation matching method as an example, the following formula is adopted to calculate the similarity result:

It will be appreciated that the accuracy of the standard correlation matching method and the standard squared error matching method is relatively high. In the standard correlation matching method, the higher the matching value is, the better the matching effect is. In the standard square error matching method, the lower the matching value is, the better the matching effect is, and the specific method should be flexibly determined by combining actual requirements.

In the embodiment of the application, a mode of generating the matching result by adopting a template matching method is provided, and through the mode, the similar result is obtained in a sliding window mode, the maximum similar result is determined, and the matching result is determined through the comparison of the maximum similar result and the similarity threshold.

Optionally, on the basis of the foregoing respective embodiments corresponding to fig. 5, in another optional embodiment of the simulation operation method provided in this embodiment of the present application, the determining, by the server, the target operation position according to the target type information and the M first detection images may include:

if the category information of the M first detection images has the target type information, the server acquires the target detection images from the M first detection images, wherein the target detection images correspond to the target type information;

In this embodiment, if the category information of the M first detection images includes the target type information, it indicates that there is an object that is preferentially attacked in the current game interface. Taking a fishing game as an example, in general, a target corresponding to the captured target type information can obtain a higher score or reward, and therefore, such objects are often attacked preferentially to obtain a higher game addition. When the category information of the M first detection images includes the target type information, the server acquires a target detection image from the M first detection images, where the target detection image is an image corresponding to the target type information, and finally determines the position information corresponding to the target detection image as the position information to be operated.

Specifically, assuming that the category information of the M first detection images includes "turtle", "shark", "treasure box", and "fort", the target type information corresponding to the second detection image is determined to be "turtle" by the method described in the foregoing embodiment, and therefore, the first detection image whose category information is "turtle" in the game scene needs to be preferentially found. In practical situations, a plurality of objects with the same category information may appear in the same game scene, for example, the target type information is "turtle", and the currently existing 3 first detection images with category information of "turtle" may determine the to-be-operated position information in the following manner.

The first mode is as follows: the server may randomly select one first detection image from the 3 first detection images as a target detection image, and determine position information corresponding to the target detection image as position information to be operated.

In a second way, the server obtains the linear distance between the 3 first detection images and the "platform", for example, the linear distance between the "first detection image of the turtle" a and the "platform" is 30, the linear distance between the "first detection image of the turtle" B and the "platform" is 50, the linear distance between the "first detection image of the turtle" C and the "platform" is 25, the shortest linear distance, that is, 25, is selected, the first detection image of the "turtle" C is determined as the target detection image, and the position information corresponding to the target detection image is determined as the to-be-operated position information.

The third mode is that after the server acquires 3 first detection images, it is first determined whether there is a skill at present, for example, the skill a is a skill for triggering multiple fishing nets simultaneously, if the skill a can be used, the server determines all the 3 first detection images as target detection images, determines position information corresponding to the target detection images as position information to be operated, and feeds back three position information to be operated to the client.

In the embodiment of the application, a mode of performing simulation operation on task information in a task scene is provided, and through the mode, when a target detection image corresponding to target type information is identified, operation on the target detection image is preferentially triggered, so that the task can be completed more efficiently, the game income is improved, and the reasonability of the game is enhanced.

Optionally, on the basis of the foregoing respective embodiments corresponding to fig. 5, in another optional embodiment of the simulation operation method provided in this embodiment of the present application, determining the target operation position according to the target type information may include:

and if the category information of the M first detection images does not have the target type information, the server randomly determines the position information to be operated from the preset range.

In this embodiment, if the category information of the M first detection images does not include the target type information, it indicates that there is no object that is preferentially attacked in the current game interface. Taking a fishing game as an example, in general, a target corresponding to the captured target type information can obtain a higher score or reward, and therefore, such objects are often attacked preferentially to obtain a higher game addition. And under the condition that the category information of the M first detection images does not have the target type information, the server randomly determines the position information to be operated from the preset range.

Specifically, when the game image does not have the detection image corresponding to the target type information, a random click operation may be performed, and in order to avoid touching some unknown keys due to the random click, a range that can be clicked may be preset, that is, a preset range may be obtained. Assuming that the size of the game image is 600 × 800, i.e., 800 × 800 and 600 × high, the width of the preset range may be 0.25 × 0.25=200, and the height of the preset range may be 0.1 × 0.1=60. I.e. the predetermined range is 60 x 200 in size.

For easy understanding, referring to fig. 12, fig. 12 is a schematic diagram illustrating an embodiment of randomly determining to-be-operated position information in the embodiment of the present application, and as shown in the figure, it is assumed that the object type information in the second detection image is recognized as "turtle", but there is no detection image with category information of "turtle" in the current game image, and therefore, the random click operation can be performed within the preset range indicated by G1. The position information to be operated may be randomly determined from the preset range in the following manner.

The first mode is as follows: a position at which a game element is just present is randomly selected within a preset range, for example, position information 1 corresponds to the first detection image with the category information of "turtle", position information 2 corresponds to the first detection image with the category information of "shark", and there may be no game element at the position indicated by the position information, for example, position information 3 falls in the game background region.

The second way is: and detecting the game elements in a preset range, and preferably initiating attack operation on the game elements. For example, the position information 4 corresponds to the first detection image whose category information is "turtle", the position information 5 corresponds to the first detection image whose category information is "shark", and the position information 6 corresponds to the first detection image whose category information is "small fish". Then, one piece of position information can be randomly determined from the position information 4, the position information 5, and the position information 6 as the position information to be operated.

In the embodiment of the application, another mode of performing simulation operation on task information in a task scene is provided, and in the above mode, when a target detection image corresponding to target type information is not identified, one piece of to-be-operated position information can be randomly selected in a specified range, that is, in the case of a task, even if an object specified in the task does not exist in a current game interface, simulation operation can be initiated, so that game benefits can be obtained at a certain probability, and therefore, the game benefits are improved, and the reasonability of a game is enhanced.

acquiring a third detection image from the at least two detection images according to the category information of the at least two detection images, wherein the third detection image belongs to the auxiliary function category;

In this embodiment, the server may further acquire a third detected image from the at least two detected images, the third detected image corresponding to a category of an auxiliary function, where the auxiliary function represents an operation function related to the game main operation. Taking a fishing game as an example, the main operation is to control a 'gun platform' to attack a fish school, and the auxiliary operation can be to upgrade the 'gun platform', or release skills, or pick up timing rewards, and the like. And the server determines a matching result based on the template image to be matched and the third detection image, wherein the template image to be matched belongs to any one template image in the template image set. And if the matching result is that the matching is successful, the server acquires the position information to be operated corresponding to the third detection image.

For convenience of understanding, please refer to fig. 13, where fig. 13 is a diagram illustrating an embodiment of identifying a third detection image based on a task scene according to the embodiment of the present application, as shown in fig. 13, a game image shown in (a) of fig. 13 includes a reward extraction H1, where the reward extraction H1 belongs to an auxiliary function category, and therefore, 1 third detection image can be obtained, so that the server obtains a template image to be matched from a template image set, where the template image to be matched refers to any one template image in the template image set, for example, the template image to be matched is a template image corresponding to "reward extraction", or the template image to be matched is a template image corresponding to "prop up available", or the template image to be matched is a template image corresponding to "platform upgrade". For example, the template image to be matched shown in fig. 13 (B) is "earning rewards", and if the third detection image is successfully matched with the template image to be matched is "earning rewards", it indicates that the third detection image is of the category of "earning rewards". If the third detection image fails to be matched with the template image to be matched of the reward claim, other template images can be continuously adopted for matching.

Optionally, in practical applications, the template image may include not only picture information but also text information. And if the text information of the template image to be matched is 'gun platform upgrade' and the third detection image comprises the text information of 'earning reward', the matching is failed. Similarly, if the text information of the template image to be matched is 'fortress upgrade', and the third detection image extracted from the task bar includes the text information of 'fortress upgrade', the matching is successful. The auxiliary function category corresponding to the third detected image may then be determined.

Secondly, in the embodiment of the application, a mode for performing simulation operation on the auxiliary function type is provided, through the above mode, when no target type exists in the image, the simulation operation is not stopped, but the information of the position to be operated is randomly determined from the preset range, so that the subsequent simulation operation is continuously performed, and the situation is closer to the real operation situation.

Optionally, on the basis of each embodiment corresponding to fig. 5, in another optional embodiment of the simulation operation method provided in the embodiment of the present application, sending the target operation position to the client, so that the client performs the simulation operation according to the target operation position, may include:

and the server sends the position information to be operated to the client so that the client triggers attack operation in the direction corresponding to the position information to be operated.

In this embodiment, after determining the location information to be operated, the server may send the location information to be operated to the client, so that the client triggers an attack operation in a direction corresponding to the location information to be operated.

Specifically, when the to-be-operated position information is position information corresponding to an attacked category, the client may perform simulated click operation based on the to-be-operated position information, for example, a fishing game, an attack is launched from the "fort" to a position indicated by the to-be-operated position information, and as seen from the game interface, the "fort" may launch a "fishing net", "laser", or "cannonball" to the position indicated by the to-be-operated position information, and if a target (for example, "shark", "turtle", or "treasure box") is hit, some special effects may be further presented on the game interface displayed by the client, for example, a firework is released or a prompt is popped up.

Furthermore, in the embodiment of the application, a mode of performing simulation operation by a client is provided, through the above mode, different to-be-operated position information is determined based on different intentions, and the client performs simulated click operation according to at least one to-be-operated position information, so that not only can rewards be automatically picked up and skills are released, but also automatic attack operation can be realized, thereby realizing the flexibility of operation, enriching the types of operation, promoting the diversity of playing methods, and being beneficial to increasing the viscosity of users.

Optionally, on the basis of each embodiment corresponding to fig. 5, in another optional embodiment of the simulation operation method provided in the embodiment of the present application, the server sends the target operation position to the client, so that the client performs the simulation operation according to the target operation position, which may include:

and the server sends the position information to be operated to the client so that the client triggers click operation according to the position information to be operated.

In this embodiment, after determining the position information to be operated, the server may send the position information to be operated to the client, so that the client triggers a click operation according to the position information to be operated.

Specifically, when the position information to be operated is the position information corresponding to the auxiliary function type, the client may perform simulated click operation based on the position information to be operated, taking a fishing game as an example, wherein the fishing game includes a plurality of auxiliary functions, for example, an upgrade operation may be performed by clicking a "fort", so that the power of the "fort" is stronger, and the number of fish schools captured at a time is larger. For another example, the reward can be received free of charge by clicking the treasure box every 10 minutes. For another example, clicking on the skill icon after the skill cooling time is over may trigger the corresponding skill. Therefore, the client can simulate and click the auxiliary functions according to the position information to be operated. After the auxiliary function is clicked, special effects such as releasing fireworks or popping up prompts can be presented on the game interface displayed by the client.

Furthermore, in the embodiment of the application, a mode of performing simulation operation by the client is provided, through the above mode, different to-be-operated position information is determined based on different intentions, and the client performs simulated click operation according to at least one to-be-operated position information, so that not only can automatic attack operation be realized, but also rewards can be automatically picked up, skills can be released, and thus the flexibility of operation is realized, the operation types are enriched, the diversity of playing methods is improved, and the increase of the viscosity of users is facilitated.

With reference to the above description, the method for testing a game in the present application will be described below, please refer to fig. 14, where fig. 14 is a schematic diagram of an embodiment of the method for testing a game in the embodiment of the present application, and as shown in the figure, the embodiment of the method for testing a game in the embodiment of the present application includes:

201. the server receives a game image to be tested sent by the client, wherein the game image to be tested belongs to a game image corresponding to the state in the game;

in this embodiment, the server receives a game image to be tested sent by the client. The game image to be tested can be an image intercepted in real time when the game is in progress, can also be an image intercepted in a game recording video, and can also be a synthesized image. In order to improve the efficiency of game testing, the game images to be tested used in the method belong to the game images corresponding to the game-play state, and for the game images in the non-game-play state, the game images can be removed in the process of data preprocessing.

202. The server acquires an image recognition result of the game image to be tested through the object detection model, wherein the image recognition result comprises category information of at least two detection images and position information of the at least two detection images;

in this embodiment, the server inputs the game image to be tested into the trained object detection model, and the object detection model outputs the image recognition result corresponding to the game image to be tested, for convenience of introduction, please refer to fig. 6 (B), the game interface is used as the game image to be tested, and the game image to be tested includes a detection image with type information of "fort stage" and a detection image with type information of "turtle". And outputting a detection image A with type information of 'gun platform' in the game image to be tested and position information of the detection image A through the object detection model. Similarly, a detection image B of the type information "turtle" in the game image to be tested, and position information of the detection image B are output through the object detection model. And outputting a detection image C with the type information of 'shark' in the game image to be tested and the position information of the detection image C through the object detection model. The position information may be expressed as (x, y, w, h) or (x, y), x representing the upper left x coordinate of the detection image, y representing the upper left y coordinate of the detection image, w representing the image width of the detection image, and h representing the image height of the detection image. It is understood that, in practical applications, the x and y in the position information may also be coordinates of a lower left corner, an upper right corner, a lower right corner or a center point of the detected image.

203. The server determines a target operation position according to the image recognition result, wherein the target operation position comprises at least one piece of position information to be operated;

in this embodiment, the server determines a target operation position corresponding to the game image to be tested according to the image identification result corresponding to the game image to be tested, where the target operation position includes at least one piece of information about the position to be operated. Specifically, the image recognition result includes category information of at least two detection images and position information corresponding to each detection image, for example, the two category information included in the image recognition result are "fort stage" and "turtle", respectively, the position information corresponding to the "fort stage" is (80, 80), and the position information corresponding to the "turtle" is (85, 40), so that one to-be-operated position information included in the target operation position may be a position of the "fort stage", that is, (80, 80), or one to-be-operated position information included in the target operation position may be a position of the "turtle", that is, (85, 40).

204. The server sends the target operation position to the client so that the client carries out simulation operation according to the target operation position;

in this embodiment, the server sends the target operation position to the client, and the client performs the simulation operation according to the target operation position. Specifically, after the client acquires the target operation position, based on at least one to-be-operated position information included in the target operation position, a simulated click is performed on the corresponding position, and if the current task is to capture a turtle, but the current interface does not detect the turtle, a random click operation can be performed, that is, an attacked object is identified and then an attack is directly performed. In order to avoid the click invalidation due to the random click, a range of the random click may be set, for example, the clickable range is 60 percent of the area in the game image, or the width of the clickable range is 0.25 × the width of the game image, and the height of the clickable range is 0.75 × the height of the game image. In practical applications, the attack can also be performed only on the target object, for example, if the current task information is "capture turtle", the attack is performed only on the position information (85, 40) corresponding to the "turtle".

205. The server receives a simulation operation result sent by the client;

in this embodiment, after performing the simulation operation according to the target operation position, the client may also send a simulation operation result to the server, for convenience of understanding, please refer to table 1, where table 1 is an illustration of the simulation operation result.

TABLE 1

Image identification	Capture of randomness	Directional capture	Preset range catch
				1	Successful	Successful	Failure of
2	Failed by	Successful	Failure of
				3	Failure of	Successful	Failed by
4	Failed by	Failed by	Successful
				5	Successful	Failure of	Failed by
6	Failure of	Successful	Successful
				7	Successful	Failure of	Failure of
8	Successful	Failed by	Failed by
				9	Failure of	Successful	Successful
10	Successful	Failed by	Failure of

As can be seen from table 1, the client may perform simulation and statistics for different simulation operations, where the randomness capturing manner is that, in a task-free scene, the server randomly selects one assaultable object from each game image to be tested, each assaultable object corresponds to one target operation position, and the client initiates an attack based on the target operation position and records an attack situation. Under the scene with a task, and each game image to be tested has an attack object specified by the task, the server respectively selects the attack object specified by the task from each game image to be tested, each attack object corresponds to a target operation position, the client side launches an attack based on the target operation position, and the attack condition is recorded. In a scene with a task, but an attack object specified by the task does not exist in each game image to be tested, the server randomly selects a target operation position from each game image to be tested, the client initiates an attack based on the target operation position, and the attack condition is recorded.

206. And the server generates a game test report according to the simulation operation result.

In this embodiment, the server outputs a corresponding game test report according to the simulation operation result generated by the client. Continuing with the example of Table 1, in the task-free scenario, the success rate of capture is 50%. Under the scene with tasks, when attack objects appointed by the tasks exist in each game image to be tested, the capturing success rate is 50%. Under the scene with tasks, when the attack objects appointed by the tasks do not exist in each game image to be tested, the capturing success rate is 30 percent. It is understood that the simulation results shown in table 1 are merely illustrative. Specifically, the simulation operation result may further include other indexes, for example, an accuracy of determining the target operation position based on the image recognition result. Therefore, the game test report can include the image recognition result, the target operation position and the simulation operation result, and the developer can adjust the game content in time based on the game test report.

The embodiment of the application provides a game testing method, which comprises the steps of firstly receiving a game image sent by a client, obtaining an image recognition result of the game image through an object detection model when the game image meets an image recognition condition, determining a target operation position according to the image recognition result, wherein the target operation position comprises at least one piece of position information to be operated, and finally sending the target operation position to the client so that the client can carry out simulation operation according to the target operation position. The client feeds back the simulation operation result to the server, and the server generates a game test report according to the simulation operation result. In the above manner, the game image corresponding to the game state of the client in the game play is used for identifying the category information and the position information of the game element, and the target operation position is determined based on the information, so that the client performs the simulation operation, thereby enhancing the controllability of the operation. In addition, the simulation operation result is determined through the simulation operation, and the game test report generated based on the simulation operation result can reflect the authenticity of the simulation operation, and the simulation operation can be adjusted through the game test report, so that the authenticity of the simulation operation is improved. Secondly, the automatic game testing can assist the tester to automatically catch fish or other actions by simulating the interaction between the player and the game, thereby saving the testing time and the testing cost.

Referring to fig. 15, fig. 15 is a schematic view of an embodiment of the analog operation device in the embodiment of the present application, and as shown in the drawing, the analog operation device 30 includes:

a receiving module 301, configured to receive a game image sent by a client;

an obtaining module 302, configured to obtain, through the object detection model, an image recognition result of the game image received by the receiving module if the game image meets an image recognition condition, where the image recognition result includes category information of at least two detection images and location information of the at least two detection images;

a determining module 303, configured to determine a target operation position according to the image recognition result obtained by the obtaining module, where the target operation position includes at least one piece of to-be-operated position information;

and a sending module 304, configured to send the target operation position determined by the determining module to the client, so that the client performs a simulation operation according to the target operation position.

Alternatively, on the basis of the embodiment corresponding to fig. 15, in another embodiment of the analog operation device 30 provided in the embodiment of the present application,

the obtaining module 302 is further configured to obtain running state information corresponding to the game image after the receiving module receives the game image sent by the client;

the determining module 303 is further configured to determine that the game image meets the image recognition condition if the running state information is the game-in-game state;

the determining module 303 is further configured to determine that the game image does not satisfy the image recognition condition if the running state information is the pre-game state;

the determining module 303 is further configured to determine that the game image does not satisfy the image recognition condition if the running state information is the game-play end state.

a determining module 303, configured to obtain M first detection images from the at least two detection images according to the category information of the at least two detection images, where the first detection image belongs to an attacked category, and M is an integer greater than or equal to 1;

a determining module 303, configured to obtain M first detection images and M second detection images from the at least two detection images according to category information of the at least two detection images, where the first detection image belongs to an attacked category, the second detection image belongs to a task category, and M is an integer greater than or equal to 1;

a determining module 303, configured to obtain at least one similar result from the second detection image in the form of a sliding window by using the template image to be matched;

the determining module 303 is specifically configured to, if the category information of the M first detection images includes target type information, acquire a target detection image from the M first detection images, where the target detection image corresponds to the target type information;

the determining module 303 is specifically configured to randomly determine the information of the position to be operated from a preset range if the category information of the M first detection images does not include the target type information.

a determining module 303, configured to obtain a third detection image from the at least two detection images according to the category information of the at least two detection images, where the third detection image belongs to the auxiliary function category;

the sending module 304 is specifically configured to send the location information to be operated to the client, so that the client triggers an attack operation to a direction corresponding to the location information to be operated.

the sending module 304 is specifically configured to send the position information to be operated to the client, so that the client triggers a click operation according to the position information to be operated.

Referring to fig. 16, fig. 16 is a schematic view of an embodiment of a game testing device according to an embodiment of the present application, and as shown in the drawing, the game testing device 40 includes:

the receiving module 401 is configured to receive a game image to be tested sent by a client, where the game image to be tested belongs to a game image corresponding to a game-playing state;

an obtaining module 402, configured to obtain, through the object detection model, an image recognition result of the game image to be tested, where the image recognition result includes category information of at least two detection images and position information of the at least two detection images, where the game image to be tested is received by the receiving module;

a determining module 403, configured to determine a target operation position according to the image recognition result obtained by the obtaining module, where the target operation position includes at least one to-be-operated position information;

a sending module 404, configured to send the target operation position determined by the determining module to the client, so that the client performs a simulation operation according to the target operation position;

the receiving module 401 is further configured to receive a simulation operation result sent by the client;

a generating module 405, configured to generate a game test report according to the simulation operation result received by the receiving module.

The present embodiment further provides another simulation operation apparatus and a game testing apparatus, both of which are deployed in a computer device, where the computer device may be a terminal device or a server, and reference is made to fig. 17, where fig. 17 is a schematic diagram of an embodiment of a server in the present embodiment, as shown in the figure, the computer device 500 may generate a relatively large difference due to different configurations or performances, and may include one or more Central Processing Units (CPUs) 522 (e.g., one or more processors) and a memory 532, and one or more storage media 530 (e.g., one or more mass storage devices) storing an application program 542 or data 544. Memory 532 and storage media 530 may be, among other things, transient storage or persistent storage. The program stored on the storage medium 530 may include one or more modules (not shown), each of which may include a series of instruction operations for the computer device. Still further, the central processor 522 may be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the computer device 500.

The computer device 500 may also include one or more power supplies 526, one or more wired or wireless network interfaces 550, one or more input-output interfaces 558, and/or one or more operating systems 541, such as a Windows Server ^TM ，Mac OS X ^TM ，Unix ^TM ，Linux ^TM ，FreeBSD ^TM And so on.

The steps performed by the computer device in the above embodiments may be based on the server structure shown in fig. 17.

Embodiments of the present application further provide a computer-readable storage medium, in which a computer program is stored, and when the computer program runs on a computer, the computer is caused to execute the method described in the foregoing embodiments.

Embodiments of the present application also provide a computer program product comprising a program, which when run on a computer, causes the computer to perform the steps as performed in the previous embodiments.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims

1. A method of simulating operation, comprising:

receiving a game image sent by a client;

if the game image meets an image recognition condition, acquiring an image recognition result of the game image through an object detection model, wherein the image recognition result comprises category information of at least two detection images and position information of the at least two detection images;

the determining the target operation position according to the image recognition result comprises:

acquiring M first detection images and M second detection images from the at least two detection images according to the category information of the at least two detection images, wherein the first detection images belong to an attacked category, the second detection images belong to a task category, M is an integer greater than or equal to 1, the first detection images are detection images corresponding to the category information capable of being operated in an attacking manner, and the second detection images are detection images containing task information;

determining a matching result according to a template image to be matched and the second detection image, wherein the template image to be matched belongs to any one template image in a template image set, the template image set comprises at least one template image, the template image comprises picture information and text information, and the template image to be matched is a template image corresponding to category information which can be operated in an attack mode;

if the matching result is successful, determining target type information corresponding to the second detection image, wherein the target type information is category information which can be operated in an attack mode in the task information;

determining position information to be operated according to the target type information and the M first detection images;

determining the target operation position according to the target type information and the M first detection images includes:

if the category information of the M first detection images has the target type information, acquiring a target detection image from the M first detection images, wherein the target detection image corresponds to the target type information;

determining the position information corresponding to the target detection image as the position information to be operated;

and if the category information of the M first detection images does not have the target type information, randomly determining the position information to be operated from a preset range.

2. The method of claim 1, wherein after receiving the game image sent by the client, the method further comprises:

acquiring running state information corresponding to the game image;

if the running state information is the game-in-game state, determining that the game image meets the image recognition condition;

if the running state information is the state before game matching, determining that the game image does not meet the image identification condition;

and if the running state information is the game-play ending state, determining that the game image does not meet the image identification condition.

3. The method of claim 1, wherein determining a target operation position from the image recognition result comprises:

acquiring M first detection images from the at least two detection images according to the category information of the at least two detection images, wherein the first detection images belong to an attacked category, and M is an integer greater than or equal to 1;

4. The method according to claim 1, wherein the determining a matching result according to the template image to be matched and the second detection image comprises:

acquiring at least one similar result from the second detection image in a sliding window mode by adopting the template image to be matched;

if the maximum similarity result is larger than or equal to a similarity threshold, determining that the matching result is a successful matching result;

and if the maximum similarity result is smaller than the similarity threshold, determining that the matching result is a matching failure.

5. The method of claim 1, wherein determining a target operation position from the image recognition result comprises:

acquiring a third detection image from the at least two detection images according to the category information of the at least two detection images, wherein the third detection image belongs to an auxiliary function category;

determining a matching result according to a template image to be matched and the third detection image, wherein the template image to be matched belongs to any one template image in a template image set, and the template image set comprises at least one template image;

6. The method according to any one of claims 1 to 5, wherein the sending the target operation position to the client to enable the client to perform a simulation operation according to the target operation position comprises:

and sending the position information to be operated to the client so that the client triggers an attack operation to the direction corresponding to the position information to be operated.

7. The method according to any one of claims 1 to 5, wherein the sending the target operation position to the client to enable the client to perform a simulation operation according to the target operation position comprises:

and sending the position information to be operated to the client so that the client triggers click operation according to the position information to be operated.

8. A method of game testing, comprising:

receiving a game image to be tested sent by a client, wherein the game image to be tested belongs to a game image corresponding to a game-playing state;

acquiring an image recognition result of the game image to be tested through an object detection model, wherein the image recognition result comprises category information of at least two detection images and position information of the at least two detection images;

receiving a simulation operation result sent by the client;

generating a game test report according to the simulation operation result;

the determining the target operation position according to the image recognition result comprises the following steps:

determining a matching result according to a template image to be matched and the second detection image, wherein the template image to be matched belongs to any one template image in a template image set, the template image set comprises at least one template image, the template image comprises image information and text information, and the template image to be matched is a template image corresponding to category information which can be operated in an attack mode;

if the type information of the M first detection images has the target type information, acquiring target detection images from the M first detection images, wherein the target detection images correspond to the target type information;

9. A simulated operating device comprising:

the receiving module is used for receiving the game images sent by the client;

an obtaining module, configured to obtain, by an object detection model, an image recognition result of the game image received by the receiving module if the game image meets an image recognition condition, where the image recognition result includes category information of at least two detection images and position information of the at least two detection images;

the determining module is specifically configured to:

the determining module, when determining the to-be-operated position information according to the target type information and the M first detection images, is specifically configured to:

10. A game testing apparatus, comprising:

the receiving module is used for receiving a game image to be tested sent by a client, wherein the game image to be tested belongs to a game image corresponding to the game-playing state;

the acquisition module is used for acquiring the image recognition result of the game image to be tested, which is received by the receiving module, through an object detection model, wherein the image recognition result comprises category information of at least two detection images and position information of the at least two detection images;

the receiving module is further configured to receive a simulation operation result sent by the client;

the generating module is used for generating a game test report according to the simulation operation result received by the receiving module;

the determining module is specifically configured to:

11. A server, comprising: a memory, a transceiver, a processor, and a bus system;

wherein the memory is used for storing programs;

the processor is configured to execute a program in the memory, the processor is configured to perform the method of any one of claims 1 to 7 or the method of claim 8 according to instructions in the program code;

12. A computer-readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 7, or perform the method of claim 8.