CN112742019A

CN112742019A - Information processing method, device and system of virtual chessboard and storage medium

Info

Publication number: CN112742019A
Application number: CN202010389039.2A
Authority: CN
Inventors: 王洁梅; 张力柯
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-05-09
Filing date: 2020-05-09
Publication date: 2021-05-04
Anticipated expiration: 2040-05-09
Also published as: CN112742019B

Abstract

The invention provides an information processing method, an information processing device, an information processing system and a storage medium for a virtual chessboard, wherein the information processing method for the virtual chessboard comprises the following steps: acquiring a chessboard image of the virtual chessboard; carrying out image recognition on the chessboard image to obtain the operation state information of the virtual chessboard; determining an operation for a first virtual object in the virtual board based on the operational state information of the virtual board; and providing the determined operation for the first virtual object in the virtual chessboard. The present invention obtains the current state information of the game through the image recognition technology instead of the internal API interface of the game environment, the realization is more flexible and the universality is stronger, and in the scenes of the test or the debugging of the game platform, the present invention does not need the personnel to participate, thereby greatly saving the labor cost.

Description

Information processing method, device and system of virtual chessboard and storage medium

Technical Field

The present invention relates to the field of computer technologies, and in particular, to an information processing method, apparatus, system, and storage medium for a virtual chessboard.

Background

A complete set of game interaction platforms typically includes all implementations of front-end interfaces (e.g., Graphical User Interfaces (GUIs)), back-end Services (SVRs), and so on. Taking the chess game interaction platform as an example, the complete chess walking sub-process generally can include: the chess piece moving algorithm module acquires current chess piece surface information from the environment module through an Application Programming Interface (API) realized in the platform, outputs piece moving operation based on the acquired current chess piece surface information, calls an internal API Interface to change the chess piece surface information maintained in the environment module, displays the changed chess piece surface information to a current front-end Interface and the like. However, some developed chess games do not have available internal API interfaces, and in a game testing or debugging scene, a user cannot acquire the current state of a chessboard through the internal API interfaces or cannot apply a walking sub-operation to a chess environment through the internal API interfaces. Therefore, a new method based on artificial intelligence is needed to acquire and process game state information accordingly.

Disclosure of Invention

The embodiment of the invention provides an information processing method of a virtual chessboard, which comprises the following steps: acquiring a chessboard image of the virtual chessboard; carrying out image recognition on the chessboard image to obtain the operation state information of the virtual chessboard; determining an operation for a first virtual object in the virtual board based on the operational state information of the virtual board; and providing the determined operation for the first virtual object in the virtual chessboard.

According to an embodiment of the present invention, the virtual chessboard comprises the first virtual object and the second virtual object, wherein the image recognition of the chessboard image to obtain the operation state information of the virtual chessboard comprises: performing first image recognition based on the chessboard image to obtain a reconstructed virtual chessboard; performing a second image recognition based on the board image to determine corresponding positions of the first and second virtual objects on the reconstructed virtual board; and determining operational state information of the virtual board based on the corresponding positions of the first virtual object and the second virtual object on the reconstructed virtual board.

According to an embodiment of the invention, wherein the performing a first image recognition based on the chessboard images to obtain a reconstructed virtual chessboard comprises: performing first image matching on a template image corresponding to the virtual chessboard and the chessboard image, and obtaining the reconstructed virtual chessboard based on the chessboard image if a result of the first image matching satisfies a first threshold condition.

According to an embodiment of the present invention, a virtual object library to be matched is preset, where the virtual object library to be matched includes a plurality of first virtual sub-objects to be matched and a plurality of second virtual sub-objects to be matched, and performing second image recognition based on the board image to determine corresponding positions of the first virtual object and the second virtual object on the reconstructed virtual board includes: for each to-be-matched virtual sub-object in the to-be-matched first virtual sub-objects and the to-be-matched second virtual sub-objects, performing second image matching on the template image corresponding to the to-be-matched virtual sub-object and the chessboard image; and determining the position of the virtual sub-object to be matched on the chessboard image as the corresponding position of the virtual sub-object corresponding to the position on the reconstructed virtual chessboard under the condition that the result of the second image matching meets a second threshold condition.

According to the embodiment of the present invention, the virtual sub-objects corresponding to the first virtual sub-objects to be matched are first virtual sub-objects, at least one of the first virtual sub-objects constitutes the first virtual object, the virtual sub-objects corresponding to the second virtual sub-objects to be matched are second virtual sub-objects, and at least one of the second virtual sub-objects constitutes the second virtual object.

According to an embodiment of the present invention, wherein the operation state information of the virtual chessboard comprises: information indicating that the virtual chessboard is currently in a pre-match state, a mid-match state or a match end state.

According to an embodiment of the present invention, wherein the virtual chessboard comprises a first virtual object and a second virtual object, the first virtual object comprising at least one first virtual sub-object and the second virtual object comprising at least one second virtual sub-object.

According to an embodiment of the present invention, wherein determining an operation for a first virtual object in the virtual chess board based on the operation state information of the virtual chess board comprises: under the condition that the operation state information of the virtual chessboard indicates that the virtual chessboard is currently in a game-alignment state, acquiring object information and an object position of at least one first virtual sub-object in the first virtual objects; acquiring object information and an object position of at least one second virtual sub-object in second virtual objects in the virtual chessboard; and determining an operation for the first virtual object in the virtual chessboard based on the acquired object information and object position of at least one first virtual sub-object in the first virtual object and the acquired object information and object position of at least one second virtual sub-object in the second virtual object.

According to an embodiment of the present invention, wherein determining an operation for a first virtual object in the virtual chess board based on the operation state information of the virtual chess board comprises: in a case that the operation state information of the virtual chessboard indicates that the virtual chessboard is currently in a game-in state, determining an operation for a first virtual object in the virtual chessboard by using a neural network model, wherein the neural network model is a convolutional neural network pre-trained by a pre-acquired sample set of operations for the first virtual object in the virtual chessboard, wherein the convolutional neural network comprises a convolutional layer, a normalization layer, an activation layer, a residual block and a full connection layer.

According to an embodiment of the present invention, wherein determining an operation for a first virtual object in the virtual chess board based on the operation state information of the virtual chess board further comprises: for each of the first and second virtual sub-objects, generating a corresponding reconstructed virtual sub-checkerboard, wherein a plurality of reconstructed virtual sub-checkerboards constitute the reconstructed virtual checkerboard, and using the plurality of reconstructed virtual sub-checkerboards as inputs to the convolutional neural network.

According to an embodiment of the present invention, wherein obtaining the board image of the virtual board comprises: acquiring a board image of the virtual board by intercepting or externally photographing an image including the virtual board.

According to an embodiment of the invention, wherein the virtual board is a virtual board of a chess game, the first virtual object is a set of pieces of a first color, and the second virtual object is a set of pieces of a second color; the first virtual object comprises at least one first virtual sub-object of a first color and the second virtual object comprises at least one second virtual sub-object of a second color, wherein determining an operation for a first virtual object in the virtual board based on the operational state information of the virtual board comprises: an operation of determining one of the at least one first virtual sub-object comprised by the first virtual object, the operation indicating a movement of the first virtual sub-object from its current position to a target position.

An embodiment of the present invention provides an information processing apparatus for a virtual chessboard, including: the image acquisition module is used for acquiring a chessboard image of the virtual chessboard; the information extraction module is used for carrying out image recognition on the chessboard image so as to obtain the operation state information of the virtual chessboard; an operation determination module for determining an operation for a first virtual object in the virtual chessboard based on the operation state information of the virtual chessboard; and an operation providing module for providing the determined operation for the first virtual object in the virtual chessboard.

An embodiment of the present invention provides an information processing system for a virtual chessboard, including: a processor; and a memory having stored thereon computer-executable instructions that, when executed by the processor, are for implementing any of the methods according to embodiments of the invention.

Embodiments of the present invention provide a computer-readable storage medium having stored thereon computer-executable instructions for implementing any of the methods according to embodiments of the present invention when executed by a processor.

The embodiment of the invention provides an information processing method, device, system and storage medium of a virtual chessboard, which obtains the current state information of a game through an image recognition technology instead of an internal API (application program interface) of a game environment, is more flexible to realize and stronger in universality, processes the current state information of the game through a specially designed external artificial intelligence algorithm and outputs targeted operation, and can simulate a user to interact with a game platform. Under the scenes of testing or debugging a game platform and the like, the method can assist testers in carrying out automatic testing (particularly long-time performance testing) without the participation of the testers, so that the labor cost can be greatly saved. And, by using the AI algorithm to determine the operation, the tester does not need to know complicated related game rules or game skills, thereby saving the learning cost of the tester.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. It is obvious that the drawings in the following description are only some exemplary embodiments of the invention, and that other drawings can be derived from these drawings by a person skilled in the art without inventive effort.

FIG. 1 illustrates a schematic structural diagram of a game interaction platform;

FIG. 2 shows a schematic process flow of an information processing method of a virtual chessboard according to an embodiment of the invention;

FIG. 3 shows a flow chart of an information processing method of a virtual chessboard according to an embodiment of the invention;

FIG. 4 shows a schematic diagram of a virtual chessboard reconstruction process according to an embodiment of the invention;

FIG. 5 is a diagram illustrating a matching similarity result matrix according to an embodiment of the present invention;

figure 6 shows a schematic diagram of a pawn reconstruction process according to an embodiment of the invention;

FIG. 7 shows an illustrative process for determining walking sub-operations using a neural network model in accordance with an embodiment of the invention;

FIG. 8 shows a schematic network structure of a convolutional neural network according to an embodiment of the present invention;

FIG. 9 shows an exemplary system architecture and data flow diagram for chess game automation testing, according to an embodiment of the present invention;

FIG. 10 is a schematic diagram illustrating an automatic interaction flow of an information processing method of a virtual chessboard according to an embodiment of the present invention;

FIG. 11 shows a schematic diagram of an information processing apparatus of a virtual chessboard according to an embodiment of the invention;

FIG. 12 shows a schematic diagram of an information processing system of a virtual chessboard according to an embodiment of the invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of embodiments of the invention and not all embodiments of the invention, with the understanding that the invention is not limited to the example embodiments described herein.

In the present specification and the drawings, steps and elements having substantially the same or similar characteristics are denoted by the same or similar reference numerals, and repeated description of the steps and elements will be omitted. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance or order.

In the specification and drawings, elements are described in singular or plural according to embodiments. However, the singular and plural forms are appropriately selected for the proposed case only for convenience of explanation and are not intended to limit the present invention thereto. Thus, the singular includes the plural, and the plural also includes the singular, unless the context clearly dictates otherwise.

Embodiments of the present invention relate to using artificial intelligence to make a move sub-decision for a chess game, and for ease of understanding, the following first introduces the relevant concepts of artificial intelligence.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making. In this context, a go sub-decision of a chess game may be made based on artificial intelligence.

Illustrative applications and example embodiments of the invention are further described below with reference to the accompanying drawings.

Fig. 1 shows a schematic structural diagram of a game interaction platform 100.

As shown in FIG. 1, the game interaction platform 100 may include a front-end interface (e.g., GUI 101) and a back-end service (e.g., SVR 104) that may communicate through various communication interfaces, such as API interfaces. Taking the chess game interaction platform as an example, the back-end service (SVR 104) may further include an environment module 102 and a chess move sub-algorithm module 103, which may communicate through an internal API interface 105. Generally, the chess move algorithm module 103 may obtain current chess surface information from the environment module 102 through the internal API interface 105, determine a move operation recommended to the user based on the obtained current chess surface information, call the internal API interface 105 to change the chess surface information maintained inside the environment module 102, and then the environment module 102 may display the changed chess surface information to the GUI 101 through the communication interface, thereby enabling the user to know the move operation recommended by the game interaction platform.

However, in some embodiments, some developed game platforms may not have an available internal API interface, and in a test or debug scenario of the game platform, a user may not be able to directly obtain the current state of the board through the internal API interface, or may not be able to apply a walking operation to the game environment through the internal API interface. Therefore, a new method based on artificial intelligence is needed to acquire and process game state information accordingly. The following description will still be given taking a chess game as an example, however, it should be understood that the embodiments of the present invention can be applied to various chess game interaction platforms, such as chess, go, gobang, checkers, and the like.

Fig. 2 shows a schematic processing flow of an information processing method 200 of a virtual chessboard according to an embodiment of the invention.

As shown in fig. 2, a chess game may be run on the electronic device 201. The electronic device 201 may be various types of electronic devices such as a smart phone, a tablet computer, a desktop computer, a game device, etc., and may display the board status of the chess game on its display screen in real time. According to an embodiment of the present invention, the board image 202 including the chess board may be acquired by intercepting the screen of the electronic device 201 or by externally photographing the screen of the electronic device 201. Further, the checkerboard image 202 may be sent to the AI server 203 for analysis and processing. The AI server 203 may perform image recognition based on the board image 202 to generate a walking operation for the current board and output the operation to the chess game running on the electronic device 201 for execution. In one embodiment, the move operation may be a move operation for a particular pawn. For example, the move operation may be an operation of moving the chess piece a from the current position pos1 to the position pos 2. In one embodiment, in the event that the AI server 203 fails to successfully generate a go sub-operation based on the currently received board image 202, the AI server 203 may also send a request to re-intercept the image to the electronic device 201. In addition, the AI server 203 may also send various other debugging commands or requests to and from the electronic device 201, which is not described herein again.

In one embodiment, the AI server 203 may be deployed on the same electronic device as the chess game, e.g., the AI server 203 may be deployed on the electronic device 201 as the chess game. In another embodiment, the AI server 203 may also be deployed on a different electronic device than the chess game, for example, the chess game may be run on a smartphone, and the AI server may be deployed on a desktop computer communicatively connected to the smartphone. In this case, the AI server 203 may feed back the go sub-operation to the chess game running on the smart phone through a debug command (e.g., ADB command) interface or the like for further execution. Under the automatic test or debugging scene, the method can simulate the tester and the chess game to carry out automatic fighting interaction, does not need the tester to participate in person, and can greatly save the test time and the test cost.

More specifically, fig. 3 shows a flowchart of an information processing method 200 of a virtual chessboard according to an embodiment of the invention.

As shown in fig. 3, first, in step S301, a board image of a virtual board may be acquired.

As described above, in one embodiment, the board image of the virtual board may be acquired by intercepting or externally photographing an image including the virtual board.

In step S302, image recognition may be performed on the board image to acquire operation state information of the virtual board.

In one embodiment, a first virtual object and a second virtual object may be included in the virtual board, the first virtual object may include at least one first virtual sub-object, and the second virtual object may include at least one second virtual sub-object. In this case, the image recognition of the board image to acquire the operation state information of the virtual board may include: performing first image recognition based on the chessboard image to obtain a reconstructed virtual chessboard; performing second image recognition based on the chessboard image to determine corresponding positions of the first virtual object and the second virtual object on the reconstructed virtual chessboard; and determining operating state information of the virtual board based on the corresponding positions of the first virtual object and the second virtual object on the reconstructed virtual board.

In one embodiment, performing the first image recognition based on the checkerboard image to obtain the reconstructed virtual checkerboard may include: the template image corresponding to the virtual chessboard and the chessboard image are subjected to first image matching, and a reconstructed virtual chessboard is obtained based on the chessboard image in the case that the result of the first image matching satisfies a first threshold condition.

In particular, FIG. 4 shows a schematic diagram of a virtual chessboard reconstruction process 400 according to an embodiment of the invention.

As shown in fig. 4, in one embodiment, a template image corresponding to the virtual chessboard may be previously acquired (as shown in (a) of fig. 4) and saved in the template sample library. When the board image 202 is acquired from the electronic device 201 as shown in fig. 2, the template image may be subjected to a first image matching with the acquired board image 202. Fig. 2 only schematically shows an example case where the board image 202 includes substantially only one complete board of the chess board for the sake of clarity, however, it should be understood that the board image 202 acquired by the cutout screen or the external photographing may also include other image interface elements other than the board, for example, the board image 202 may also include other display areas (e.g., an area displaying the current time) on the screen of a mobile phone, and the like. In one embodiment, it may be determined whether the currently acquired board image 202 contains a virtual board through first image matching, as shown in (b) of fig. 4. In one embodiment, in case it is determined that the currently acquired board image 202 contains a virtual board, a reconstructed virtual board may be obtained based on the board image 202. For example, in the case where it is determined that the currently acquired board image 202 contains a virtual board, it is also possible to determine the position of the virtual board in the board image 202 and size information such as the width and height of the virtual board by first image matching, and obtain a reconstructed virtual board based on the size information. For example, in the case of chinese chess, the chessboard of chinese chess is a 9-row and 8-column checkerboard layout, so the width of the determined virtual chessboard is divided by 8 and the height is divided by 9 to obtain the size of one reconstructed cell, and then the reconstructed cells are rearranged in a 9-row and 8-column layout to generate a reconstructed virtual chessboard, as shown in (c) of fig. 4. In another embodiment, in case it is determined that the currently acquired board image 202 contains a virtual board, the size of the template board corresponding to the virtual board, which is stored in advance, may also be directly taken as the size of the reconstructed virtual board.

In one embodiment, the main steps of the first image matching may include:

step 1: starting from a specific position of the chessboard image 202, the matching similarity between the template image and each window image is sequentially calculated in a sliding window manner by a specific sliding window step length. In one embodiment, the size of the sliding window may be the same as the size of the template image. For example, the step size of the sliding window may be set to "1", the size of the sliding window may be set to be the same as the size of the template image, and the matching similarity between the template image and each window image may be calculated sequentially from left to right, from top to bottom, starting from the upper left corner of the checkerboard image 202.

In one embodiment, the matching similarity may be calculated using squared difference matching, as shown in equation (1) below.

Where R (x, y) represents the matching similarity, T represents the template image, I represents the test image (e.g., the checkerboard image 202), x represents the abscissa on the test image, y represents the ordinate on the test image, x 'represents the abscissa on the template image, and y' represents the ordinate on the template image. Further, x and y may represent the location coordinates of the origin (e.g., the top left corner) of the current window image on the test image.

In one embodiment, the matching similarity may be calculated using standard squared error matching, as shown in equation (2) below.

In one embodiment, the matching similarity may be calculated in a correlation matching manner, as shown in the following equation (3).

In one embodiment, the matching similarity may be calculated in a standard correlation matching manner, as shown in the following equation (4).

Step 2: the calculated matching similarity is stored in the result matrix as shown in fig. 5.

FIG. 5 is a diagram illustrating a matching similarity result matrix according to an embodiment of the present invention. Specifically, fig. 5 shows an example case where: assuming that the size of the acquired checkerboard image 202 is (m +2) × (n +2), the size of the sliding window is set to m × n, and the sliding window step size is set to "1", moving the sliding window from left to right, from top to bottom in order from the upper left corner of the checkerboard image 202 makes it possible to obtain 3 × 3 window images in total, and 3 × 3 matching similarities can be calculated accordingly. Each matching similarity in the result matrix shown in fig. 5 may indicate a degree of matching of the window image corresponding to the matching similarity with the template image. If the matching similarity is calculated by adopting the square error matching or standard square error matching method, the smaller the matching similarity value is, the more matching is represented; if the matching similarity is calculated by the correlation matching or the standard correlation matching method as described above, a larger matching similarity value indicates a better match.

And step 3: the best match value is found in the result matrix. As described above, the best matching value may be the maximum value or the minimum value in the result matrix depending on the calculation method of the matching similarity. For example, when the matching similarity is calculated using the standard correlation matching method, the best matching value may be the maximum value "0.99" as shown in fig. 5.

And 4, step 4: the best match value is compared to a first threshold, if the best match value satisfies a first threshold condition, the match is deemed to succeed in finding the virtual chessboard in the chessboard image 202, and the position of the virtual chessboard in the chessboard image 202 may be determined based on the position of the best match value in the result matrix. It should be appreciated that if the matching similarity is calculated using a squared error matching or a standard squared error matching method as described above, then the best match value may be considered less than a first threshold (e.g., 0.1) as the best match value satisfying a first threshold condition; if the matching similarity is calculated using correlation matching or a standard correlation matching method as described above, the best match value greater than a first threshold (e.g., 0.95) may be considered as the best match value satisfying a first threshold condition.

In one embodiment, a plurality of template images with different sizes may also be used for the first image matching with the acquired chessboard image 202 to acquire size information such as width and height of the virtual chessboard in the chessboard image 202.

It should be appreciated that any other image recognition method may be employed in addition to the image matching method to determine whether a virtual chessboard is present in the chessboard image 202 and to obtain information about its position and size when a virtual chessboard is present.

Returning to step S302, in one embodiment, a first virtual object and a second virtual object may be included in the virtual chessboard, the first virtual object may include at least one first virtual sub-object, and the second virtual object may include at least one second virtual sub-object. In an embodiment, a virtual object library to be matched may be further preset, the virtual object library to be matched may include a first virtual object library to be matched and a second virtual object library to be matched, the first virtual object library to be matched may include a plurality of first virtual sub-objects to be matched, and the second virtual object library to be matched may include a plurality of second virtual sub-objects to be matched. In one embodiment, the virtual sub-objects corresponding to the first virtual sub-objects to be matched are first virtual sub-objects, at least one of the first virtual sub-objects constitutes the first virtual object, the virtual sub-objects corresponding to the second virtual sub-objects to be matched are second virtual sub-objects, and at least one of the second virtual sub-objects constitutes the second virtual object. In this case, performing the second image recognition based on the checkerboard image to determine the corresponding positions of the first virtual object and the second virtual object on the reconstructed virtual checkerboard may include: for each of a plurality of first virtual sub-objects to be matched in a first virtual object library to be matched, performing second image matching on a template image corresponding to the first virtual sub-object to be matched and a chessboard image; under the condition that the result of the second image matching meets a second threshold value condition, determining the position of the first virtual sub-object to be matched on the chessboard image as the corresponding position of the first virtual sub-object corresponding to the first virtual sub-object on the reconstructed virtual chessboard, and performing second image matching on the template image corresponding to the second virtual sub-object to be matched and the chessboard image for each of a plurality of second virtual sub-objects to be matched in the second virtual object library to be matched; and determining the position of the second virtual sub-object to be matched on the chessboard image as the corresponding position of the second virtual sub-object corresponding to the position on the reconstructed virtual chessboard under the condition that the result of the second image matching meets a second threshold condition.

In particular, still taking chinese chess as an example, the first virtual object may be a set of red chess pieces, and the first virtual sub-object may be any kind of red chess pieces (e.g., car of red, horse, phase, man, commander, cannon, soldier), while the second virtual object may be a set of black chess pieces, and the second virtual sub-object may be any kind of black chess pieces (e.g., car of black, horse, elephant, soldier, general, cannon, pawn).

In one embodiment, a virtual object library to be matched may be preset, and the virtual object library to be matched may include a first virtual object library to be matched and a second virtual object library to be matched. For example, still taking Chinese chess as an example, the first virtual object library to be matched may be a set of all categories of the chess pieces of the red side (e.g., a set of seven chess pieces of the red side, horse, opponent, person, general, cannon, soldier types), and the first virtual sub-object to be matched may be each of the chess pieces of the red side in the first virtual object library to be matched. The second virtual object library to be matched may be a set of all categories of black square pieces (e.g., a set of seven types of pieces such as black square, horse, elephant, soldier, general, cannon, pawn), and the second virtual sub-object to be matched may be each type of black square piece in the second virtual object library to be matched. In one embodiment, template images corresponding to each of the two red and black chess pieces may be collected in advance and stored in a template sample library, and each of the chess pieces may be numbered, taking chinese chess as an example (the two red and black chess pieces each include 7 chess pieces), as shown in table 1 below.

TABLE 1 chessman numbering

Then, the template image corresponding to each of the two chessmen of red and black may be respectively subjected to the second image matching with the chessboard image 202. The process of second image matching may be similar to the first image matching process described above. For example, for a red square chess piece "horse", matching similarity between the template image corresponding to red square chess piece "horse" and each window image may be calculated sequentially in a sliding window manner starting from a specific position of board image 202 at a specific sliding window step size, and then the matching similarity values may be compared to a second threshold value to determine whether they satisfy a second threshold condition. If the matching similarity value satisfies a second threshold condition, then the match is deemed to succeed in finding the red chess piece "horse" in the board image 202, and the position of the red chess piece "horse" in the board image 202 may be determined based on the position of the matching similarity value in the result matrix. The position of the closest checkerboard to the red chess piece "horse" may then be calculated on the virtual chess board reconstructed as described above, and the position of the closest checkerboard may be determined as the corresponding position of the red chess piece "horse" on the reconstructed virtual chess board. A second image matching may be performed for each of the two parties, red and black, and the corresponding position of each matched chess piece on the reconstructed virtual chessboard is determined, as shown in fig. 6.

Figure 6 shows a schematic view of a pawn reconstruction process 600 according to an embodiment of the invention. Specifically, (a) in fig. 6 shows an example of a board image containing a plurality of kinds of chessmen of both red and black. Fig. 6(b) shows a second image matching result after performing second image matching on the chessboard images based on all the virtual sub-objects to be matched (i.e., 14 chessmen for red and black) in the virtual object library to be matched.

It should be appreciated that in the grayscale diagram of fig. 6, as an example, the pieces on the top half of the board are taken as black chess pieces and the pieces on the bottom half of the board are taken as red chess pieces. However, the invention is not limited thereto, and the red and black pawns may be located anywhere on the board.

Each number in fig. 6(b) may represent a matching similarity with which the window image at that position is identified as a corresponding pawn. Specifically, as shown in the figure, taking the second chess piece "horse" in the lower left corner of (b) as an example, the "0.91, Hongma" in the figure can represent: the matching similarity value of the window image at this position and the template image corresponding to red "horse" is "0.91". It should be appreciated that when a second image match is made to the board image based on a particular virtual sub-object to be matched (e.g., red chess piece "horse") in the library of virtual objects to be matched, matching similarity values that satisfy a second threshold condition (e.g., greater than a threshold of 0.9) may be obtained at a plurality of locations. Fig. 6 (c) shows an example of a reconstructed virtual pawn determined on a reconstructed virtual chessboard after a second image matching. As shown, the number of each pawn may be reconstructed on the reconstructed virtual chessboard according to table 1.

Returning again to step S302, in one embodiment, the operational status information of the virtual chessboard may include information indicating that the virtual chessboard is currently in a pre-alignment state, an in-alignment state, or an end-of-alignment state. For example, the current state of the virtual chess board may be determined based on the corresponding position of each of the chess pieces on the reconstructed virtual chess board as determined by the reconstruction process of the virtual chess board and the chess pieces described above. For example, in case it is determined that no pawns are present on the reconstructed virtual board, the virtual board may be determined to be in a pre-game state; in case that it is determined that a certain number of pieces exist at a certain position on the reconstructed virtual chessboard, the virtual chessboard may be determined to be in a game-playing state, and in case that it is determined that there is no "commander" or "will" piece on the reconstructed virtual chessboard, the virtual chessboard may be determined to be in a game-playing end state.

In another embodiment, the current state of the virtual chessboard may also be determined first based on pre-stored image samples. For example, a third image recognition may be performed on the acquired chessboard image 202, and when a pre-stored key image sample such as "start" or "start game" is recognized to be included in the chessboard image 202, the current state of the virtual chessboard may be determined as the pre-game state; when the chessboard image 202 is identified to include a key image sample such as "game end", the current state of the virtual chessboard can be determined as a game end state; when it is recognized that a certain number of two-sided pawns are present in the board image 202, the current state of the virtual board may be determined as a hit-in-game state.

It should be understood that the operation status information of the virtual chessboard may include not only information indicating that the virtual chessboard is currently in the pre-game state, the in-game state or the end-game state, but also information such as the position and size of the virtual chessboard, and the specific positions of all chess pieces currently existing on the virtual chessboard.

Next, returning to fig. 3, in step S303, an operation for the first virtual object in the virtual chessboard may be determined based on the operation state information of the virtual chessboard. In one embodiment, in a case that the operation state information of the virtual chessboard indicates that the virtual chessboard is currently in the game-in state, the object information and the object position of at least one first virtual sub-object among the first virtual objects may be acquired, and the object information and the object position of at least one second virtual sub-object among the second virtual objects in the virtual chessboard may be acquired; and determining an operation for the first virtual object in the virtual chessboard based on the acquired object information and object position of the at least one first virtual sub-object in the first virtual object and the acquired object information and object position of the at least one second virtual sub-object in the second virtual object. The object information of the virtual sub-object may be information indicating which kind of chess the virtual sub-object is (e.g., car, horse, elephant, soldier, etc.). In one embodiment, assuming that the moving of the red side is currently performed, the moving operation for the red side may be determined based on the game type information and the position information of each piece of the currently existing red and black sides, which are acquired in step S302. For example, the move operation may be an operation of moving the chess piece a of redplayer from the current position pos1 to the position pos2, which is determined after analyzing the piece type information and the position information of each of the currently existing chessmen of both redplayer and blackplayer.

In one embodiment, the move operation for one side (e.g., the red side) of the virtual chessboard may be determined based on pre-stored chess move rules according to the game type information and position information of each piece of both red and black sides currently existing, which are acquired from the chessboard image 202.

In another embodiment, a neural network model may also be utilized to determine a walk operation for one of the virtual boards (e.g., the red square), as shown in FIG. 7.

In particular, FIG. 7 shows an illustrative process 700 for determining walking sub-operations using a neural network model in accordance with an embodiment of the invention.

As shown in fig. 7, a walk sub-operation for one of the virtual boards may be determined using a convolutional neural network 702 pre-trained with a pre-acquired sample set of operations for that party.

In one embodiment, the virtual chessboard and chess pieces reconstructed in step S302 may be split according to different chess piece categories to serve as the input 701 to the convolutional neural network 702. That is, for each type of pawn, its corresponding chessman is constructed and these chessmans are taken as input 701 to the convolutional neural network 702. For example, taking chinese chess as an example, the reconstructed virtual chessboard and chess pieces can be split into 14 sub-chequers of 10 × 9 (for example, each position takes a value of 0 or 1) according to table 1 as the input 701 of the convolutional neural network 702. Specifically, as mentioned above, 14 is the total category of all the chessmen of the two parties of red and black in the Chinese chess game. Each sub chessboard can represent the position of a kind of chess piece, in the sub chessboard, the position of the corresponding chess piece of the sub chessboard can be 1, and the rest positions can be 0. For example, for a checkerboard corresponding to a red chess piece "horse," the position of red chess piece "horse" may be set to 1, while the remaining positions may all be set to 0.

In another embodiment, a 10 x 9 feature plane may additionally be entered to indicate whether the move operation is determined for a red chess piece or a black chess piece. For example, when the input value of the 10 × 9 feature plane is all 1, it may indicate that the go operation is determined for a red chess piece, and when the input value of the 10 × 9 feature plane is all 0, it may indicate that the go operation is determined for a black chess piece. In this case, the input 701 of the convolutional neural network 702 may be 15 10 × 9 planes.

Through the processing of the convolutional neural network 702, the walking operation determined based on the current board information may be output. For example, the convolutional neural network 702 may output a fall probability 703 represented by a one-dimensional vector of length 2086, where the one-dimensional vector of length 2086 may represent a set of all walks that all chess pieces may make according to the chess rules. It should be appreciated that the output vector length 2086 is merely an example, and in other embodiments, the convolutional neural network 702 may output vectors of any other particular length and particular dimensions. In one embodiment, based on the output of the convolutional neural network 702, the walk sub-operation corresponding to the maximum probability value may be determined as the current walk sub-operation.

FIG. 8 shows a schematic network structure of a convolutional neural network 702, according to an embodiment of the present invention.

As shown in FIG. 8, in one embodiment, convolutional neural network 702 may employ the network structure of ResNet. In the embodiment shown in fig. 8, the input of the network may be 14 chessboard patterns of 10 × 9, where each chessboard pattern may represent a position of a chess piece, and the position of the chess piece may be 1, and the rest positions may be 0. The first layer of the network may be a convolution layer, and after 256 5 × 5 convolution kernels, the data may be normalized (BatchNormalize) using a normalization layer to reduce the difference between the data. After normalization, the normalized data may be input to the activation function as the output of the present layer. In one embodiment, a ReLU may be employed as the activation function.

The convolutional neural network 702 may include 7 residual blocks in the middle, each residual Block may include two convolutional layers (Convolution blocks) therein, the size of the convolutional core may be 3 × 3, and the number may be 256. Each residual block may be performed in turn: 256 convolution operations of 3 × 3, data normalization, activation, 256 convolution operations of 3 × 3, data normalization, fusion with the input of the residual block, activation, and the like.

After the residual processing, the operations of convolution, data normalization, activation and full concatenation flattening may be performed once, and finally a one-dimensional probability vector with length 2086 may be output, where each probability may represent the probability that each chess piece under the current chessboard can perform a specific operation.

Returning again to FIG. 3, in step S304, the determined operation for the first virtual object in the virtual chessboard may be provided. In one embodiment, the operation may be provided to a user for reference or selectively performed by the user. In another embodiment, this operation may be provided to an electronic device running the board game, as described above in connection with fig. 2, and the determined move operation may be provided to the chess game running on the smart phone for further execution through an interface such as a debug command (e.g., ADB command).

Fig. 9 shows an exemplary system architecture 900 and data flow diagram applied to chess game automation testing, according to an embodiment of the present invention.

As shown in fig. 9, in an automated testing scenario of a chess game, a data source 901 may be a to-be-tested chinese chess game running on a smart phone, and an Input Output (IO) module 902 may intercept a board image (e.g., board image 202) including a board of the chess game from the smart phone and distribute it to a Management Center (MC) module 903. In one embodiment, the MC module 903 may first send the board image to a User Interface (UI) module 904, and the UI module 904 may match the received board image to a pre-stored image sample and determine the current game state as described above. When the UI module 904 recognizes that the chess game is currently in a non-hit state (e.g., a pre-hit state or a hit end state), a UI action may be returned to the MC module 903. In one embodiment, the UI action may be a request to retrieve a board image. The MC module 903 may forward the UI action to the IO module 902, and the IO module 902 may further forward the UI action back to the data source 901 to retrieve the board image.

In one embodiment, when the UI module 904 recognizes that the chess game is currently in the game-play state, the MC module 903 may be notified, and the MC module 903 may further send the board image to a game image recognition (GameReg) module 905 for the image recognition operation as described in step S302. The GameReg module 905 may send the image recognition results to an Artificial Intelligence (AI) module 906, and the AI module 906 may make decisions for AI actions based on the received image recognition results. In one embodiment, the AI action may be an operation to instruct pawn A to move from current position pos1 to position pos 2. Further, AI actions may be output via the MC module 903, the IO module 902, e.g., may be sent to the data source 901 for execution. And completing one automatic interaction.

The whole automatic interaction flow is shown in fig. 10. In particular, fig. 10 shows a schematic diagram of an automatic interaction flow 1000 of an information processing method of a virtual chessboard according to an embodiment of the present invention. As shown in fig. 10, in a scenario of an automated test of a chess game, etc., a chess game image may be obtained in real time from an electronic device (e.g., a mobile phone terminal) running the chess game. As described above, the chess game image can be acquired in real time by means of screen capturing or external shooting. Next, interface elements in the chess game image may be identified (e.g., a board and pieces are identified) using an image identification module. The AI module may reconstruct the board and pieces based on the identified interface elements in the chess game image and make a decision to move based on the current board state, as described above. Finally, the move operation may be fed back to the electronic device (e.g., a mobile phone terminal) running the chess game for execution.

It should be understood that the information processing method of the virtual chessboard according to the embodiment of the present invention can be used not only for information processing or testing of the game of Chinese chess, but also for information processing or testing of other games including chess, go, checkers, gobang, etc. in which one or more parties perform game-to-game interaction.

Fig. 11 shows a schematic diagram of an information processing apparatus 1100 of a virtual chessboard according to an embodiment of the invention.

As shown in fig. 11, the information processing apparatus 1100 of the virtual chessboard according to the embodiment of the present invention may include: an image acquisition module 1101, an information extraction module 1102, an operation determination module 1103, and an operation providing module 1104. The image obtaining module 1101 may be configured to obtain a chessboard image of the virtual chessboard; the information extraction module 1102 may be configured to perform image recognition on the chessboard image to obtain the operation state information of the virtual chessboard; the operation determination module 1103 may be configured to determine an operation for the first virtual object in the virtual chessboard based on the operation state information of the virtual chessboard; and the operation providing module 1104 may be used to provide the determined operation for the first virtual object in the virtual board, for example, to a user, or to an electronic device running a board game.

In one embodiment, the image recognition of the chessboard image to obtain the operation status information of the virtual chessboard may comprise: performing first image recognition based on the chessboard image to obtain a reconstructed virtual chessboard; performing second image recognition based on the chessboard image to determine corresponding positions of the first virtual object and the second virtual object on the reconstructed virtual chessboard; and determining operating state information of the virtual board based on the corresponding positions of the first virtual object and the second virtual object on the reconstructed virtual board.

In one embodiment, determining an operation for a first virtual object in the virtual board based on the operational state information of the virtual board may comprise: under the condition that the operation state information of the virtual chessboard indicates that the virtual chessboard is currently in a game-alignment state, acquiring object information and an object position of at least one first virtual sub-object in the first virtual object; acquiring object information and an object position of at least one second virtual sub-object in second virtual objects in the virtual chessboard; and determining an operation for the first virtual object in the virtual chessboard based on the acquired object information and object position of the at least one first virtual sub-object in the first virtual object and the acquired object information and object position of the at least one second virtual sub-object in the second virtual object.

FIG. 12 shows a schematic diagram of an information processing system 1200 for a virtual chessboard according to an embodiment of the invention.

As shown in fig. 12, an information processing system 1200 of a virtual chessboard according to an embodiment of the invention may include a processor 1201 and a memory 1202, which may be interconnected by a bus 1203.

The processor 1201 can perform various actions and processes according to a program or code stored in the memory 1202. In particular, the processor 1201 may be an integrated circuit chip having signal processing capabilities. The processor may be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. Various methods, steps, flows, and logic blocks disclosed in embodiments of the invention may be implemented or performed. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, which may be the X86 architecture or the ARM architecture or the like.

The memory 1202 stores executable instructions for implementing an information processing method of a virtual chessboard according to an embodiment of the present invention when executed by the processor 1201. The memory 1202 can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Synchronous Link Dynamic Random Access Memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that the memories of the methods described herein are intended to comprise, without being limited to, these and any other suitable types of memory.

The present invention also provides a computer-readable storage medium having stored thereon computer-executable instructions, which, when executed by a processor, may implement an information processing method of a virtual chessboard according to an embodiment of the present invention. Similarly, computer-readable storage media in embodiments of the invention may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. It should be noted that the memories of the methods described herein are intended to comprise, without being limited to, these and any other suitable types of memory.

It is to be noted that the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises at least one executable instruction for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In general, the various exemplary embodiments of this invention may be implemented in hardware or special purpose circuits, software, firmware, logic or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device. While various aspects of the embodiments of the invention may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that the blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

The exemplary embodiments of the invention, as set forth in detail above, are intended to be illustrative, not limiting. It will be appreciated by those skilled in the art that various modifications and combinations of the embodiments or features thereof may be made without departing from the principles and spirit of the invention, and that such modifications are intended to be within the scope of the invention.

Claims

1. An information processing method of a virtual chessboard comprises the following steps:

acquiring a chessboard image of the virtual chessboard;

carrying out image recognition on the chessboard image to obtain the operation state information of the virtual chessboard;

determining an operation for a first virtual object in the virtual board based on the operational state information of the virtual board; and

providing the determined operation for the first virtual object in the virtual chessboard.

2. The information processing method of claim 1, wherein the virtual board comprises the first virtual object and a second virtual object, wherein image recognizing the board image to obtain the operation state information of the virtual board comprises:

performing first image recognition based on the chessboard image to obtain a reconstructed virtual chessboard;

performing a second image recognition based on the board image to determine corresponding positions of the first and second virtual objects on the reconstructed virtual board; and

determining operational state information of the virtual board based on corresponding positions of the first virtual object and the second virtual object on the reconstructed virtual board.

3. The information processing method of claim 2, wherein performing a first image recognition based on the checkerboard image to obtain a reconstructed virtual checkerboard comprises:

performing a first image matching of a template image corresponding to the virtual chessboard with the chessboard image, an

Obtaining the reconstructed virtual chessboard based on the chessboard images if a result of the first image matching satisfies a first threshold condition.

4. The information processing method according to claim 2, wherein a virtual object library to be matched is set in advance, the virtual object library to be matched includes a plurality of first virtual sub-objects to be matched and a plurality of second virtual sub-objects to be matched,

wherein performing second image recognition based on the checkerboard image to determine corresponding positions of the first and second virtual objects on the reconstructed virtual checkerboard comprises:

for each of the plurality of first virtual sub-objects to be matched and the plurality of second virtual sub-objects to be matched,

carrying out second image matching on the template image corresponding to the virtual sub-object to be matched and the chessboard image;

and determining the position of the virtual sub-object to be matched on the chessboard image as the corresponding position of the virtual sub-object corresponding to the position on the reconstructed virtual chessboard under the condition that the result of the second image matching meets a second threshold condition.

5. The information processing method according to claim 1, wherein the operation state information of the virtual chessboard includes: information indicating that the virtual chessboard is currently in a pre-match state, a mid-match state or a match end state.

6. The information processing method of claim 4, wherein determining an operation for a first virtual object in the virtual board based on the operational state information of the virtual board comprises:

in case the operation state information of the virtual chessboard indicates that the virtual chessboard is currently in a game-in state,

acquiring object information and an object position of at least one first virtual sub-object in the first virtual object;

acquiring object information and an object position of at least one second virtual sub-object in second virtual objects in the virtual chessboard; and

determining an operation for the first virtual object in the virtual chessboard based on the obtained object information and object position of at least one first virtual sub-object in the first virtual object and the obtained object information and object position of at least one second virtual sub-object in the second virtual object.

7. The information processing method of claim 4, wherein determining an operation for a first virtual object in the virtual board based on the operational state information of the virtual board comprises:

determining an operation for a first virtual object in the virtual board using a neural network model in case the operational state information of the virtual board indicates that the virtual board is currently in a hit state, wherein,

the neural network model is a convolutional neural network pre-trained with a pre-acquired sample set of operations for a first virtual object in the virtual chessboard, wherein,

the convolutional neural network includes a convolutional layer, a normalization layer, an activation layer, a residual block, and a full-link layer.

8. The information processing method of claim 7, wherein determining an operation for a first virtual object in the virtual board based on the operational state information of the virtual board comprises further comprising:

for each of the first virtual sub-object and the second virtual sub-object, generating a corresponding reconstructed virtual sub-chessboard, wherein a plurality of reconstructed virtual sub-chessboards constitute the reconstructed virtual chessboard, an

Using the plurality of reconstructed virtual chessboard as an input of the convolutional neural network.

9. The information processing method according to claim 1, wherein acquiring a board image of the virtual board comprises:

acquiring a board image of the virtual board by intercepting or externally photographing an image including the virtual board.

10. The information processing method according to claim 1,

the virtual chessboard is a virtual chessboard of a chess game, the first virtual object is a chess piece set with a first color, and the second virtual object is a chess piece set with a second color;

the first virtual object comprising at least one first virtual sub-object of a first color, the second virtual object comprising at least one second virtual sub-object of a second color, wherein,

determining, based on the operational state information of the virtual board, an operation for a first virtual object in the virtual board comprises:

an operation of determining one of the at least one first virtual sub-object comprised by the first virtual object, the operation indicating a movement of the first virtual sub-object from its current position to a target position.

11. An information processing apparatus of a virtual chessboard, comprising:

the image acquisition module is used for acquiring a chessboard image of the virtual chessboard;

the information extraction module is used for carrying out image recognition on the chessboard image so as to obtain the operation state information of the virtual chessboard;

an operation determination module for determining an operation for a first virtual object in the virtual chessboard based on the operation state information of the virtual chessboard; and

an operation providing module for providing the determined operation for the first virtual object in the virtual chessboard.

12. The information processing apparatus according to claim 11, wherein the virtual board comprises the first virtual object and a second virtual object, wherein image recognizing the board image to acquire the operation state information of the virtual board comprises:

13. The information processing apparatus according to claim 12, wherein determining, based on the operation state information of the virtual board, an operation for a first virtual object in the virtual board comprises:

14. An information processing system for a virtual board comprising:

a processor; and

memory having stored thereon computer-executable instructions for implementing the method according to any one of claims 1-10 when executed by a processor.

15. A computer-readable storage medium having stored thereon computer-executable instructions for implementing the method according to any one of claims 1-10 when executed by a processor.