CN107019901B - Method for establishing chess and card game automatic gaming robot based on image recognition and automatic control - Google Patents
Method for establishing chess and card game automatic gaming robot based on image recognition and automatic control Download PDFInfo
- Publication number
- CN107019901B CN107019901B CN201710208525.8A CN201710208525A CN107019901B CN 107019901 B CN107019901 B CN 107019901B CN 201710208525 A CN201710208525 A CN 201710208525A CN 107019901 B CN107019901 B CN 107019901B
- Authority
- CN
- China
- Prior art keywords
- game
- card
- artificial intelligence
- automatic
- chess
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 72
- 238000013473 artificial intelligence Methods 0.000 claims abstract description 30
- 230000010485 coping Effects 0.000 claims abstract description 11
- 238000000605 extraction Methods 0.000 claims description 20
- 230000006870 function Effects 0.000 claims description 7
- 238000003708 edge detection Methods 0.000 claims description 5
- 238000002790 cross-validation Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 claims description 3
- 239000013589 supplement Substances 0.000 claims description 2
- 238000000691 measurement method Methods 0.000 claims 1
- 238000012360 testing method Methods 0.000 abstract description 16
- 241000282414 Homo sapiens Species 0.000 abstract description 15
- 238000011160 research Methods 0.000 abstract description 8
- 238000009825 accumulation Methods 0.000 abstract description 2
- 238000011173 large scale experimental method Methods 0.000 abstract description 2
- 238000010998 test method Methods 0.000 abstract description 2
- 238000012549 training Methods 0.000 description 10
- 244000144730 Amygdalus persica Species 0.000 description 4
- 235000006040 Prunus persica var persica Nutrition 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 244000099147 Ananas comosus Species 0.000 description 2
- 235000007119 Ananas comosus Nutrition 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 241000694589 Gesneria viridiflora Species 0.000 description 1
- 244000183278 Nephelium litchi Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F1/00—Card games
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F1/00—Card games
- A63F1/06—Card games appurtenances
- A63F1/18—Score computers; Miscellaneous indicators
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/55—Controlling game characters or game objects based on the game progress
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F1/00—Card games
- A63F2001/008—Card games adapted for being playable on a screen
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method for establishing an automatic chess and card game gaming robot based on image recognition and automatic control. The method comprises the following steps: automatically identifying a game scene based on an image identification technology; executing corresponding operation on each game scene, and entering a game fighting game scene; identifying an action sequence of a current game participant and current game information; transmitting the identified information into an artificial intelligence computing system, and giving a coping strategy by the artificial intelligence computing system; and carrying out automatic operation according to the coping strategy given by the artificial intelligence computing system by adopting an automatic operation technology. The invention makes the machine game system and the human player possible to test in large scale, and provides a new battle test method, a large scale experiment method and a database data accumulation method for developing the game problem research of the human player in the artificial intelligence field.
Description
Technical Field
The invention belongs to the technical field of artificial intelligence and machine gaming, and particularly relates to a method for establishing an automatic chess and card game gaming robot based on an image recognition technology and an automatic control technology and combined with a machine gaming technology. The invention can provide a large-scale on-line test platform for testing human players for researchers and scientific research institutions researching machine game technology; the platform can be used for carrying out large-scale test with human players by establishing the automatic game robot, and more persuasive intelligent level verification can be provided for the field research of machine game. Meanwhile, the invention provides a method for establishing a large-scale man-machine game information database, and provides a data basis for the research represented by a deep learning method in the current artificial intelligence field. The invention is realized by the design of a game scene identification method, a game information identification method and an automatic mouse operation method.
Background
Artificial intelligence is an important branch of the computer field, and the central task of the artificial intelligence is to study how to make a computer perform work which originally can only be completed by human intelligence. Machine gaming, a research field of artificial intelligence, is one of means for checking the development level of artificial intelligence. For more than half a century, machine gaming has been an innovative hotbed for the development of artificial intelligence, and the success of this has been a significant milestone in the development history of artificial intelligence. From deep blue (chess), to immortal queen (texas poker), to AlphaGo (go), machine gaming systems challenge the human being's most intelligent in one area over another. That is, the machine gaming system may already compete with the human player. However, current machine gaming systems do not have the capability to operate on the common chess game internet platform. In order to enable the machine game system to compete with human players on the same stand, the invention adopts a method based on image recognition to enable a computer to understand all scenes in a game platform and obtain related information from the scenes.
Image recognition refers to a technique of processing, analyzing and understanding an image with a computer to recognize various different patterns of objects and objects. At present, image recognition technology is becoming more and more sophisticated, and new records are constantly being created in every image recognition competition. While the current art has not been able to achieve a hundred percent accuracy in some of the more complex recognition tasks, in the task of the present invention, image recognition is relatively simple. The main reasons are: (1) the original pictures obtained on the game platform are relatively stable and have relatively less noise; (2) the characteristics of the category to be identified are simple, and the game scene and the game information are greatly different.
Disclosure of Invention
Aiming at the technical problems in the prior art, the invention provides a method for establishing an automatic chess and card game gaming robot based on an image recognition technology and an automatic control technology. The system of the invention is actually an automatic gaming robot based on a human player internet gaming platform, and comprises the following components: a game information identification system based on an image identification technology; the 'hand': an automated operating system; and a "brain": an artificial intelligence machine gaming system. Visual information which can be recognized by human beings is converted into a data structure which can be recognized by a computer through a recognition system, strategy calculation is carried out through a machine game system, and finally operation interaction with an internet game platform is realized through an automatic operation system.
The invention relates to a method for establishing an automatic chess and card game gaming robot based on image recognition and automatic control, which comprises the following steps:
a method for establishing an automatic chess and card game gaming robot based on image recognition and automatic control comprises the following steps:
1) automatically identifying a game scene based on an image identification technology;
2) executing corresponding operation on each game scene, and entering a game fighting game scene;
3) identifying an action sequence of a current game participant and current game information;
4) transmitting the information identified in the step 3) into an artificial intelligence computing system, and giving a coping strategy by the artificial intelligence computing system;
5) and carrying out automatic operation according to the coping strategy given by the artificial intelligence computing system by adopting an automatic operation technology.
Further, the result of the game is recorded after the win or lose is found out in the game, and then the game scene is automatically identified to enter the next game.
Further, the step 1) distinguishes different game scenes by using the characteristics of each scene of the game platform; the features include RGB features for a plurality of reference locations.
Further, the action sequence of the game participants in the step 3) comprises whether the game participants turn to play cards, and the current game information comprises hands and the cards highlighted by each player.
Further, in the step 3), when the identification is carried out, firstly, a game process is converted into a picture frame by using a method of screenshot of a windows system, and the picture frame is subjected to feature extraction, and then a k neighbor-based classifier is adopted to classify the pictures.
Further, the image feature extraction method adopted in the step 3) comprises binarization, edge detection and horizontal threading feature extraction, then a proper window is selected according to the size of the image, and information in the window is counted as features.
Further, in the step 3), when the pictures are classified based on a k-nearest neighbor method, a cross validation method is adopted to select an optimal k value; the picture classification feature uses euclidean distance as the metric distance.
Further, the method of presenting the corresponding policy in step 4) depends on the specific artificial intelligence computing system implemented in the prior art, such as: chess artificial intelligence systems, etc.; the corresponding strategy should conform to the specific chess and card game rules currently applied, but the quality of the strategy does not influence the operation of the system stated by the invention.
Further, step 5) utilizes C + + to call the automatic control function of the windows system to operate on the game platform, so as to realize complete automatic computer game.
An automatic chess and card game gaming robot based on image recognition and automatic control comprises an image recognition system, an automatic operation system and an artificial intelligent computing system, wherein the image recognition system automatically recognizes game scenes by adopting an image recognition technology and recognizes the action sequence and the current game information of current game participants after entering a game fighting game scene; the artificial intelligence computing system gives coping strategies to the information identified by the image identification system; the automatic operation system adopts an automatic operation technology and carries out automatic operation according to a coping strategy given by the artificial intelligence computing system.
The invention has the main significance and beneficial effects that:
an automatic game robot for man-machine chess playing automatically does not exist in the prior art. Even in a game in which AlphaGo challenges the human litchis, the AlphaGo system still requires a person to help it perform scene recognition and implementation operations. This situation has led to training and testing in previous machine gaming system research, mainly from the fight between itself and other machine gaming systems, as only this has enabled large-scale fight testing. Occasionally, research institutes invite professional human players to perform small scale combat tests. Therefore, the invention makes the large-scale test of the machine game system and the human player possible, and provides a new battle test method, a large-scale experiment method and a database data accumulation method for developing the game problem research of the human player in the field of artificial intelligence. The invention has the function of automatic error correction, and the accuracy rate of identifying each effective information of the game platform reaches 99.9 percent.
Drawings
FIG. 1 is a general block diagram of the process of the present invention.
FIG. 2 is a flow chart of determining card placement.
FIG. 3 is a flow of feature extraction in image recognition.
FIG. 4. example of binarization in feature extraction.
Figure 5. horizontal threading feature extraction process of figure 5.
Figure 6. horizontal threading feature extraction results for the playing card face.
FIG. 7 shows an example of a statistical feature extraction result.
FIG. 8 illustrates an image recognition process.
Detailed Description
The invention is further illustrated by the following example figures and accompanying drawings.
Fig. 1 is an overall framework of the method of the present invention, and the specific steps include: automatically identifying a game scene based on an image identification technology; executing corresponding operation on each scene, and entering a game fighting scene; identifying whether the card is played by the player and the current table information (including hands and cards highlighted by each player), and classifying by adopting a k nearest neighbor-based classifier; transmitting the current table information into an AI function, and giving a coping strategy by the AI; adopting an automatic operation technology to carry out corresponding card arrangement according to the strategy given by the AI; and recording the score of the game after the game wins and loses. The following is a detailed description.
1. Identifying a game scene
The application scene of the invention is that a mobile phone virtual machine is used in a computer to run a game platform, and the game is a chess and card game. To reduce the operational flow, we focus the main functions on the larger operations associated with the battle, so assume that the game platform has been opened and logged in, and refer to the logged-in page as the home page. In scene recognition, there are a total of three scenes: and (4) identifying no scene, common fight and special fight. In each scenario, there is a corresponding operation, and here, the specific implementation of the operation is implemented by means of an API (application programming interface) based on the windows system, which will be described in detail later.
1) The unrecognized scene is: other scenes besides the battle scene are referred to, such as the main page or the popup window is not closed, and the like. Collectively referred to as anomalies. At this time, the computer is allowed to judge which kind of abnormality belongs to, and corresponding operation is selected according to the abnormality. If the player stops at the main page, clicking to match the player and entering a game fighting page; if there is a small window that is not closed, click the close button.
2) And (3) common fight: the operation on the general battle page is also the main content of the invention. It is necessary to let the computer know the table condition at this time and make corresponding operations. The common battle is divided into three cases: (1) in the first round, we will recognize the five cards dealt in the first round and other two-player cards, and transmit the information to the artificial intelligence computing system (i.e. AI in fig. 1, which may also be called machine game system), and then play the cards according to the obtained five-card playing method, and determine whether the playing is correct. And then checking the game record of the previous game, and recording the result of the previous game to count the performance of the machine game system. (2) Waiting for other players, which does not require any action, simply lets the process hibernate for a period of time to reduce computer overhead. (3) And in other rounds, the three dealt hands, the other two card placing conditions and the rest cards are identified, the information is transmitted into an artificial intelligence computing system, and then the cards are placed according to the obtained placing method, and whether the cards are placed correctly is determined.
3) Special fight: in the pineapple game, special battles can be entered in some cases, and some platforms are called as fantasy continents. The scene at this time is slightly different from the layout of the common battle, but the corresponding operations are slightly different. Firstly, identifying all issued hands and other two card-placing conditions, transmitting the information into an artificial intelligence computing system, then placing cards according to the obtained placing method, and determining whether the card-placing is correct. And then checking the game record of the previous game, and recording the result of the previous game to count the performance of the machine game system.
In the process of playing cards, it is necessary to determine whether the playing cards are correct. In the actual running process, some exceptions which can be handled by the game platform itself can be encountered, for example, when the network connection is unstable, the situation of playing cards can occur: in the process of operating the mouse, the network connection is disconnected. At this time, any card swinging operation cannot be successful, namely, the mouse does not have any effect when being operated in the absence. In order to solve the problem, a scheme for determining the playing cards is established, and the specific process is shown in fig. 2, and the specific method is as follows: after the program operation is finished, identifying whether a card exists in the target position, and if the card does not exist, then placing the card again; if yes, identifying whether the card is the card which should be placed in the operation, if not, removing the card and placing the card again.
2. Image feature extraction
In the task of image recognition, the extraction of image features plays a very important role, and a good feature expression has a very large influence on the accuracy of the subsequent recognition. The existing image feature extraction methods are many, such as Histogram of Oriented Gradient (HOG) features, Local Binary Pattern (LBP) features, Scale Invariant Feature Transform (SIFT) features, and the like. In the task of the present invention, the contents of pictures to be recognized are relatively simple, but the number of times of recognition is relatively large in order to keep up with the game flow, and therefore, the amount of calculation needs to be reduced and the speed of picture recognition needs to be increased. For this reason, the present invention employs a simpler statistical information as a feature without affecting the accuracy of subsequent identification.
The main feature extraction process of a picture I is shown in FIG. 3, and specifically includes the following steps:
1) binarization: the binarization is the first step of all operations and is mainly used for eliminating the influence of animation and light shadow masks in games on picture identification. The binarization function is to convert the picture from the time-limited mode of the gray RGB value to 01 display. Fig. 4 is an example of a binarized picture, in which one picture is on the right and the result of binarization processing is on the left.
2) Edge detection: the purpose of edge detection is to align the recognition regions to improve recognition efficiency and accuracy. The target of edge detection in this embodiment is to align the coordinate point at the upper left corner of the recognition area. This is achieved by identifying the crossing pixel points where 1 appears in the first column and 1 appears in the first row.
3) Flower color identification: the pattern recognition is a simpler recognition step in the system, but plays a key role in improving the recognition speed of the system. The specific method comprises the following steps: the suit of the playing card is determined by taking the RGB value of the pixel in a specific area of the playing card. The suit characteristics determined by the system are shown in table 1.
Table 1: pattern and color identification characteristic interval
| Playing card suit | R | G | |
| Square sheet | |||
| 0～25 | 0～25 | 200～255 | |
| |
0～25 | 200～255 | 0～25 |
| Red peach | 200～255 | 0～25 | 0～25 |
| Black peach | 200～255 | 200～255 | 200～255 |
| Others | -- | -- | -- |
The system confirms the suit of the playing cards through the upper table, if the suit does not fall into the identification section of any suit, the card is identified as a non-playing card or a card back pattern, and the system jumps out of the identification process of the playing card. Otherwise, the suit identification result is combined with the playing card contents identified by other processes to form a final result. For example: black peach + K ═ black peach K.
4) Horizontal threading feature extraction: the horizontal threading characteristic is one of the keys for identifying the playing cards in the system, and the method is to acquire the pixel characteristic in the horizontal direction by transversely identifying the 01 distribution in the playing cards. In the embodiment, 6 characteristics of the playing cards are collected totally, which are respectively as follows: a horizontal midline feature, an upper 1/4 line feature, a lower 1/4 line feature, and the first 1 feature to which the three features correspond. The method for obtaining the features is transverse traversal, and the intersection of the horizontal line and a plurality of pixel points consisting of 1 is recorded. The first 1 characteristic refers to: the first intersection is on the left or right side of the identification area, the left side is 1 and the right side is 0.
Figure 5 shows the numeric threading characteristics of the number 5 in the playing card, and it can be seen that the number of 1's of 3 threading crossings is 1, 1 respectively, and the positions where 1 is encountered first are right, left and right respectively, so the numeric threading characteristics of 5 is 111010.
Figure 6 shows the results of horizontal threading feature extraction for all playing card faces.
5) And (3) extracting the characteristics of the regional digital columns: the module divides the identification area of the playing card into 9 areas of 3 x 3, and horizontal threading feature extraction is carried out again to supplement the global threading feature, and the specific process is similar to the step 4), and the process is not described again.
6) And (3) feature comparison: selecting a proper window w according to the size of the picture I, counting information in the window w as characteristics, wherein the ratio of pixel points of the content is counted, namely the ratio of the pixel points with the counting value of 1 in the window, because the difference between the background of the original picture and the content is obvious; the statistical feature vector X of picture I is represented by the statistical information of all windows. Namely:
X＝(f(w1),…,f(wn))
wherein f (w)i) Is shown in window wiThe proportion of the pixels with the median value of 1.
Fig. 7 is an example of a feature extraction result. As shown in fig. 7, we obtain the original image (playing card K) obtained by clipping, then perform feature extraction on the original image to obtain the image, and obtain the final feature vector X by using an 8 × 10 window w. The window is adjustable, and the dimension of the final feature vector is mainly fixed, so that the next operation is facilitated.
3. Image recognition
Fig. 8 shows the process of image recognition: firstly, preprocessing the picture, mainly performing operations such as cutting and the like on the picture to intercept the picture of the key area. And then, extracting features, inputting the features into a k-nearest neighbor classifier, and finally obtaining a class label of the picture.
For image recognition, it is necessary to prepare enough training data sets for the k-neighbor classifier, i.e., to perform data set collection. The method adopted here is to intercept the required data from the game platform, including different playing cards, scores of winning or losing, and the like. The specific method comprises the following steps: and counting the positions and the sizes of various data in the game platform, then carrying out screen capture by using a program, and capturing the corresponding positions and sizes.
After extracting the features of the image, a classifier is needed to classify the features to obtain the required information. In the task of the invention, it is desirable to identify the game information of different platforms in the same way and to reduce the extra operations when adapting to different platforms. The k-means neighbor method (k-NN) was found to meet the above requirements by investigation. Firstly, it works well in simple tasks and is a classifier that does not require explicit learning, i.e. does not need to be trained in advance on a training data set. This means that only the game information data set of the platform to be adapted needs to be collected when adapting the platform.
The k-nearest neighbor method is a basic classification and regression method. Its input is the eigenvector of the example, corresponding to the points of the eigenspace; the output is the category of the instance. The k-nearest neighbor method assumes that a training data set is given, in which the class of instances is determined. And when classifying, predicting the new examples by means of majority voting and the like according to the categories of the k nearest neighbor training examples. The main problems are: k value selection, distance measurement and classification decision rules.
k-nearest neighbor algorithm:
1) inputting a training set:
T＝{(x1,y1),(x2,y2),……(xN,yN)}，
wherein,
is a feature vector, yi∈Y＝{c1,c2…ckIs category;
2) neighborhood Nk(x) The method comprises the following steps A neighborhood of x encompassing the k points;
3) determining the category y of x:
the following describes the specific selection of three elements in the task:
a) and k value selection:
k points in the training set adjacent to x. If the k value is too small, it means that the overall model becomes complex and overfitting easily occurs, i.e. if the neighboring instance points happen to be noise, the prediction will be erroneous. An increase in the value of k means that the overall model becomes simple, and in the extreme case k is equal to N, where N represents the number of instances in the training set, and then whatever the input instance is, it is simply predicted that it belongs to the most classes in the training set, and such a model is too simple. The k value is generally a relatively small value, and a cross-validation method is usually adopted to select the optimal k value. That is, the value of k is chosen to pass multiple tests to ensure a good result.
After a number of trials, different k values were determined for different recognition tasks. For example, in the identification of playing cards, the value of k is selected to be 1, i.e., the nearest neighbor method is used here.
b) Distance measurement:
wherein L isp(xi,xj) Representing a feature vector xiAnd xjN represents the number of features, and is a manhattan distance when p is 1, and is a euclidean distance when p is 2. In our task we choose p-2.
c) Classification decision rules: generally, a majority voting rule, i.e., using the majority class of k training instances in the result as the class of x.
4. Automated operating system
The automatic operating system mainly enables a program to be operated on a game platform, realizes complete automatic computer game and does not need human intervention in the fighting process. And C + + can be used for calling the automation control function of the windows system to operate on the game platform. Specifically, after the program and the corresponding game platform (the game platform runs on a virtual machine of a mobile phone) are opened, the program obtains a handle of the virtual machine by means of an API of a windows, obtains a position of a display window of a chess and card game in the windows system, which needs to be identified, and activates the window. There are two main reasons for this:
(a) if the game platform is completely or partially covered by other programs, when the information of the covered part needs to be covered, the captured picture cannot reflect the content actually occurring in the game platform, so that the subsequent information identification has no effect, and the game system is adversely affected.
(b) If the game platform is completely or partially covered by other programs, the mouse cannot act on the game platform when the mouse is required to operate at the position of the game platform, so that the abnormal conditions or the cards cannot be processed in real time.
After the window is activated, the program will perform the process of automatic gaming, in which screenshot is required to identify relevant information and the mouse is operated to complete necessary actions. The method comprises the following steps:
(1) screenshot
When the image is captured, the whole interface can be captured, and then the regions are matched, so that the required information can be obtained. However, such a procedure is very expensive in computer, and requires consumption of control time as a real-time match. Considering that the layout of the game platform cannot be easily changed, the following is taken: first, for a game platform, it is necessary to collect useful information in each scene and its position relative to the virtual machine window, and size, such as the position and size of the hand, the position and size of the game score, etc. Then, when information of a certain size needs to be identified actually, the position of the window and the position of the information relative to the window can be used for positioning the information in the screen. Finally, a picture of the size of the position of the screen is intercepted.
(2) Operating mouse
Windows provides a very well-functioning API to control mouse movement or clicking, which facilitates automated operation of the present invention. Similar to the screenshot, information related to each operation needs to be collected, for example, the start position and the destination position of the card are needed for the card swing operation, and the action of clicking the card is needed for the dragging process. And then rely on the collected information to implement with the corresponding API interface when the operation is needed. It should be noted that in two consecutive mouse operations, the program needs to be suspended for a while to allow the game platform to respond to the operation. 0.3 seconds is used in this embodiment.
Table 2 shows the test data of a single PC computer in 2017 in 24-3-14 days of the system of the invention on the domestic game platform joint. The total score of the test is 10702 minutes and the average score of each round is 1.39 minutes when the test is carried out for a total of 7691 discs in 10 days. In addition to the contribution of the AI system, the present invention also makes a key contribution: 1. the recognition accuracy is 98.87%, and the correct calculation of the AI system is basically guaranteed. 2. The automatic operation accuracy rate is 95.65%, the normal operation of the system is guaranteed under the conditions of normal and short-time disconnection and the like, the average testing time per day exceeds 20 hours, and the total number of testing stations per day exceeds 700. It can be seen that if the present invention is applied to more platforms and loaded on more computers, the accumulated amount of test data will be further greatly increased.
Table 2: test data of system in crowd platform big pineapple game
The above embodiments are only intended to illustrate the technical solution of the present invention and not to limit the same, and a person skilled in the art can modify the technical solution of the present invention or substitute the same without departing from the spirit and scope of the present invention, and the scope of the present invention should be determined by the claims.
Claims (8)
1. A method for establishing an automatic chess and card game gaming robot based on image recognition and automatic control comprises the following steps:
1) automatically identifying a game scene based on an image identification technology; the game scenes comprise unidentified scenes, common battles and special battles, wherein the unidentified scenes are other scenes except the battle scenes;
2) executing corresponding operation on each game scene, and entering a game fighting game scene;
3) identifying an action sequence of a current game participant and current game information; during the identification, firstly converting a game process into a picture frame by using a method of screenshot of a windows system, extracting the characteristics of the picture frame, and then classifying the picture by using a k neighbor-based classifier; the feature extraction includes:
3.1) binarization: the method is used for eliminating the influence of animation and light shadow masks in the game on picture identification;
3.2) edge detection: aiming at aligning the identification area to improve the identification efficiency and accuracy;
3.3) flower color identification: determining the suit according to the RGB value of the pixels in the chess and card specific area;
3.4) horizontal threading feature extraction: acquiring pixel characteristics in the horizontal direction by transversely identifying 01 distribution in the chess;
3.5) extracting the characteristics of the regional digital columns: dividing the chess and card identification area into a plurality of areas, and performing horizontal threading feature extraction again to supplement global threading features;
3.6) feature alignment: selecting a proper window according to the size of the picture, and counting information in the window as characteristics;
4) transmitting the information identified in the step 3) into an artificial intelligence computing system, and giving a coping strategy by the artificial intelligence computing system;
5) and carrying out automatic operation according to the coping strategy given by the artificial intelligence computing system by adopting an automatic operation technology.
2. The method of claim 1, wherein the result of the game is recorded after the win or loss of the game, and then a game scene is automatically recognized to enter the next game.
3. The method of claim 1, wherein step 1) uses the characteristics of each scene of the game platform to distinguish different game scenes; the features include RGB features for a plurality of reference locations.
4. The method of claim 1, wherein the sequence of actions of the game participants of step 3) includes whether it is their turn to play, and the current game information includes hands, cards that each player has highlighted.
5. The method as claimed in claim 1, wherein, in the step 3), when the pictures are classified based on a k-nearest neighbor method, a cross validation method is adopted to select an optimal k value; the distance measurement method of the image classification features comprises the following steps:
euclidean distance is used as the metric distance.
6. The method of claim 1, wherein step 5) utilizes C + + calls to automate control functions of the windows system to operate on the game platform to effect a full computer automated game.
7. The method of claim 1, wherein the following method is used to determine whether the hand is correct during the hand-placing: after the program operation is finished, identifying whether a card exists in the target position, and if the card does not exist, then placing the card again; if yes, identifying whether the card is the card which should be placed in the operation, if not, removing the card and placing the card again.
8. An automatic chess and card game gaming robot based on image recognition and automatic control by adopting the method of any one of claims 1-7 is characterized by comprising an image recognition system, an automatic operating system and an artificial intelligence computing system, wherein the image recognition system automatically recognizes a game scene by adopting an image recognition technology and recognizes the action sequence and the current game information of current game participants after entering the game fighting game scene; the artificial intelligence computing system gives coping strategies to the information identified by the image identification system; the automatic operation system adopts an automatic operation technology and carries out automatic operation according to a coping strategy given by the artificial intelligence computing system.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710208525.8A CN107019901B (en) | 2017-03-31 | 2017-03-31 | Method for establishing chess and card game automatic gaming robot based on image recognition and automatic control |
| PCT/CN2017/089220 WO2018176650A1 (en) | 2017-03-31 | 2017-06-20 | Method for building robot capable of automatically playing board and card games based on image recognition and automated control |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710208525.8A CN107019901B (en) | 2017-03-31 | 2017-03-31 | Method for establishing chess and card game automatic gaming robot based on image recognition and automatic control |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107019901A CN107019901A (en) | 2017-08-08 |
| CN107019901B true CN107019901B (en) | 2020-10-20 |
Family
ID=59526730
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710208525.8A Expired - Fee Related CN107019901B (en) | 2017-03-31 | 2017-03-31 | Method for establishing chess and card game automatic gaming robot based on image recognition and automatic control |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107019901B (en) |
| WO (1) | WO2018176650A1 (en) |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108090561B (en) * | 2017-11-09 | 2021-12-07 | 腾讯科技(成都)有限公司 | Storage medium, electronic device, and method and device for executing game operation |
| CN109508789B (en) * | 2018-06-01 | 2022-03-15 | 北京信息科技大学 | Method, storage medium, processor and apparatus for predicting hand |
| CN109529352B (en) * | 2018-11-27 | 2023-03-28 | 腾讯科技（深圳）有限公司 | Method, device and equipment for evaluating scheduling policy in virtual environment |
| CN110163238B (en) * | 2018-12-13 | 2023-04-07 | 腾讯科技（深圳）有限公司 | Information prediction method, model training method and server |
| CN109857663B (en) * | 2019-01-26 | 2022-04-12 | 北京工业大学 | Keyword driving and image similarity combined automatic test platform |
| CN111598976B (en) | 2019-02-01 | 2023-08-22 | 华为技术有限公司 | Scene recognition method and device, terminal and storage medium |
| CN110456975A (en) * | 2019-07-26 | 2019-11-15 | 成都龙渊网络科技有限公司 | A kind of method and apparatus executing touch control operation on touching terminal |
| CN110665235A (en) * | 2019-11-05 | 2020-01-10 | 淮安鱼鹰航空科技有限公司 | Unmanned aerial vehicle amusement system that targets |
| CN110909630B (en) * | 2019-11-06 | 2023-04-18 | 腾讯科技（深圳）有限公司 | Abnormal game video detection method and device |
| CN111359213B (en) * | 2020-03-24 | 2022-11-25 | 腾讯科技（深圳）有限公司 | Method and apparatus for controlling virtual players in game play |
| CN111437608B (en) * | 2020-03-24 | 2023-09-08 | 腾讯科技（深圳）有限公司 | Game play method, device, equipment and storage medium based on artificial intelligence |
| CN112057876A (en) * | 2020-09-10 | 2020-12-11 | 重庆五诶科技有限公司 | Automatic settlement system and method based on personalized rules |
| CN112560872A (en) * | 2020-12-16 | 2021-03-26 | 北京曲奇智能科技有限公司 | Multimode perception mahjong assisting method based on artificial intelligence |
| CN113082711B (en) * | 2021-03-22 | 2023-08-29 | 北京达佳互联信息技术有限公司 | Game robot control method, game robot control device, server and storage medium |
| CN113546399B (en) * | 2021-07-30 | 2024-01-23 | 重庆五诶科技有限公司 | Chess and card game data acquisition system and method based on artificial intelligence algorithm |
| CN114140259A (en) * | 2021-11-29 | 2022-03-04 | 中国平安财产保险股份有限公司 | Artificial intelligence-based wind control method, device, equipment and storage medium for underwriting |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2527324Y (en) * | 2002-03-26 | 2002-12-25 | 龙建 | Chess set distinguished and operated by machine |
| CN102034116B (en) * | 2010-05-07 | 2013-05-01 | 大连交通大学 | Commodity image classifying method based on complementary features and class description |
| CN103405909A (en) * | 2013-08-05 | 2013-11-27 | 昆山塔米机器人有限公司 | Unattended chess-playing robot system |
| CN103802111A (en) * | 2013-12-23 | 2014-05-21 | 北京晨鑫意科技有限公司 | Chess playing robot |
| CN106182006B (en) * | 2016-08-09 | 2018-07-27 | 北京光年无限科技有限公司 | Chess and card interaction data processing method towards intelligent robot and device |
-
2017
- 2017-03-31 CN CN201710208525.8A patent/CN107019901B/en not_active Expired - Fee Related
- 2017-06-20 WO PCT/CN2017/089220 patent/WO2018176650A1/en active Application Filing
Also Published As
| Publication number | Publication date |
|---|---|
| WO2018176650A1 (en) | 2018-10-04 |
| CN107019901A (en) | 2017-08-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107019901B (en) | Method for establishing chess and card game automatic gaming robot based on image recognition and automatic control | |
| WO2020151489A1 (en) | Living body detection method based on facial recognition, and electronic device and storage medium | |
| US12070693B2 (en) | System and method for toy recognition | |
| CN108090561B (en) | Storage medium, electronic device, and method and device for executing game operation | |
| US20090060352A1 (en) | Method and system for the detection and the classification of events during motion actions | |
| CN109117897A (en) | Image processing method, device and readable storage medium storing program for executing based on convolutional neural networks | |
| CN102819749A (en) | Automatic identification system and method for offside of football based on video analysis | |
| CN108764447A (en) | A kind of group robot Majiang game intelligence dynamicization system and mahjong identification learning algorithm | |
| WO2013075295A1 (en) | Clothing identification method and system for low-resolution video | |
| CN105631405B (en) | Traffic video intelligent recognition background modeling method based on Multilevel Block | |
| CN116612386A (en) | Pepper disease and pest identification method and system based on hierarchical detection double-task model | |
| Jin | Original Research Article Video analysis and data-driven tactical optimization of sports football matches: Visual recognition and strategy analysis algorithm | |
| Gao et al. | Design of an efficient multi-objective recognition approach for 8-ball billiards vision system | |
| CN115193059A (en) | Shooting game cheating detection method | |
| CN103632380A (en) | On-line game playing card identification method based on key point decision trees | |
| Chen et al. | Object-oriented state abstraction in reinforcement learning for video games | |
| CN117475187A (en) | Method, device, equipment and storage medium for training image classification model | |
| Ranasinghe et al. | ChessEye: An integrated framework for accurate and efficient chessboard reconstruction | |
| Sunal et al. | Real time fencing move classification and detection at touch time during a fencing match | |
| JP6593928B2 (en) | Information processing apparatus and program | |
| CN110415256A (en) | The fast multi-target recognition methods of view-based access control model and system | |
| US20180211397A1 (en) | Object extraction from video images system and method | |
| Huszár et al. | Securing Phygital Gameplay: Strategies for Video-Replay Spoofing Detection | |
| CN108710433A (en) | A kind of scene application artificial intelligence exchange method and its system | |
| Pham et al. | ATGW: A Machine Learning Framework for Automation Testing in Game Woody |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20201020 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |