CN116797728A - AR-based billiard game referee method, device, storage medium and AR glasses - Google Patents

Abstract

The application discloses an AR-based billiard game judging method, device and equipment and a storage medium. The method comprises determining billiard table position information and a plurality of billiard position information on the billiard table; determining identity information of players participating in a billiard game; obtaining ball condition information of each billiard ball on the billiard table through tracking so as to score corresponding players according to a scoring rule of billiard match; when the white ball needs to be reset, determining the position of the restored white ball, and displaying the position of the restored white ball on a billiard table in a terminal equipment screen and/or in a live broadcast picture in an AR mode; and displaying the scores of the players in the form of AR on a screen of the terminal device so that the referee can read the scores in the competition process and/or judge the competition result according to the scores after the competition is finished. The embodiment of the application can realize AR virtual-real fusion display in billiard games.

Description

AR-based billiard game referee method, device, storage medium and AR glasses

Technical Field

The application relates to the technical field of augmented reality (Augmented Reality, AR for short), in particular to an AR-based billiard game judging method, an AR-based billiard game judging device, an electronic device, a computer-readable storage medium, a computer program product and AR glasses.

Background

Billiards are also called billiards or billiards, and are one of ball sports. Snooker, which means "hindrance, obstacle" also called handicap billiards, is one type of billiard game. The stanok is taken as a gentleman sport, in the process of the stanok competition, the judgment of scoring and rules has complex regulations, and the referee needs to pay attention to the operation of contestants and to score, so that the referee needs to have extremely high understanding on the rules and the competitive characteristics of the contest; when the ball-solving foul needs to be hit again by contestants, the referee is required to accurately restore the position of the white ball; during the game, the referee needs to memorize the hit score of the player in real time and read out the hit score in real time. The above all require that the referee of the snooker is concentrated in spirit at any time in the competition, and the spirit is in a relatively stressed state, so that referee errors are easy to occur.

Disclosure of Invention

In view of the above, embodiments of the present application provide an AR-based billiard game referee method, apparatus, device, computer-readable storage medium, computer program product, and AR glasses for solving at least one technical problem.

In a first aspect, an embodiment of the present application provides an AR-based billiard game referee method, including: determining the position information of a billiard table and the position information of a plurality of billiards on the billiard table; determining identity information of players participating in a billiard game; in the billiard competition process, the ball condition information of each billiard ball on a billiard table is obtained through tracking, so that scoring is carried out on the corresponding player according to the scoring rule of the billiard competition; wherein, the ball condition information comprises the color, the ball position and the information of whether the billiards fall into the bags or not; when the white ball needs to be reset, determining the position of the restored white ball, and displaying the position of the restored white ball on a billiard table in a terminal equipment screen and/or in a live broadcast picture in an AR mode; and displaying the scores of the players in the form of AR on a screen of the terminal device so that the referee can read the scores in the competition process and/or judge the competition result according to the scores after the competition is finished.

Optionally, the method further comprises: if the billiards falling into the bags are white, judging information is displayed on a screen of the terminal equipment according to the competition rules to be foul.

Optionally, the determining the position of the restored white ball includes: and acquiring the position of the white ball restored by the eagle eye system.

Alternatively, a goal route of the player is predicted, and the prediction result is displayed in an AR form in the live view.

Optionally, the walking mode of the player is predicted, and the prediction result is displayed in the live broadcast picture in an AR form.

Optionally, the method further comprises: and calculating and outputting the prediction result by a machine learning model, wherein a learning sample of the machine learning model is a previous billiard game video of the player.

In a second aspect, an embodiment of the present application provides an AR-based billiard game referee device, including: the system comprises a first determining module, a second determining module, a scoring processing module, a resetting processing module and a display processing module. The first determining module is used for determining the position information of the billiard table and the position information of a plurality of billiards on the billiard table; a second determining module for determining identity information of players participating in the billiard game; the scoring processing module is used for tracking and acquiring the ball condition information of each billiard ball on the billiard ball table in the billiard ball competition process so as to score the corresponding player according to the scoring rule of the billiard ball competition; wherein, the ball condition information comprises the color, the ball position and the information of whether the billiards fall into the bags or not; the reset processing module is used for determining the position of the restored white ball when the white ball needs to be reset, and displaying the position of the restored white ball on a billiard table in a screen of the terminal equipment in an AR form and/or in a live broadcast picture; and the display processing module is used for displaying the scores of the players on a screen of the terminal equipment in an AR form so as to enable judges to read the scores in the competition process and/or judge competition results according to the scores after the competition is finished.

In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the method of any of the above.

In a fourth aspect, embodiments of the present application provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement a method as claimed in any one of the preceding claims.

In a fifth aspect, embodiments of the present application provide a computer program product comprising computer program instructions which, when executed by a processor, implement a method as claimed in any one of the preceding claims.

In a sixth aspect, embodiments of the present application provide AR glasses comprising a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, causes the AR glasses to implement the method of any one of the preceding claims.

The embodiment of the application can identify the conditions of batting, bagging and the like of players in real time in the billiard competition process, realize automatic scoring and prompt a referee to read the score. When the white ball is required to be reset, the application can accurately restore and AR display the position of the white ball, thereby facilitating the referee to intuitively and rapidly reset the white ball, not only accelerating the game rhythm, but also reducing the working pressure of the referee. In addition, by means of the embodiment of the application, the batting position route, technical habit and the like of the player can be predicted through machine learning, and the prediction result is displayed in the live broadcast picture in an AR mode, so that the participation of the audience can be improved, and the better viewing experience is brought.

Drawings

In order to more clearly describe the technical solution of the embodiments of the present application, the following description briefly describes the drawings in the embodiments of the present application.

Fig. 1 is a schematic diagram of an AR system architecture based on a server and a terminal device according to an embodiment of the present application.

Fig. 2 is a schematic diagram of a virtual-real fusion image for AR navigation by using a mobile phone APP.

FIG. 3 is a block flow diagram of an AR-based billiard game referee method according to an embodiment of the present application.

Fig. 4 is a schematic diagram of a situation in which a referee resets a white ball in an embodiment of the present application.

Fig. 5 is a schematic view of a scene of an embodiment of the application looking at a white ball presentation through the eagle eye system.

Fig. 6 is a schematic view of an effect scenario in which an embodiment of the present application predicts and presents a player's goal route.

Fig. 7 is a block diagram showing the construction of an AR-based billiard game referee device according to an embodiment of the present application.

Fig. 8 is a schematic structural view of an electronic device for implementing an AR-based billiard game referee method according to an embodiment of the present application.

Fig. 9 is a schematic software architecture of an exemplary terminal device according to an embodiment of the present application.

Detailed Description

The principles and spirit of the present application will be described below with reference to several exemplary embodiments. It will be appreciated that such embodiments are provided to make the principles and spirit of the application clear and thorough, and enabling those skilled in the art to better understand and practice the principles and spirit of the application. The exemplary embodiments provided herein are merely some, but not all embodiments of the application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the embodiments herein, are within the scope of the present application.

Those skilled in the art will appreciate that embodiments of the application may be implemented as a system, apparatus, device, method, computer readable storage medium, or computer program product. Accordingly, the present application may be embodied in at least one of the following forms: complete hardware, complete software, or a combination of hardware and software. According to the specific embodiment of the application, the application claims an AR-based billiard game judging method, an AR-based billiard game judging device, AR-based billiard game judging equipment, an AR-based billiard game judging device, a AR-based billiard game judging method, AR-based billiard game judging device, a AR-based billiard game judging device and a AR-based billiard game judging computer program product.

In this document, terms such as first, second, third, etc. are used solely to distinguish one entity (or action) from another entity (or action) without necessarily requiring or implying any order or relationship between such entities (or actions).

The embodiment of the application can be applied to a server and terminal equipment. Referring to fig. 1, a schematic diagram of an AR system architecture based on a server and a terminal device is schematically shown. The AR system architecture comprises a server 10 and several terminal devices 20. In some examples, the terminal device 20 is an AR device, which may be a dedicated AR device, such as a Head-mounted display (HMD), smart glove, apparel, or other smart wearable electronic device. In some examples, the terminal device 20 may be a general purpose AR device, such as a cell phone, portable computer, notebook computer, tablet computer, virtual Reality (VR) device, vehicle mounted device, navigation device, game device, and the like.

Taking an AR helmet or AR glasses as an example, a head-mounted display, a machine vision system, a mobile computer and the like can be integrated and arranged in a wearable device, the device is provided with a display similar to the glasses in appearance, and is worn on the head of a user in operation, and the device can transmit augmented reality information to the display or projected to eyeballs of the user, so that the visual immersion of the user is enhanced. In some examples, the AR device also has a camera, which may be a wide angle camera, a tele camera, or a structured light camera (also referred to as a point cloud depth camera, a 3D structured light camera, or a depth camera). The structured light camera is based on a 3D vision technology, and can acquire plane and depth information of an object. The structured light camera can project light with certain structural characteristics onto a photographed object through the near infrared laser, then the infrared camera collects reflected light, the reflected light is processed by the processor chip, and the calculation principle is that the position and depth information of the object are calculated according to the change of light signals caused by the object, and a 3D image is displayed. The conventional terminal equipment, such as a mobile phone, presents a two-dimensional image, cannot display the depth of different positions on the image, and can acquire 3D image information data by shooting through a structured light camera, namely, not only can acquire information of colors and the like of different positions in the image, but also can acquire depth information of different positions, and can be used for AR ranging. Of course, the common terminal device can acquire the 2D image based on the optical camera and acquire the depth information of the 2D image by combining a deep learning algorithm and the like, and finally can also present the 3D image.

In some examples, the terminal device 20 has installed therein AR-enabled software or an application APP. The server 10 may be a management server or an application server of the software or APP. The server 10 may be one server, or may be a server cluster formed by a plurality of servers, or may be a cloud server or cloud server. The terminal device 20 has integrated therein a module having a networking function, such as a Wireless-Fidelity (Wifi) module, a bluetooth module, a 2G/3G/4G/5G communication module, etc., so as to be connected to the server 10 through a network.

Illustratively, a user may log into the user account through an APP installed in the cell phone, and the user may also log into the user account through software installed in the AR glasses.

Taking APP with an AR navigation function as an example, the APP may have, for example, a high-precision map navigation capability, an environment understanding capability, a virtual-real fusion rendering capability, and the like, and the APP may report current geographic location information to the server 10 through the terminal device 20, where the server 10 provides an AR navigation service for a user based on the real-time geographic location information. Taking the terminal device 20 as a mobile phone as an example, in response to a user starting an APP, the mobile phone may start a camera to collect an image of a real environment, then perform AR augmentation on the image of the real environment collected by the camera through the system, blend or superimpose a rendered AR effect (such as a navigation route identifier, a road name, merchant information, advertisement presentation, etc.) in the image of the real environment, and present the image of the virtual-real fusion on a screen of the mobile phone.

Fig. 2 schematically shows a virtual-real fusion image of AR navigation by using a mobile phone APP, wherein an indication arrow of AR navigation is superimposed on a real road surface and in a space in the figure, and electronic resources promoted by merchants float at a designated position in the space in the form of a parachute carrying gift box.

Embodiments of the present application relate to a terminal device and/or a server. The principles and spirit of the present application will be explained in detail below by means of several exemplary examples or representative embodiments.

First, related contents such as concepts and technical terms which may be related to the embodiments of the present application will be briefly described.

Rules relating to billiard games

The general scoring rules for billiard games include two types of rules: american billiard scoring rules and English billiard scoring rules. Taking the scoring rules of a billiard as an example, the goal of each game is to hit a specified number of balls with a minimum number of pins. The players can break the hair in turn before the game starts, the broken hair side needs to hit the red ball with the white ball first, and then can continue to hit balls with other colors (yellow, green, brown, blue, powder and black) until the fault or no ball can be hit. The balls of each color have a fixed point number, and finally, after all the balls of all the colors are put into the bag, the scoring player wins. After a successful hit by one of the players and any of the balls in the pocket, the player will continue to hit the ball until the ball is not entered or a penalty is made. The penalty may result in an opponent obtaining the ball weight and may choose to reinstall the location of the volleyball. The foul of a billiard ball includes: no balls are driven into the pockets, white balls are driven into any one of the pockets, no contact is made with the ball to be driven, balls that have been driven into the pockets by an opponent are driven into the pockets again, and so on.

(II) related art terms that may be referred to

Regarding VSLAM (Visual Simultaneous Localization and Mapping), the technique of simultaneous localization and mapping for vision is a method of combining computer vision technology and robot navigation technology. The method is based on the fact that sensors such as cameras and laser radars acquire surrounding environment information, the surrounding environment information is processed into a map through an algorithm, and the position of a robot in the map is located.

The implementation of VSLAM technology can be roughly divided into the following steps:

1) Feature extraction: and extracting feature points from the continuous video frames of the surrounding environment acquired by the camera through an image processing algorithm.

2) Feature matching: the feature points of two adjacent frames are matched to determine their actual positions in physical space.

3) Motion estimation: and estimating the motion trail of the equipment according to the matching result of the characteristic points.

4) And (3) building a diagram: and carrying out three-dimensional reconstruction on the estimated motion trail and the matched characteristic point positions to construct an environment map.

5) Position estimation: and estimating the position of the equipment in the map by using the map and the current sensor data of the equipment, and realizing positioning.

Regarding the positioning of the eagle eye system, the eagle eye system is widely applied to indoor positioning, particularly to the competitive sports, and can achieve more accurate positioning, for example, the eagle eye system can be used for restoring the position of a white ball in billiard games.

In terms of technical implementation, the eagle eye system can acquire indoor object position information by means of indoor positioning technologies such as WiFi positioning, bluetooth BLE positioning, ultrasonic positioning and the like. These techniques measure and analyze signals by placing multiple beacons or receivers indoors to determine device location. By comparing with beacons with known positions, the position and direction of the device can be calculated, so that the accurate positioning of the indoor object is realized.

Snooker is a popular billiard sport and its playing rules are complex, making the work of referees very burdensome. In the course of billiard game, the work that the referee needs to do mainly includes: confirming rules of the game, ordering the game, maintaining order of the game, scoring, supervising player behavior, issuing warnings and penalties, declaring results of the game, etc. Therefore, in order to ensure that the game is fairly and orderly played throughout the course of the game, the referee needs to keep extremely high alertness and concentration all the time, so that there is a high demand on the referee both mentally and physically.

FIG. 3 is a block flow diagram of an AR-based billiard game referee method according to an embodiment of the present application, comprising the steps of:

s101, determining position information of a billiard table and position information of a plurality of billiards on the billiard table;

s102, determining identity information of players participating in a billiard match;

s103, in the billiard competition process, obtaining the ball condition information of each billiard ball on a billiard table through tracking so as to score corresponding players according to the scoring rules of the billiard competition; wherein, the ball condition information comprises the color, the ball position and the information of whether the billiards fall into the bags or not;

s104, when the white ball needs to be reset, determining the position of the restored white ball, and displaying the position of the restored white ball on a billiard table in a screen of terminal equipment in an AR form and/or in a live broadcast picture;

s105, displaying the scores of the players on a screen of the terminal device in an AR mode so that judges can read the scores in the competition process and/or judge competition results according to the scores after the competition is finished.

By using the embodiment of the application, a referee can use AR terminal equipment (such as AR glasses, AR helmets and the like) to have the function of identifying the field match in real time in the billiard match process, can identify the field match condition in real time, and can use a computer to visually track the ball position condition of different color balls and the billiard bag falling condition to automatically score, prompt, judge and score, and be used for judging the win-lose condition of the match and the like. For the case of re-striking due to ball-solving foul, the referee needs to accurately restore the position of the white ball. The application can display the position where the white ball is supposed to be restored through the AR glasses augmented reality technology, and assist the referee to better place the white ball at the position before the foul, thereby not only accelerating the competition rhythm, but also reducing the working pressure of the referee. By utilizing the face recognition and tracking technology, the application can also prompt the match data of the current player (such as the goal condition, single score, hit times, foul times, total score, current score and the like of the current player) for the referee through the AR terminal equipment. The embodiment of the application can display the results of the existing automatic scoring and referee system by using the AR augmented reality technology, and inform referees through AR glasses to assist the referees to judge the race fouls and the like, thereby greatly improving the timeliness and accuracy of referees.

In some embodiments, optionally, if the billiards falling into the bag are white balls, the judge information is displayed as a foul on the screen of the terminal device according to the rules of the match. The reason for this is that in the snooker game, the white ball acts to hit the other colored ball and cause it to fall into a pocket. White ball drop bags are not normally allowed. By utilizing the embodiment of the application, the referee can be timely prompted to penalty the player of the white ball falling bag.

In the snooker game, there are various situations in which the white ball needs to be reset, for example, if a player hits the white ball or any other ball into a side pocket, the corresponding ball must be put back in place and the ball must be kicked off with the white ball; as another example, if a player makes a foul while playing the safe ball (e.g., does not hit the target ball or hits the target ball into a pocket), the opponent may choose to leave the white ball in place and restart his own attack.

Fig. 4 is a schematic diagram of a scenario in which a referee resets a white ball presentation in a real environment according to an embodiment of the present application. Fig. 5 is a schematic diagram of a scene of viewing a white ball presentation through a three-dimensional simulated viewing angle of a sphere by an eagle eye system in a real environment according to an embodiment of the present application. When the above-mentioned foul situation occurs or a proper shot cannot be performed in the snooker game, the referee needs to reset the white ball as shown in fig. 4. Normally, the referee can reset the white ball by means of own memory, or the referee can determine the position where the white ball should be reset by means of video playback. For the former way, the memory of the referee is deviated due to long and heavy and tension work, so that the reset position of the white ball is inaccurate. While, as shown in fig. 5, for the second mode, although the position of the white ball can be accurately reset by carefully comparing the playback videos, it takes a long time to repeatedly watch and confirm the video playback, so that the length of the game increases, and the game rhythm of the player and the look and feel of the audience are also affected.

In some embodiments of the present application, optionally, determining the restored white ball position includes: and acquiring the position of the white ball restored by the eagle eye system. As an example, the application records the desktop sphere through a video technology, when the white sphere is required to be reset, firstly, the eagle eye system in the playing field is utilized to acquire and three-dimensionally restore the position of the white sphere on the billiard table, and then the AR glasses augmented reality technology is utilized to display the position where the white sphere should be restored. The referee can intuitively see the position of the virtual white ball overlapped on the AR equipment, so that the referee is assisted to place the white ball to the position before the foul better, the problem that the reset position of the white ball is inaccurate due to memory deviation is avoided, or the trouble of repeated comparison of the positions before and after the white ball falls into the bag is avoided by playing back the video. The application not only quickens the competition rhythm, but also lightens the heavy work pressure of referees.

In some embodiments, optionally, the goal route of the player is predicted, and the prediction result is displayed in the live view in the form of AR. Fig. 6 is a schematic view of an effect scenario in which an embodiment of the present application predicts and presents a player's goal route. As an example, embodiments of the present application may also be utilized to learn and record competitive habits of a player by training with artificial intelligence machine learning. And finally, identifying the player information through AI, and analyzing the current competition situation in real time, such as ball shape of the current desktop, player hit pre-judgment and the like.

Specifically, the embodiment of the application recognizes player information in real time through the AR terminal equipment and the AI algorithm, records the goal condition and the batting route of the player in the playing process, converts the goal condition and the batting route into three-dimensional data, reserves a white ball batting pole method, the goal route and a white ball position route through data screening and extraction, and inputs the three-dimensional data into a computer to train by utilizing artificial intelligence machine learning, thereby learning and recording the competitive habit of the player. Finally, the player information is identified through AI, and the current competition situation, such as the spherical shape of the current desktop, the player hit pre-judgment and the like, is predicted.

In some embodiments of the present application, optionally, the prediction result is calculated and output by a machine learning model, and a learning sample of the machine learning model is a billiard game video of the player in the past. That is, according to the past competitive habits of the current player, the AR technique is used to automatically assist in predicting a possible goal route of the current player and predicting the player's position, and simultaneously, the prediction result is displayed in an AR form on a live broadcast screen, so that the audience can watch the prediction result in real time, thereby improving the participation of the audience and bringing better viewing experience.

The AR terminal equipment provided by the embodiment of the application has the function of identifying the field match in real time in the billiard match, can identify the field match condition in real time, utilizes the computer vision to track the ball position condition of balls with different colors and the billiard bag falling condition for automatic scoring, can prompt a referee to accurately read the score, is used for judging the match win-or-lose condition and the like, and further lightens the working pressure of the referee.

The operation of the AR-based billiard game referee method according to the embodiment of the present application will be described in detail with reference to specific examples.

Firstly, the referee wears AR glasses (or other terminal equipment with AR function), and the AR glasses position the billiard table by comprehensively utilizing VPS and VSLAM technologies so as to determine the position of the billiard table and the positions of all billiards on the billiard table. For example, the snooker game uses a rectangular table top with 6 pockets and 21 balls, including 15 red balls, and 6 different colored balls, including yellow, green, brown, blue, powder and black balls. In the competition process, the AR glasses can position and track billiards in real time and identify the positions of the billiards with different colors.

In the competition, two players strike the red ball and the colored ball with the white ball in turn, when one billiard ball is hit into the bag, the AR glasses can track and identify the balls falling into the bag, then the scoring condition is automatically recorded, the scoring condition is enhanced and displayed on the AR glasses, and a referee can directly read the scoring. AR glasses may also penalty based on the field billiard ball placement in combination with rules, e.g., AR glasses may determine if they should be handed by an opponent based on the color and location of the ball.

In a game, when a referee is required to reset a white ball, for example, due to the white ball falling into a pocket. The AR glasses can directly display the position where the white ball is to be restored on the billiard table in an AR mode, and can also be displayed in a live broadcast picture. That is, the referee can directly see the virtual image of the place where the white ball needs to be reset through the AR glasses, and can conveniently reset the white ball according to the prompt. In addition, the judge can be informed timely through the AR glasses, and the behavior of the race foul is judged. At the end of the game, the AR glasses may automatically calculate and display the winner's score.

By utilizing the AR glasses and the method for judging the billiard game based on the AR, disclosed by the embodiment of the application, automatic game scoring can be realized, the burden of a judge is reduced, the possibility of human errors is reduced, the fairness and accuracy of the game are improved, the reset position of the white ball can be displayed in a visual way, and the reset operation of the judge is assisted.

Corresponding to the method embodiment provided by the application, the application also provides an AR-based billiard game referee device 100. Fig. 7 is a block diagram of the AR-based billiard game referee device according to the embodiment of the present application, which includes a first determination module 110, a second determination module 120, a scoring processing module 130, a reset processing module 140, and a display processing module 150, as shown in fig. 7.

The first determining module 110 is configured to determine position information of a billiard table and position information of a plurality of billiards on the billiard table.

A second determining module 120 for determining identity information of players participating in the billiard game.

The scoring processing module 130 is configured to track and acquire ball condition information of each billiard ball on the billiard table during a billiard match, so as to score corresponding players according to a scoring rule of the billiard match; wherein, the ball condition information comprises the color, the ball position and the information of whether the billiard ball falls into the bag.

And the reset processing module 140 is used for determining the position of the restored white ball when the white ball needs to be reset, and displaying the position of the restored white ball on a billiard table in a screen of the terminal equipment in an AR form and/or in a live broadcast picture.

And a display processing module 150, configured to display the score of the player on the screen of the terminal device in AR form, so that the referee can read the score during the competition and/or determine the competition result according to the score after the competition is finished.

Those skilled in the art will appreciate that the embodiments described herein are presently preferred embodiments, and that the acts, steps, modules, or units, etc. that are involved are not necessarily required by embodiments of the application. In the foregoing embodiments, the descriptions of the embodiments of the present application are emphasized, and in part, not described in detail in one embodiment, reference may be made to related descriptions of other embodiments.

Fig. 8 is a schematic structural diagram of an electronic device 60 according to an embodiment of the present application, where the electronic device 60 includes a processor 61, a memory 62, and a communication bus for connecting the processor 61 and the memory 62, where a computer program that can be run on the processor 61 is stored in the memory 62, and when the processor 61 runs the computer program, the steps in the method according to the embodiments of the present application are executed or called implemented. The electronic device 60 also includes a communication interface for receiving and transmitting data. The electronic device 60 may be a server in the embodiment of the present application, and the electronic device 60 may also be a cloud server. The electronic device 60 may also be a terminal device or an AR device in an embodiment of the present application. Electronic devices may also be referred to as computing devices where appropriate.

In some embodiments, the processor 61 may be a central processor (Central Processing Unit, CPU), a graphics processor (graphics processing unit, GPU), an application processor (application processor, AP), a modem processor, an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, a neural-network processor (neural-network processing unit, NPU), or the like; the processor 61 may also be other general purpose processors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general purpose processor may be a microprocessor, but in the alternative, it may be any conventional processor or the like. The NPU can rapidly process input information and can continuously perform self-learning by referring to the biological neural network structure. Applications such as intelligent recognition, image recognition, face recognition, semantic recognition, voice recognition, text understanding, etc. can be implemented by the NPU electronic device 60.

In some embodiments, the memory 62 may be an internal storage unit of the electronic device 60, such as a hard disk or memory of the electronic device 60; the memory 62 may also be an external storage device of the electronic device 60, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, which are provided on the electronic device 60. Memory 62 may also include both internal storage units and external storage devices of electronic device 60. The memory 62 may be used to store an operating system, application programs, boot loader (BootLoader), data, and other programs, such as program code for a computer program. Memory 62 includes, but is not limited to, random access memory (random access memory, RAM), read-only memory (ROM), erasable programmable read-only memory (erasable programmable read only memory, EPROM), or portable read-only memory (compact disc read-only memory, CD-ROM). Memory 62 is used to store program codes executed by electronic device 60 and transmitted data. The memory 62 may also be used to temporarily store data that has been output or is to be output.

It will be appreciated by those skilled in the art that fig. 8 is merely an example of the electronic device 60 and is not intended to limit the electronic device 60, and that the electronic device 60 may include more or fewer components than shown, or may combine certain components, or may include different components, such as may also include input-output devices, network access devices, and the like.

Fig. 9 is a schematic software structure of a terminal device according to an embodiment of the present application. Taking a mobile phone operating system as an Android system as an example, in some embodiments, the Android system is divided into four layers, which are respectively: an application layer, an application framework layer (FWK), a system layer, and a hardware abstraction layer, the layers communicating via a software interface.

First, the application layer may include a plurality of application packages, which may be various application apps such as call, camera, video, navigation, weather, instant messaging, education, etc., or may be application apps based on AR technology.

Second, the application framework layer FWK provides an application programming interface (application programming interface, API) and programming framework for application programs of the application layer. The application framework layer may include some predefined functions, such as functions for receiving events sent by the application framework layer.

The application framework layer may include a window manager, a resource manager, a notification manager, and the like.

Wherein the window manager is used for managing window programs. The window manager can acquire the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like. The content provider is used to store and retrieve data and make such data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebooks, etc.

Among other things, the resource manager provides various resources to the application, such as localization strings, icons, pictures, layout files, video files, and so forth.

The notification manager enables the application program to display notification information in a status bar, can be used for conveying notification type information, and can automatically disappear after a short stay without user interaction. Such as notification manager is used to inform that the download is complete, message alerts, etc. The notification manager may also be a notification in the form of a chart or scroll bar text that appears on the system top status bar, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in a status bar, a prompt tone is emitted, the electronic device vibrates, and an indicator light blinks, etc.

In addition, the application framework layer may also include a view system that includes visual controls, such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views, for example, a text display view and a picture display view may be included on the display interface of the sms notification icon.

Third, the system layer may include a plurality of functional modules, such as a sensor service module, a physical state recognition module, a three-dimensional graphics processing library (e.g., openGLES), and so forth.

The sensor service module is used for monitoring sensor data uploaded by various sensors of the hardware layer and determining the physical state of the mobile phone; the physical state recognition module is used for analyzing and recognizing gestures, faces and the like of a user; the three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.

In addition, the system layer may also include a surface manager and a media library. The surface manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications. Media libraries support a variety of commonly used audio, video format playback and recording, still image files, and the like.

Finally, a hardware abstraction layer is a layer between hardware and software. The hardware abstraction layer may include display drivers, camera drivers, sensor drivers, etc. for driving the relevant hardware of the hardware layer, such as the display screen, camera, sensor, etc.

The embodiments of the present application also provide a computer-readable storage medium storing a computer program or instructions which, when executed, implement the steps in the method designed in the above embodiments.

Embodiments of the present application also provide a computer program product comprising a computer program or instructions which, when executed, implement the steps in the method devised in the embodiments described above. The computer program product may be, for example, a software installation package.

Those skilled in the art will appreciate that the methods, steps, or functions of related modules/units described in the embodiments of the present application may be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product, or in the form of computer program instructions executed by a processor. Wherein the computer program product comprises at least one computer program instruction, which may be comprised of corresponding software modules, which may be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, a removable disk, a compact disk read-only memory (CD-ROM), or any other form of storage medium known in the art. The computer program instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer program instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center, by wire or wirelessly. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), optical medium, or semiconductor medium (e.g., SSD), etc.

With respect to each of the apparatuses/products described in the above embodiments, the modules/units included therein may be software modules/units, or may be hardware modules/units, or may be partly software modules/units, or partly hardware modules/units. For example, for an application or a device/product integrated on a chip, each module/unit included in the device/product may be implemented in hardware such as a circuit, or at least some modules/units may be implemented in software programs, and run on a processor integrated inside the chip, where the remaining modules/units are implemented in hardware such as a circuit. For another example, for an application or a device/product integrated in a terminal, each module/unit included in the device/product may be implemented in hardware such as a circuit, or at least some modules/units may be implemented in software program, and run on a processor integrated in the terminal, where the rest of modules/units may be implemented in hardware such as a circuit.

In the foregoing, only the specific embodiments of the present application are described, and it will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, modules and units described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein. It should be understood that the scope of the present application is not limited thereto, and any equivalent modifications or substitutions can be easily made by those skilled in the art within the technical scope of the present application, and they should be included in the scope of the present application.

Claims (11)

1. An AR-based billiard game referee method, comprising:

determining the position information of a billiard table and the position information of a plurality of billiards on the billiard table;

determining identity information of players participating in a billiard game;

in the billiard competition process, the ball condition information of each billiard ball on a billiard table is obtained through tracking, so that scoring is carried out on the corresponding player according to the scoring rule of the billiard competition; wherein, the ball condition information comprises the color, the ball position and the information of whether the billiards fall into the bags or not;

when the white ball needs to be reset, determining the position of the restored white ball, and displaying the position of the restored white ball on a billiard table in a terminal equipment screen and/or in a live broadcast picture in an AR mode; and

the scores of the players are displayed on the screen of the terminal device in the form of AR for the referee to read the scores during the competition and/or to judge the competition result according to the scores after the competition is finished.

2. The method as recited in claim 1, further comprising:

if the billiards falling into the bags are white, judging information is displayed on a screen of the terminal equipment according to the competition rules to be foul.

3. The method of claim 1, wherein determining the restored white ball position comprises: and acquiring the position of the white ball restored by the eagle eye system.

4. The method as recited in claim 1, further comprising:

predicting the goal route of the player, and displaying the prediction result in the live broadcast picture in an AR form.

5. The method as recited in claim 4, further comprising:

predicting the walking mode of the player, and displaying the prediction result in an AR form in a live broadcast picture.

6. The method according to claim 5, wherein the prediction result is calculated and outputted by a machine learning model, and a learning sample of the machine learning model is a billiard game video of the player in the past.

7. An AR-based billiard game referee device, comprising:

the first determining module is used for determining the position information of the billiard table and the position information of a plurality of billiards on the billiard table;

a second determining module for determining identity information of players participating in the billiard game;

the scoring processing module is used for tracking and acquiring the ball condition information of each billiard ball on the billiard ball table in the billiard ball competition process so as to score the corresponding player according to the scoring rule of the billiard ball competition; wherein, the ball condition information comprises the color, the ball position and the information of whether the billiards fall into the bags or not;

the reset processing module is used for determining the position of the restored white ball when the white ball needs to be reset, and displaying the position of the restored white ball on a billiard table in a screen of the terminal equipment in an AR form and/or in a live broadcast picture; and

and the display processing module is used for displaying the scores of the players on a screen of the terminal equipment in an AR form so as to enable judges to read the scores in the competition process and/or judge competition results according to the scores after the competition is finished.

8. An electronic device, comprising: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the method of any of claims 1-6.

9. A computer readable storage medium, characterized in that it has stored thereon computer program instructions which, when executed by a processor, implement the method according to any of claims 1-6.

10. A computer program product comprising computer program instructions which, when executed by a processor, implement the method of any of claims 1-6.

11. An AR glasses, comprising a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, causes the AR glasses to implement the method of any one of claims 1-6.