CN113453032B - Gesture interaction method, device, system, server and storage medium - Google Patents
Gesture interaction method, device, system, server and storage medium Download PDFInfo
- Publication number
- CN113453032B CN113453032B CN202110720064.9A CN202110720064A CN113453032B CN 113453032 B CN113453032 B CN 113453032B CN 202110720064 A CN202110720064 A CN 202110720064A CN 113453032 B CN113453032 B CN 113453032B
- Authority
- CN
- China
- Prior art keywords
- gesture
- game
- anchor
- image
- gesture information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/441—Acquiring end-user identification, e.g. using personal code sent by the remote control or by inserting a card
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4781—Games
Abstract
The invention relates to the technical field of computers, and provides a gesture interaction method, device, system, server and storage medium. The method comprises the steps that gesture information sent by a main broadcasting end participating in a preset gesture game of the current round, namely a prepared main broadcasting end, is received, when the gesture information sent by all the prepared main broadcasting ends is received within preset time, a game result of each prepared main broadcasting end is obtained according to each gesture information and a game rule of the preset gesture game, and a live video generated according to all the game results is sent to a spectator end having a binding relationship with any one prepared main broadcasting end. The gesture interaction of multiple people is realized, the display form of man-machine interaction is enriched, and the interest and the entertainment of live broadcast are improved.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a gesture interaction method, a gesture interaction device, a gesture interaction system, a gesture interaction server and a storage medium.
Background
With the development of image recognition and gesture recognition technologies, it is possible for machine equipment to recognize gesture operations. In the prior art, the human-computer interaction based on gesture recognition is usually only single-person interaction, and the problem of single display mode exists in some entertainment scenes such as live scenes.
Disclosure of Invention
In view of the above, the present invention provides a gesture interaction method, device, system, server and storage medium.
In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solutions:
in a first aspect, the present invention provides a gesture interaction method, including:
receiving gesture information sent by a prepared anchor terminal; the preparation anchor terminal is any anchor terminal participating in the preset gesture game of the current round;
when a set number of gesture information is received within a preset time, obtaining a game result of each prepared anchor terminal according to each gesture information and a game rule of the preset gesture game; the set number is the total number of anchor terminals participating in the current round of preset gesture games;
and sending the live video generated according to all game results to a spectator end having a binding relationship with any one of the prepared anchor ends.
In an alternative embodiment, the gesture information includes an initial gesture image and an affine matrix;
the step of obtaining the game result of each prepared anchor terminal according to each piece of gesture information and the game rule of the preset gesture game comprises the following steps:
acquiring any one piece of target gesture information from all pieces of gesture information;
obtaining a target gesture image according to the initial gesture image and the affine matrix in the target gesture information;
obtaining a game result of the preparation anchor end corresponding to the target gesture information according to the target gesture image and the game rule;
and traversing each piece of gesture information to obtain a game result of the preparation anchor corresponding to each piece of gesture information.
In an optional implementation manner, the step of obtaining the target gesture image according to the initial gesture image and the affine matrix in the target gesture information includes:
performing affine transformation on the initial gesture image in the target gesture information according to the affine matrix in the target gesture information to obtain a gesture image to be recognized; displaying the gestures in the gesture image to be recognized according to a preset angle;
and carrying out image segmentation on the gesture image to be recognized to obtain the target gesture image.
In an optional implementation manner, the step of obtaining, according to the target gesture image and the game rule, a game result of the preparation anchor corresponding to the target gesture information includes:
obtaining a gesture template matched with the target gesture image according to the target gesture image and a gesture template library of the preset gesture game;
and obtaining a game result of the preparation anchor corresponding to the target gesture information according to the game rule and the gesture template.
In an optional implementation manner, the step of sending the live video generated according to all game results to the spectator end having a binding relationship with any one of the prepared anchor ends includes:
generating a live broadcast video according to all game results; the live video comprises a live picture and a game special effect of each prepared anchor terminal;
and sending the live video to a viewer end having a binding relationship with any one of the prepared anchor ends.
In an optional embodiment, after the step of receiving gesture information sent by the preparation anchor, the method further comprises:
when a set number of gesture information is not received within the preset time, obtaining a game matching failure mark;
and sending the feedback live video generated according to the game matching failure mark to any audience end having a binding relationship with the prepared anchor end.
In a second aspect, the present invention provides a gesture interaction device, comprising:
the receiving module is used for receiving gesture information sent by the preparation anchor terminal; the preparation anchor terminal is any anchor terminal participating in the preset gesture game of the current round;
the processing module is used for obtaining a game result of each prepared anchor terminal according to each piece of gesture information and a game rule of a preset gesture game when a set number of pieces of gesture information are received within a preset time; the set number is the total number of anchor terminals participating in the current round of preset gesture games;
and the sending module is used for sending the live video generated according to all game results to a spectator end having a binding relationship with any one of the prepared anchor ends.
In a third aspect, the present invention provides a system comprising a server, a broadcaster end, and a viewer end; the server is respectively in communication connection with the anchor terminal and the audience terminal;
the server is used for executing the gesture interaction method in any one of the preceding embodiments;
the anchor terminal is used for sending the gesture information to the server;
and the audience terminal is used for receiving the live video sent by the server.
In a fourth aspect, the present invention provides a server comprising a processor and a memory, wherein the memory stores a computer program, and the processor implements the method of any one of the preceding embodiments when executing the computer program.
In a fifth aspect, the present invention provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of the preceding embodiments.
According to the gesture interaction method, the gesture interaction device, the gesture interaction system, the gesture interaction server and the storage medium, gesture information sent by a main player is prepared by receiving the main player participating in the preset gesture game in the current round; then when gesture information sent by all prepared anchor terminals is received within preset time, obtaining a game result of each prepared anchor terminal according to each gesture information and a game rule of a preset gesture game; and finally, sending the live video generated according to all game results to a spectator end having a binding relationship with any one of the prepared anchor ends. Therefore, gesture interaction of multiple people is realized, the display form of man-machine interaction is enriched, and the interest and entertainment of live broadcast are improved.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 illustrates a system diagram provided by an embodiment of the invention;
FIG. 2 is a block diagram of a server provided by an embodiment of the invention;
FIG. 3 is a flow chart illustrating a gesture interaction method according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart illustrating a gesture interaction method according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating an example of a gesture interaction method provided by an embodiment of the invention;
FIG. 6 is a diagram illustrating another example of a gesture interaction method according to an embodiment of the present invention;
FIG. 7 is a schematic flowchart illustrating a gesture interaction method according to an embodiment of the present invention;
FIG. 8 is a schematic flow chart illustrating a gesture interaction method according to an embodiment of the present invention;
FIG. 9 is a diagram illustrating another example of a gesture interaction method provided by an embodiment of the invention;
FIG. 10 is a functional block diagram of a gesture interaction apparatus according to an embodiment of the present invention.
Icon: 110-a bus; 120-a processor; 130-a memory; 150-a communication module; 300-a gesture interaction device; 310-a receiving module; 330-a processing module; 350-sending module.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
It is noted that relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Fig. 1 is a schematic diagram of a system according to an embodiment of the present invention. The system comprises a server, a main broadcasting end and a spectator end. The server is respectively in communication connection with the anchor terminal and the audience terminal, and data interaction can be achieved.
The anchor terminal is a mobile terminal of an anchor user, and the viewer terminal is a mobile terminal of a viewer user, which are all installed with an application program for supporting live broadcasting. The mobile terminal may be a smartphone, a personal computer, a tablet computer, or the like. The server is used for providing services for the anchor terminal and the audience terminal.
Fig. 2 is a schematic block diagram of a server according to an embodiment of the present invention. The server includes a bus 110, a processor 120, a memory 130, and a communication module 150.
The processor 120 may receive commands from the above-described other elements (e.g., the memory 130, the communication module 150, etc.) through the bus 110, may interpret the received commands, and may perform calculations or data processing according to the interpreted commands.
The processor 120 may be an integrated circuit chip having signal processing capabilities. The Processor 120 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components.
The memory 130 may store commands or data received from the processor 120 or other elements (e.g., the communication module 150, etc.) or commands or data generated by the processor 120 or other elements.
The Memory 130 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Read Only Memory (EPROM), an electrically Erasable Read Only Memory (EEPROM), and the like.
The communication module 150 may be used for communicating signaling or data with other node devices.
It will be appreciated that the architecture shown in fig. 2 is merely a schematic illustration of a server, and that a server may include more or fewer components than shown in fig. 2, or have a different configuration than that shown in fig. 2. The components shown in fig. 2 may be implemented in hardware, software, or a combination thereof.
The gesture interaction method provided by the embodiment of the invention will be described based on the above system architecture diagram.
In the live broadcast process, in order to strengthen the interest of live broadcast, the live broadcast room is provided with an interactive game based on gesture recognition, namely, a plurality of anchor users carry out gesture games through the anchor end, such as guessing boxing games like stone scissors cloth, and audience users watch live broadcast contents including the gesture games through the audience end so as to improve the entertainment of live broadcast.
Referring to fig. 3, fig. 3 is a schematic flowchart illustrating a gesture interaction method according to an embodiment of the invention.
Step S202, receiving gesture information sent by a preparation anchor terminal;
the prepared anchor terminal represents any anchor terminal participating in the current round of preset gesture games. The preset gesture game can be a piece of stone scissors cloth or a finger guessing game compared with numbers and the like.
Optionally, the server matches a plurality of anchor terminals participating in the same preset gesture game. When the anchor user starts the current round of preset gesture games at the anchor end, the anchor end collects gesture information of the anchor user and sends the gesture information to the server, and the server receives the gesture information sent by the anchor end.
Step S204, when a set number of gesture information is received within a preset time, obtaining a game result of each prepared anchor terminal according to each gesture information and a game rule of a preset gesture game;
the set number represents the total number of the anchor terminals participating in the current round of the preset gesture game.
It can be understood that, because the requirement on timeliness in the live broadcast field is high, a time length can be preset, and if the time length is within a preset time, the server receives gesture information sent by all anchor terminals participating in the preset gesture game in the current round, the game matching can be considered to be successful.
Optionally, the preset time may be set to 1 minute, for example, within 1 minute after the server receives the gesture information sent by the first anchor terminal, the gesture information sent by all anchor terminals participating in the preset gesture game in the current round is received, and the game matching is considered to be successful.
Optionally, after it is confirmed that the game matching is successful, the game result of each preliminary anchor terminal may be obtained according to each gesture information and the preset game rule of the gesture game.
And step S206, sending the live video generated according to all game results to the audience end having the binding relation with any one of the prepared anchor ends.
Optionally, the server generates a live video according to the obtained all game results and live frames of each prepared anchor end received in real time, and then sends the live video to the audience ends having a binding relationship with any one of the prepared anchor ends, so that the audience users who are watching the live video can obtain the results of the current round of game.
Based on the steps, firstly, the server receives the anchor terminals participating in the preset gesture game of the current round, namely, gesture information sent by the anchor terminals is prepared; then when gesture information sent by all prepared anchor terminals is received within preset time, obtaining a game result of each prepared anchor terminal according to each gesture information and a game rule of a preset gesture game; and finally, sending the live video generated according to all game results to a spectator end having a binding relationship with any one of the prepared anchor ends. Therefore, gesture interaction of multiple people is realized, the display form of man-machine interaction is enriched, and the interest and entertainment of watching live broadcast by audience users are improved.
It can be understood that, due to the problem of network, there may be a case that the game matching fails, and further, the embodiment of the present invention provides a possible implementation manner, please refer to fig. 4, after step S202, the following steps may be further included:
step S214, when a set number of gesture information is not received within a preset time, a game matching failure mark is obtained;
it can be understood that, due to the influence of factors such as a network, the server may not receive the gesture information within the preset time, and all the gesture information sent by the preparation anchor terminals participating in the current round of the preset gesture game considers that the game matching is failed, and obtains a game matching failure flag.
Optionally, the server stores the received initial gesture image in an array of a storage area of the server, and stores the affine matrix in a preprocessing area of the server; and when the server does not receive the initial gesture image gesture information sent by all the anchor terminals participating in the current round of preset games within the preset time, namely the number of the arrays is not equal to the total number of the anchor terminals participating in the current round of preset games, obtaining a game matching failure sign.
Step S216, sending the feedback live video generated according to the game matching failure mark to a spectator end having a binding relationship with any prepared anchor end;
after confirming that the game matching fails, generating a feedback live broadcast video according to the game matching failure mark and live broadcast pictures of each prepared anchor terminal received in real time; and then the feedback live video is sent to any audience terminal with a binding relation to prepare, so that the audience users watching the live video can know that the matching of the current game is not successful.
Optionally, a prompt message may be sent to the anchor terminal according to the game matching failure flag, so as to prompt the anchor user that the current game matching fails, and the next game may be restarted.
In order to better understand the present invention, the gesture interaction method provided by the embodiment of the present invention will be described below by taking two examples, namely, the preset gesture game is stone scissors cloth, and the anchor terminals participating in the preset gesture game in the current round are two examples.
For convenience of description, two anchor terminals participating in the current round of preset gesture game are referred to as an anchor terminal a and an anchor terminal B, where the anchor terminal a represents a mobile terminal of an anchor user a, and the anchor terminal B represents a mobile terminal of an anchor user B. The audience C and the anchor A have a binding relationship, and the audience C represents a mobile terminal of an audience user C.
Fig. 5 is a diagram illustrating an embodiment of the present invention. The illustration shows a live video presentation interface of spectator C, which includes a game progress area and two anchor information areas.
One anchor information area is used for displaying a live broadcast picture of an anchor terminal A, namely a live broadcast picture A, and the name and the head portrait information of an anchor user A.
The other anchor information area is used for displaying a live broadcast picture of an anchor terminal B, namely a live broadcast picture B, and the name and the head portrait information of an anchor user B.
It should be noted that, in the embodiment of the present invention, two anchor terminals are taken as an example, so that there are two anchor information areas. It can be understood that the number of the anchor information areas is the same as the number of the anchor terminals. A plurality of anchor information areas may be set according to actual requirements, and the embodiment of the present invention is not limited.
The game progress area is used for displaying the progress of the preset gesture game. Such as game play status, game outcome, etc.
Optionally, the anchor information area may further include a gesture box for blocking a gesture of the anchor user in the live view. It can be understood that, because a finger guessing game usually requires participants to simultaneously gesture a stroke, and in the live broadcast process, there may be problems such as network delay, etc., the gesture of the anchor user in the live broadcast screen may be blocked by using a gesture frame, and the gesture of the anchor user is published after the game is finished.
The gesture interaction method provided by the embodiment of the invention will be described based on the above exemplary diagrams.
First, the step of sending gesture information by the anchor terminal will be described. It can be understood that the anchor terminal has a module for collecting images, such as a camera, and collects live broadcast pictures in real time in the live broadcast process and displays the live broadcast pictures at the anchor terminal.
Note that the live view displayed on the main broadcast side is different from the live view displayed by the viewer, and for the sake of distinction, the live view displayed on the main broadcast side is hereinafter referred to as a main broadcast side live view.
When the preset gesture game of the current round starts, the anchor side detects the hands of the anchor user in the anchor side live broadcast picture through a hand detection algorithm in the application program, a detection frame is displayed in the anchor side live broadcast picture, the detection frame comprises the hands of the anchor user, and the position of the hands in the anchor side live broadcast picture can be determined through the detection frame.
The direct broadcasting picture of the anchor side also comprises prompt information and a preset frame, and the preset frame is at a preset position in the direct broadcasting picture of the anchor side.
The anchor user can move the hand according to the prompt message to enable the detection frame to be overlapped with the preset frame, when the area overlapping ratio of the detection frame and the preset frame reaches a preset value of 85%, the anchor user gestures, and then the anchor user obtains an image in the preset frame, wherein the image is an initial gesture image.
It can be understood that the preset frame in the anchor side live broadcast picture corresponds to the gesture frame of the live video display interface of the audience, that is, when the anchor user moves the hand to the position of the preset frame of the anchor side live broadcast picture, in the live broadcast picture displayed by the audience, the hand of the anchor user is shielded by the gesture frame, so that the gesture of the anchor user cannot be seen, and the influence of network delay on the gesture game can be effectively dealt with.
It can be understood that the initial gesture image includes the detection frame, and the detection frame corresponds to an affine matrix, where the affine matrix is used to represent a placement state of the detection frame in the initial gesture image, and the placement state may be represented by an angle formed by a central axis of the detection frame and a central axis of the initial gesture image.
And after the anchor terminal acquires the initial gesture image and the affine matrix, sending the initial gesture image and the affine matrix serving as gesture information to the server.
For example, the anchor terminal a sends gesture information a to the server, wherein the gesture information a includes an initial gesture image a and an affine matrix a.
And the anchor terminal B sends the gesture information B to the server, wherein the gesture information B comprises an initial gesture image B and an affine matrix B.
Fig. 6 is an exemplary diagram of an initial gesture image provided by an embodiment of the present invention. Wherein, (a1) represents the initial gesture image a, and the angle between the central axis of the detection box a and the central axis of the initial gesture image a is r 1.
(b1) Representing an initial gesture image B, the angle between the central axis of the detection box B and the central axis of the initial gesture image B is r 2.
After the step of introducing the anchor terminal to send the gesture information, the steps of receiving the gesture information and processing the gesture information by the server are introduced. Please refer to fig. 7, which is a flowchart illustrating a gesture interaction method according to an embodiment of the present invention.
Step S204-1, acquiring any one piece of target gesture information from all pieces of gesture information;
step S204-3, obtaining a target gesture image according to the initial gesture image and the affine matrix in the target gesture information;
optionally, the server stores the received initial gesture image in an array of a storage area of the server, and stores the affine matrix in a preprocessing area; when the server receives initial gesture images sent by all anchor terminals participating in the current round of preset games within preset time, namely the number of the arrays is equal to the total number of the anchor terminals participating in the current round of preset games, all the initial gesture images are transmitted to a preprocessing area; in the pre-processing zone, each initial gesture image is processed.
Optionally, each piece of gesture information is respectively used as target gesture information, step S204-1 and step S204-3 are executed, and then a target gesture image corresponding to the initial gesture image in each piece of gesture information is obtained.
For step S204-3, the embodiment of the present invention provides a possible implementation manner.
Optionally, affine transformation may be performed on the initial gesture image according to the affine matrix to obtain a gesture image to be recognized.
And then, recognizing and segmenting the gesture in the gesture image to be recognized through a GPU operator in the server and a prestored pixel segmentation recognizer to obtain a segmented gesture image, and optionally, carrying out binarization on the segmented gesture image to obtain a target gesture image.
As shown in (a2) and (a3) in fig. 6, the gesture image a to be recognized and the target gesture image a are obtained according to the initial gesture image a and the affine matrix a, and the gesture in the gesture image a to be recognized is displayed according to a preset angle.
As shown in (B2) and (B3) of fig. 6, the gesture image B to be recognized and the target gesture image B are obtained according to the initial gesture image B and the affine matrix B, and the gesture in the gesture image B to be recognized is displayed according to the preset angle.
It can be understood that, according to the affine matrix, the affine transformation is performed on the initial gesture image in order to correct the gesture in the initial gesture image, that is, the initial gesture image is converted into an image which is more convenient for the pixel segmentation recognizer to process.
The placing state of the existing gestures of the frame body is detected, the placing state of the detection frame is adjusted according to the affine matrix, the gestures are placed, and the accuracy of image segmentation in the subsequent steps can be improved.
Continuing to refer to fig. 7, in step S204-5, a game result of the preliminary anchor corresponding to the target gesture information is obtained according to the target gesture image and the game rule;
step S204-7, traversing each gesture information to obtain a game result of the preparation anchor corresponding to each gesture information;
optionally, a gesture template library of the preset gesture game may be pre-stored in the server, then each target gesture image is matched with the gesture template library to obtain a gesture template matched with the target gesture image, according to the gesture template, a recognition result of the target gesture image may be obtained, and all the obtained recognition results may be stored in one array.
For example, the recognition result of the target gesture image a is scissors, and the recognition result of the target gesture image B is a stone.
Optionally, a game result of the standby anchor corresponding to each piece of gesture information is obtained according to the game rule and the recognition result represented by the gesture template.
For example, the recognition result of the target gesture image a is scissors, the recognition result of the target gesture image B is a stone, and according to the game rules of stone scissors, the game result of the anchor terminal a is cherished and the game result of the anchor terminal B is won.
Based on the design, the gesture information sent by the anchor terminal is received, and the initial gesture image in the gesture information is processed according to the affine matrix in the gesture information to obtain a gesture image to be recognized so as to convert the initial gesture image into an image beneficial to recognition; performing image segmentation on the gesture image to be recognized to obtain a target gesture image; and then matching the target gesture image with a gesture module library of a preset gesture game, and combining game rules to obtain a game result of each anchor terminal. The gesture is recognized in the live broadcast process, the gesture recognition accuracy is improved, multi-user gesture interaction is realized, and the interestingness of watching live broadcast by audience users is improved.
With respect to the step S206, the embodiment of the present invention provides a possible implementation manner. Please refer to fig. 8. Wherein, the step S206 may include the steps of:
step S206-1, generating a live video according to all game results;
step S206-3, sending the live video to a viewer end having a binding relationship with any one of the prepared anchor ends;
optionally, according to all game results, determining a game special effect corresponding to the game result of each anchor terminal, rendering and encoding the game special effect and a video stream of each anchor terminal received in real time, wherein the video stream comprises a live broadcast picture, so as to obtain a live broadcast video, and sending the live broadcast video to a spectator terminal having a binding relationship with any one of the prepared anchor terminals.
For example, if the result of the game of the anchor terminal a is unfortunately, the corresponding game effect is effect 1; the result of the game of the anchor terminal B is winning, and the corresponding special effect of the game is special effect 2. And then rendering and coding the live broadcast picture of the anchor terminal A and the live broadcast picture of the anchor terminal B which are received in real time, as well as the special effects 1 and 2 to obtain a live broadcast video, and then sending the live broadcast video to a viewer terminal C.
As shown in fig. 9, the example is a live video display interface of the spectator end C after the current round of the preset gesture game is finished. The game progress area of the interface displays a game result, namely the game result of the anchor user A is cherished or failed, and the game result of the anchor user B is won; and the gestures of the game round, the anchor user A and the anchor user B are compared through gesture icons.
And displaying a game special effect corresponding to the game result of the anchor end A, namely special effect 1, in the anchor information area corresponding to the anchor end A. And displaying a game special effect corresponding to the game result of the anchor terminal B, namely special effect 2, in an anchor information area corresponding to the anchor terminal B.
In order to perform the corresponding steps in the above embodiments and various possible manners, an implementation manner of the gesture interaction apparatus 300 is given below. Referring to fig. 10, fig. 10 is a functional block diagram of a gesture interaction apparatus 300 according to an embodiment of the present invention. It should be noted that the basic principle and the generated technical effects of the gesture interaction apparatus 300 provided in the present embodiment are the same as those of the above embodiments, and for brevity, reference may be made to the corresponding contents in the above embodiments for the parts not mentioned in the present embodiment. The gesture interaction apparatus 300 includes:
a receiving module 310, configured to receive gesture information sent by a preparation anchor; the preparation anchor is any anchor participating in the preset gesture game of the current round;
the processing module 330 is configured to, when a set number of gesture information is received within a preset time, obtain a game result of each prepared anchor according to each gesture information and a game rule of a preset gesture game; setting the number as the total number of the anchor terminals participating in the current round of preset gesture games;
and a sending module 350, configured to send the live video generated according to all game results to a viewer with a binding relationship with any one of the prepared anchor terminals.
Optionally, the processing module 330 is further configured to obtain any one target gesture information from all the gesture information;
obtaining a target gesture image according to the initial gesture image and the affine matrix in the target gesture information;
obtaining a game result of a preparation anchor end corresponding to the target gesture information according to the target gesture image and the game rule;
and traversing each gesture information to obtain a game result of the prepared anchor end corresponding to each gesture information.
Optionally, the processing module 330 is further configured to perform affine transformation on the initial gesture image in the target gesture information according to the affine matrix in the target gesture information to obtain a gesture image to be recognized; displaying the gesture in the gesture image to be recognized according to a preset angle;
and carrying out image segmentation on the gesture image to be recognized to obtain a target gesture image.
Optionally, the processing module 330 is further configured to obtain a gesture template matched with the target gesture image according to the target gesture image and a gesture template library of a preset gesture game;
and obtaining a game result of the prepared anchor end corresponding to the target gesture information according to the game rule and the gesture template.
Optionally, the sending module 350 is further configured to generate a live video according to all game results; the live video comprises live pictures and game special effects of each prepared anchor end;
and sending the live video to a viewer end having a binding relationship with any one of the prepared anchor ends.
Optionally, the sending module 350 is further configured to obtain a game matching failure flag when the set number of gesture information is not received within the preset time;
and sending the feedback live broadcast video generated according to the game matching failure mark to a spectator end having a binding relationship with any one of the prepared anchor ends.
The embodiment of the present invention further provides a server, which includes a processor 120 and a memory 130, where the memory 130 stores a computer program, and when the processor executes the computer program, the gesture interaction method disclosed in the above embodiment is implemented.
The embodiment of the present invention further provides a storage medium, on which a computer program is stored, and when the computer program is executed by the processor 120, the gesture interaction method disclosed in the embodiment of the present invention is implemented.
In summary, the gesture interaction method, device, system, server and storage medium provided by the embodiments of the present invention. The method comprises the steps that an anchor terminal participating in a current round of preset gesture game is received, namely gesture information sent by the anchor terminal is prepared; when gesture information sent by all prepared anchor terminals is received within preset time, obtaining a game result of each prepared anchor terminal according to each gesture information and a game rule of a preset gesture game; and then, according to the live video generated by all game results, sending the live video to a spectator end having a binding relationship with any one of the prepared anchor ends. Therefore, gesture interaction of multiple people is realized, the display form of man-machine interaction is enriched, and the interest and entertainment of live broadcast are improved.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. A gesture interaction method, the method comprising:
receiving gesture information sent by a prepared anchor terminal; the preparation anchor terminal is any anchor terminal participating in the current round of preset gesture games;
when a set number of gesture information is received within a preset time, obtaining a game result of each prepared anchor terminal according to each gesture information and a game rule of the preset gesture game; the set number is the total number of anchor terminals participating in the current round of preset gesture games;
sending the live video generated according to all game results to a spectator end having a binding relationship with any one of the prepared anchor ends;
the gesture information comprises an initial gesture image and an affine matrix; the initial gesture image comprises a detection frame, the detection frame comprises gestures of a cast user stroke, and the affine matrix is used for representing the placing state of the detection frame in the initial gesture image;
the step of obtaining the game result of each prepared anchor terminal according to each piece of gesture information and the game rule of the preset gesture game comprises the following steps:
acquiring any one piece of target gesture information from all pieces of gesture information;
obtaining a target gesture image according to the initial gesture image and the affine matrix in the target gesture information;
obtaining a game result of the preparation anchor terminal corresponding to the target gesture information according to the target gesture image and the game rule;
and traversing each piece of gesture information to obtain a game result of the preparation anchor corresponding to each piece of gesture information.
2. The method according to claim 1, wherein the step of obtaining the target gesture image according to the initial gesture image and the affine matrix in the target gesture information comprises:
performing affine transformation on the initial gesture image in the target gesture information according to the affine matrix in the target gesture information to obtain a gesture image to be recognized; displaying the gestures in the gesture image to be recognized according to a preset angle;
and carrying out image segmentation on the gesture image to be recognized to obtain the target gesture image.
3. The method according to claim 1, wherein the step of obtaining the game result of the preparation anchor corresponding to the target gesture information according to the target gesture image and the game rule comprises:
obtaining a gesture template matched with the target gesture image according to the target gesture image and a gesture template library of the preset gesture game;
and obtaining a game result of the preparation anchor corresponding to the target gesture information according to the game rule and the gesture template.
4. The method of claim 1, wherein the step of sending the live video generated according to the results of all games to the spectator in a binding relationship with any one of the alternate broadcasters comprises:
generating a live video according to all game results; the live video comprises a live picture and a game special effect of each prepared anchor terminal;
and sending the live video to a viewer end having a binding relationship with any one of the prepared anchor ends.
5. The method of claim 1, wherein after the step of receiving gesture information ready for transmission by a host, the method further comprises:
when a set number of gesture information is not received within the preset time, obtaining a game matching failure mark;
and sending the feedback live video generated according to the game matching failure mark to any audience end having a binding relationship with the prepared anchor end.
6. A gesture interaction apparatus, comprising:
the receiving module is used for receiving gesture information sent by the preparation anchor terminal; the preparation anchor terminal is any anchor terminal participating in the current round of preset gesture games;
the processing module is used for obtaining a game result of each prepared anchor terminal according to each piece of gesture information and a game rule of a preset gesture game when a set number of pieces of gesture information are received within a preset time; the set number is the total number of anchor terminals participating in the current round of preset gesture games;
the transmitting module is used for transmitting the live video generated according to all game results to a spectator end having a binding relationship with any one of the prepared anchor ends;
the gesture information comprises an initial gesture image and an affine matrix; the initial gesture image comprises a detection box comprising a gesture of a cast user stroke, and the affine matrix is used for representing a placing state of the detection box in the initial gesture image;
the processing module is further configured to: acquiring any one piece of target gesture information from all pieces of gesture information; obtaining a target gesture image according to the initial gesture image and the affine matrix in the target gesture information; obtaining a game result of the preparation anchor terminal corresponding to the target gesture information according to the target gesture image and the game rule; and traversing each piece of gesture information to obtain a game result of the preparation anchor corresponding to each piece of gesture information.
7. A system comprising a server, a broadcaster side, and a viewer side; the server is respectively in communication connection with the anchor terminal and the audience terminal;
the server is used for executing the gesture interaction method of any one of claims 1 to 5;
the anchor terminal is used for sending the gesture information to the server;
and the audience terminal is used for receiving the live video sent by the server.
8. A server, comprising a processor and a memory, the memory storing a computer program which, when executed by the processor, implements the method of any one of claims 1 to 5.
9. A storage medium, characterized in that the storage medium has stored thereon a computer program which, when executed by a processor, implements the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110720064.9A CN113453032B (en) | 2021-06-28 | 2021-06-28 | Gesture interaction method, device, system, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110720064.9A CN113453032B (en) | 2021-06-28 | 2021-06-28 | Gesture interaction method, device, system, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113453032A CN113453032A (en) | 2021-09-28 |
| CN113453032B true CN113453032B (en) | 2022-09-30 |
Family
ID=77813429
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110720064.9A Active CN113453032B (en) | 2021-06-28 | 2021-06-28 | Gesture interaction method, device, system, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113453032B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106127138A (en) * | 2016-06-21 | 2016-11-16 | 乐视控股（北京）有限公司 | The initial method of a kind of hand-type identification and device and virtual reality terminal |
| CN109165555A (en) * | 2018-07-24 | 2019-01-08 | 广东数相智能科技有限公司 | Man-machine finger-guessing game method, apparatus and storage medium based on image recognition |
| CN109213304A (en) * | 2017-06-29 | 2019-01-15 | 格局商学教育科技（深圳）有限公司 | Gesture interaction method and system for live broadcast teaching |
| CN109453510A (en) * | 2018-09-19 | 2019-03-12 | 董红伟 | Smart phone guesses the number finger-guessing game APP and smart phone is guessed the number finger-guessing game wechat small routine |
| JP6726322B1 (en) * | 2019-01-11 | 2020-07-22 | 株式会社コロプラ | Game program, method, and information processing device |
| JP2021010756A (en) * | 2019-04-26 | 2021-02-04 | 株式会社コロプラ | Program, method, and information terminal device |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7798896B2 (en) * | 2000-09-27 | 2010-09-21 | Milestone Entertainment Llc | Apparatus, systems and methods for implementing enhanced gaming and prizing parameters in an electronic environment |
| KR20110063329A (en) * | 2009-12-02 | 2011-06-10 | 김은구 | Online realtime tournament game providing system and method thereof |
| KR101507099B1 (en) * | 2013-07-15 | 2015-03-30 | 스마트한 주식회사 | Terminal Device for Providing Rock-Paper-Scissors Game and Method thereof |
| KR101725736B1 (en) * | 2016-07-29 | 2017-04-11 | 권형섭 | Method for providing rock-paper-scissors option game and apparatus for the same |
| US10846819B2 (en) * | 2017-04-12 | 2020-11-24 | Southern Methodist University | Method and apparatus to infer structural stresses with visual image and video data |
| US11303923B2 (en) * | 2018-06-15 | 2022-04-12 | Intel Corporation | Affine motion compensation for current picture referencing |
| CN110188754B (en) * | 2019-05-29 | 2021-07-13 | 腾讯科技（深圳）有限公司 | Image segmentation method and device and model training method and device |
| JP6818092B2 (en) * | 2019-06-25 | 2021-01-20 | 株式会社コロプラ | Game programs, game methods, and information terminals |
| CN110598542A (en) * | 2019-08-05 | 2019-12-20 | 广州华多网络科技有限公司 | Game interaction method, system, server and storage device based on expression recognition |
| CN110519612A (en) * | 2019-08-26 | 2019-11-29 | 广州华多网络科技有限公司 | Even wheat interactive approach, live broadcast system, electronic equipment and storage medium |
| CN113038149A (en) * | 2019-12-09 | 2021-06-25 | 上海幻电信息科技有限公司 | Live video interaction method and device and computer equipment |
| CN111866535B (en) * | 2020-07-24 | 2022-09-02 | 北京达佳互联信息技术有限公司 | Live somatosensory item interaction method, device, equipment and storage medium |
-
2021
- 2021-06-28 CN CN202110720064.9A patent/CN113453032B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106127138A (en) * | 2016-06-21 | 2016-11-16 | 乐视控股（北京）有限公司 | The initial method of a kind of hand-type identification and device and virtual reality terminal |
| CN109213304A (en) * | 2017-06-29 | 2019-01-15 | 格局商学教育科技（深圳）有限公司 | Gesture interaction method and system for live broadcast teaching |
| CN109165555A (en) * | 2018-07-24 | 2019-01-08 | 广东数相智能科技有限公司 | Man-machine finger-guessing game method, apparatus and storage medium based on image recognition |
| CN109453510A (en) * | 2018-09-19 | 2019-03-12 | 董红伟 | Smart phone guesses the number finger-guessing game APP and smart phone is guessed the number finger-guessing game wechat small routine |
| JP6726322B1 (en) * | 2019-01-11 | 2020-07-22 | 株式会社コロプラ | Game program, method, and information processing device |
| JP2021010756A (en) * | 2019-04-26 | 2021-02-04 | 株式会社コロプラ | Program, method, and information terminal device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113453032A (en) | 2021-09-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111935491B (en) | Live broadcast special effect processing method and device and server | |
| CN107820132B (en) | Live broadcast interaction method, device and system | |
| CN111246232A (en) | Live broadcast interaction method and device, electronic equipment and storage medium | |
| CN107911736B (en) | Live broadcast interaction method and system | |
| US11257293B2 (en) | Augmented reality method and device fusing image-based target state data and sound-based target state data | |
| CN112543343B (en) | Live broadcast picture processing method and device based on live broadcast with wheat | |
| CN111773667A (en) | Live game interaction method and device, computer readable medium and electronic equipment | |
| CN109299326B (en) | Video recommendation method, device and system, electronic equipment and storage medium | |
| CN111970532B (en) | Video playing method, device and equipment | |
| CN110856008B (en) | Live broadcast interaction method, device and system, electronic equipment and storage medium | |
| CN112929678B (en) | Live broadcast method, live broadcast device, server side and computer readable storage medium | |
| CN111277849A (en) | Image processing method and device, computer equipment and storage medium | |
| CN112929684B (en) | Video superimposed information updating method and device, electronic equipment and storage medium | |
| CN112218108B (en) | Live broadcast rendering method and device, electronic equipment and storage medium | |
| US20170171621A1 (en) | Method and Electronic Device for Information Processing | |
| CN112243137A (en) | Live broadcast interface updating method, device, server and system | |
| CN108985421B (en) | Method for generating and identifying coded information | |
| CN113573090A (en) | Content display method, device and system in game live broadcast and storage medium | |
| CN114697703B (en) | Video data generation method and device, electronic equipment and storage medium | |
| CN107509080B (en) | Method and device for displaying coded image and electronic equipment | |
| CN110809172A (en) | Interactive special effect display method and device and electronic equipment | |
| CN113453032B (en) | Gesture interaction method, device, system, server and storage medium | |
| CN110572686B (en) | Interaction method, live broadcast system, electronic equipment and storage device | |
| US20170171628A1 (en) | Method and electronic device for quickly playing video | |
| CN112153409B (en) | Live broadcast method and device, live broadcast receiving end and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |