CN113542774A - Video synchronization method and device, electronic equipment and storage medium - Google Patents
Video synchronization method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113542774A CN113542774A CN202110625325.9A CN202110625325A CN113542774A CN 113542774 A CN113542774 A CN 113542774A CN 202110625325 A CN202110625325 A CN 202110625325A CN 113542774 A CN113542774 A CN 113542774A
- Authority
- CN
- China
- Prior art keywords
- target
- video
- game
- match
- action
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/242—Synchronization processes, e.g. processing of PCR [Program Clock References]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Image Analysis (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
The embodiment of the application provides a video synchronization method, a video synchronization device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring at least two match videos of the same match, which are acquired from different positions; identifying a target segment corresponding to a target action in each match video, wherein the target segment is composed of images with continuous shooting time, and the target action is a symbolic action for starting a match; determining an image corresponding to a target gesture in the target segment, and taking the image corresponding to the target gesture as a synchronous frame corresponding to each competition video; and synchronizing the at least two game videos according to the synchronization frame. The method comprises the steps of identifying target segments from at least two game videos, determining a synchronization frame in each game video from the target segments, realizing video synchronization based on the synchronization frames, and being free of any auxiliary synchronization equipment, simple and convenient to operate and easy to apply in a large scale.
Description
Technical Field
The present application relates to the field of computer vision technologies, and in particular, to a video synchronization method and apparatus, an electronic device, and a storage medium.
Background
With the rapid development of science and technology, people have more and more demands on three-dimensional models and the related three-dimensional models. In the process of carrying out three-dimensional modeling on a sports game, a plurality of image acquisition devices are used for acquiring game videos at different angles, the image acquisition devices are distributed at different positions of a game field, and the angles of the acquired game videos are different. When three-dimensional modeling is performed using these game videos, it is difficult to determine the same time in at least two game videos to achieve synchronization.
At present, when videos of a plurality of image acquisition devices are synchronized, synchronization equipment sends synchronization signals to all the image acquisition devices depending on specific synchronization equipment, and shutter shooting of the image acquisition devices is controlled through the synchronization signals, so that synchronous shooting of the plurality of image acquisition devices is realized, and further video synchronization is realized. However, the synchronization method needs to depend on synchronization equipment, is complex and tedious to operate, and is difficult to realize large-scale application.
Disclosure of Invention
The embodiment of the application provides a video synchronization method, a video synchronization device, an electronic device and a storage medium, and can effectively solve the problems that video synchronization operation is complex and tedious and large-scale application is difficult to realize.
According to a first aspect of embodiments of the present application, there is provided a video synchronization method, including: acquiring at least two match videos of the same match, which are acquired from different positions; identifying a target segment corresponding to a target action in each match video, wherein the target segment is composed of images with continuous shooting time, and the target action is a symbolic action for starting a match; determining an image corresponding to a target gesture in the target segment, and taking the image corresponding to the target gesture as a synchronous frame corresponding to each competition video; and synchronizing the at least two game videos according to the synchronization frame corresponding to each game video.
According to a second aspect of embodiments of the present application, there is provided a video synchronization apparatus, including: the acquisition module is used for acquiring at least two match videos of the same match, which are acquired from different positions; the identification module is used for identifying a target segment corresponding to a target action in each competition video, wherein the target segment is composed of images with continuous shooting time, and the target action is used as a symbolic action for the beginning of a competition; a synchronous frame determining module, configured to determine an image corresponding to a target pose in the target segment, and use the image corresponding to the target pose as a synchronous frame corresponding to each match video; and the synchronization module is used for synchronizing the at least two match videos according to the synchronization frame corresponding to each match video.
According to a third aspect of embodiments of the present application, there is provided an electronic device comprising one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method as applied to an electronic device, as described above.
According to a fourth aspect of the embodiments of the present application, there is provided a computer-readable storage medium having a program code stored therein, wherein the method described above is performed when the program code runs.
The video synchronization method provided by the embodiment of the application is adopted to obtain at least two match videos of the same match collected from different positions; identifying a target segment corresponding to a target action in each match video, wherein the target segment is composed of images with continuous shooting time, and the target action is a symbolic action for starting a match; determining an image corresponding to a target gesture in the target segment, and taking the image corresponding to the target gesture as a synchronous frame corresponding to each competition video; and synchronizing the at least two game videos according to the synchronization frame. The method comprises the steps of identifying a target segment corresponding to a symbolic action at the beginning of a game from at least two game videos, and then determining a synchronization frame in each game video from the target segment, wherein the actual occurrence time of the synchronization frames is the same no matter how the acquisition modes of the game videos are based on the same target posture corresponding to the symbolic action at the beginning of the game, and the actual acquisition time points of the corresponding synchronization frames are the same.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a schematic application environment diagram of a video synchronization method according to an embodiment of the present application;
fig. 2 is a flowchart of a video synchronization method according to an embodiment of the present application;
fig. 3 is a flowchart of a video synchronization method according to another embodiment of the present application;
fig. 4 is a flowchart of a video synchronization method according to another embodiment of the present application;
FIG. 5 is a functional block diagram of a video synchronization apparatus according to an embodiment of the present application;
fig. 6 is a block diagram of an electronic device for performing a video synchronization method according to an embodiment of the present application.
Detailed Description
With the rapid development of science and technology, people have more and more demands on three-dimensional models and the related three-dimensional models. In the process of carrying out three-dimensional modeling on a sports game, a plurality of image acquisition devices are used for acquiring game videos at different angles, the image acquisition devices are distributed at different positions of a game field, and the angles of the acquired game videos are different. When three-dimensional modeling is performed using these game videos, it is difficult to determine the same time in at least two game videos to achieve synchronization.
At present, when videos of a plurality of image acquisition devices are synchronized, a synchronization device sends a synchronization signal to all the image acquisition devices depending on a specific synchronization device, and a shutter of the image acquisition devices is controlled to shoot through the synchronization signal, so that synchronous shooting of the plurality of image acquisition devices is realized, and further, video synchronization is realized. However, the synchronization method needs to depend on synchronization equipment, is complex and tedious to operate, and is difficult to realize large-scale application.
The inventor finds in research that with the continuous upgrade of image acquisition devices in mobile intelligent equipment and the rapid development of 5G and cloud computing technologies, there are mature hardware conditions for realizing video synchronization. When a plurality of image acquisition devices are used for shooting the same sports competition at different angles, it is not practical to synchronize the mobile intelligent terminal through signals sent by external synchronizing equipment, and no external synchronizing equipment is actually available. In the scene of a sports game, there is a symbolic action at the beginning of the game, for example, the symbolic action at the beginning of a basketball game is a mid-shot. Therefore, the mark actions at the beginning of the game can be identified by analyzing the game videos collected by the plurality of image collecting devices, and the frames where the mark actions are located are the synchronous frames, so that the videos of all the image collecting devices shooting the same sports game are synchronized.
Therefore, the embodiment of the application provides a video synchronization method, which is used for acquiring at least two match videos of the same match, which are acquired from different positions; identifying a target segment corresponding to a target action in each match video, wherein the target segment is composed of images with continuous shooting time, and the target action is a symbolic action for starting a match; determining an image corresponding to a target gesture in the target segment, and taking the image corresponding to the target gesture as a synchronous frame corresponding to each competition video; and synchronizing the at least two game videos according to the synchronization frame. The method comprises the steps of identifying a target segment corresponding to a symbolic action at the beginning of a game from at least two game videos, and then determining a synchronization frame in each game video from the target segment, wherein the actual occurrence time of the synchronization frames is the same no matter how the acquisition modes of the game videos are based on the same target posture corresponding to the symbolic action at the beginning of the game, and the actual acquisition time points of the corresponding synchronization frames are the same.
The scheme in the embodiment of the present application may be implemented by using various computer languages, for example, object-oriented programming language Java and transliterated scripting language JavaScript, Python, and the like.
In order to make the technical solutions and advantages of the embodiments of the present application more apparent, the following further detailed description of the exemplary embodiments of the present application with reference to the accompanying drawings makes it clear that the described embodiments are only a part of the embodiments of the present application, and are not exhaustive of all embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
Referring to fig. 1, an application environment 10 of the video synchronization method provided by the present application is shown, where the application environment 10 includes an electronic device 20, at least two image capturing devices 30, and a playing field 40. The image capturing device 30 is in communication connection with the electronic device 20, the game video captured by the image capturing device 30 is sent to the electronic device 20, and the electronic device 20 can process the game video.
The image capturing device 30 may be a mobile device with an image capturing function, such as a smart phone, a tablet computer, and the like. The image capturing devices 30 are disposed in the playing field 40 for capturing environmental information in the scene 40, wherein the shooting field of view of each image capturing device 30 covers the entire playing field 40.
In some embodiments, an application program for synchronization may be installed in the image capturing device 30, and the application program may transmit the game video captured by the image capturing device 30 to the electronic device 20 through a network. The network may be a 5G network, a 4G network, a Wi-Fi network, or the like. The electronic device 20 may be a server, an intelligent terminal, a computer, or the like.
Thus, the electronic device 20 may obtain at least two game videos of the same game; identifying a target segment corresponding to a target action in each match video, wherein the target segment is composed of images with continuous shooting time, and the target action is a symbolic action for starting a match; determining an image corresponding to a target gesture in the target segment, and taking the image corresponding to the target gesture as a synchronous frame corresponding to each competition video; and synchronizing the at least two game videos according to the synchronization frame.
Referring to fig. 2, an embodiment of the present invention provides a video synchronization method, which can be applied to an electronic device in the application environment 10, where the electronic device can be a smart phone, a computer, a server, or the like, and the method can specifically include the following steps.
At least two game videos of the same game collected from different locations are obtained, step 110.
The electronic device may obtain at least two game videos of the same game from different image capture devices. At least two image acquisition devices can be arranged in the field, and each image acquisition device can be arranged at different positions of the field, wherein the shooting visual field of each image acquisition device can cover the whole field.
After each image acquisition device shoots a match video, the shot match video can be sent to the electronic equipment, so that the electronic equipment can acquire at least two match videos of the same match collected from different positions.
And 120, identifying a target segment corresponding to a target action in each match video, wherein the target segment is composed of images with continuous shooting time, and the target action is used as a symbolic action for starting a match.
After the at least two match videos are obtained, because the positions and angles of the match videos shot by each image acquisition device are different, a target segment corresponding to a target action in each match video can be identified. The target section is a video section in the game video and is formed by images with continuous shooting time. Specifically, the target action is a symbolic action of the start of a game, for example, in a basketball game and a soccer game, a midcourt service action may be performed, and in a volleyball game, a badminton game, a table tennis game, and a tennis game, a first service action may be performed.
The identification of the target segment corresponding to the target action in the game video may be implemented by using a neural network model. Specifically, the neural network may be trained in advance to obtain a recognition model, so that the recognition model has the capability of recognizing the target motion from the game video.
When a target segment corresponding to a target action in the game video is identified, the game video can be cut into a plurality of video segments; inputting the plurality of video segments into a pre-trained prediction model to obtain candidate video segments; and determining the target segment from the candidate video segments according to the number of the candidate video segments.
Before identifying a target segment corresponding to a target action in each game video, determining a game category of the game video as a target category; and inquiring an information table according to the target category, and determining a target action and a target gesture corresponding to the target category, wherein the information table comprises the corresponding relation between the race category and the action and gesture. Therefore, the target action corresponding to the game video can be determined, and then the video segment corresponding to the target action in the game video is identified.
Since the target segment is composed of time-continuous images, a specific time needs to be determined during video synchronization, and this time can be referred to as a certain image in the target segment. That is, an image corresponding to a target pose may be determined from the target segment as a synchronization frame corresponding to the game video.
Specifically, a human skeleton key point and an object key point corresponding to each image in the target segment may be obtained; and determining an image corresponding to the target posture according to the human skeleton key points and the object key points.
According to the steps, the synchronous frames can be determined from each target segment, then each game video corresponds to one synchronous frame, and the corresponding time of the synchronous frame is the same time in the game. For example, the picture of the 100 th frame in the game video a is a synchronization frame, and the picture of the 200 th frame in the game video B is a synchronization frame, which are the time when the game starts. According to the synchronous frame, the synchronization of the competition video can be realized, and further data support is provided for three-dimensional modeling according to the competition video.
The video synchronization method provided by the embodiment of the application obtains at least two match videos of the same match collected from different positions; identifying a target segment corresponding to a target action in each match video, wherein the target segment is composed of images with continuous shooting time, and the target action is a symbolic action for starting a match; determining an image corresponding to a target gesture in the target segment, and taking the image corresponding to the target gesture as a synchronous frame corresponding to each competition video; and synchronizing the at least two game videos according to the synchronization frame. The method comprises the steps of identifying a target segment from each game video, determining a synchronization frame in each game video from the target segment, realizing video synchronization based on the synchronization frame, and being free from any auxiliary synchronization equipment, simple and convenient to operate and easy to apply in a large scale.
Referring to fig. 3, another embodiment of the present application provides a video synchronization method, which focuses on the processes of identifying a target segment and determining a synchronization frame based on the foregoing embodiments, and specifically includes the following steps.
At least two game videos of the same game collected from different locations are obtained, step 210.
Step 210 may refer to corresponding parts of the foregoing embodiments, and will not be described herein.
After the game videos are obtained, for each game video, the game video may be cut into a plurality of video segments. Specifically, the length of the cut video segment may be preset, and the cutting manner may be selected according to actual needs, which is not specifically limited herein. For example, the game video may be cut into fixed length video segments using a fixed length sliding window, sliding at a rate of 1 frame step size.
And step 230, inputting the plurality of video segments into a pre-trained prediction model to obtain candidate video segments.
After each match video is cut, a plurality of corresponding video segments can be obtained. And inputting a video segment corresponding to a competition video into the prediction model, and outputting the candidate video segment. The target action is included in the candidate video segment.
The prediction model is obtained by training in advance according to the sample segment and the marking information corresponding to the sample segment. Before training the neural network model, labeling information may be obtained by labeling each sample segment, for example, labeling a sample segment including a target action as 1, and labeling a sample segment not including the target action as 0.
And inputting the sample segment and the corresponding marking information into the neural network model, and outputting the sample segment comprising the target action. If the corresponding labeling information of the sample segment is 0, the target action is not included in the sample segment, and the parameters of the neural network model are adjusted until the target action is included in the sample segments output by the neural network model.
It is understood that a plurality of video segments including the target action may appear in the game video, and these video segments are taken as candidate video segments, and the target segment is further determined according to the number of the candidate video segments.
After the candidate video segments are obtained, the number of candidate video segments may be determined. If the number of the candidate video segments is 1, it indicates that only one video segment in the game video includes the target motion, so that the candidate video segment can be considered as the target segment.
If the number of the candidate video segments is larger than 1, it indicates that at least two video segments including the target action exist in the game video. Since the target action is a symbolic action at the beginning of the game, when at least two candidate video segments exist, the time sequence of the candidate video segments appearing in the game video can be obtained, and the candidate video segment with the earliest time sequence is determined as the target segment. For example, there are two candidate video segments, segment a and segment B, and the time of segment a in the game video is determined to be 2 minutes and segment B is determined to be 10 minutes, so that the target segment of segment a can be determined.
And step 250, acquiring human skeleton key points and object key points corresponding to each image in the target fragment.
After the target segment is obtained, the human skeleton key points and the object key points corresponding to each image in the target segment can be identified through a human skeleton key point identification algorithm and an object identification algorithm.
And step 260, determining the image corresponding to the target posture as the synchronous frame according to the human skeleton key points and the object key points.
After the human skeleton key points and the object key points are identified, whether the image is in a target posture or not can be determined based on the human skeleton key points and the object key points, and if yes, the image is determined to be a synchronous frame.
In some embodiments, a target human skeleton key point and a target object key point corresponding to a target pose may be obtained, and when the human skeleton key point is matched with the target human skeleton key point and the object key point is matched with the target object key point, the image is considered as the target pose, so that the image may be determined as a synchronous frame.
In some embodiments, the synchronous frame may be determined according to the variation trend of the human skeleton key points and the object key points in each frame of image. For example, in a table tennis stadium, a player throws a ball upward with one hand during a first serve, and the image of the highest point of the upward movement of the throwing hand is a sync frame.
It should be noted that, the two manners may be selected according to a specific target pose, so as to determine that an image corresponding to the target pose in the target segment is a synchronous frame.
Step 270, synchronizing the at least two game videos according to the synchronization frame corresponding to each game video.
Step 270 can refer to the corresponding parts of the previous embodiments, and will not be described herein.
According to the video synchronization method provided by the embodiment of the application, a target segment corresponding to a target action in a match video is identified through a prediction model; acquiring a human skeleton key point and an object key point corresponding to each image in the target fragment; determining an image corresponding to the target posture as the synchronous frame according to the human skeleton key points and the object key points; and synchronizing the at least two game videos according to the synchronization frame corresponding to each game video. The method comprises the steps of determining a target segment in each competition video from at least two competition videos, determining a synchronization frame from the target segments, realizing video synchronization based on the synchronization frame, and being free of any auxiliary synchronization equipment, simple and convenient to operate and easy to apply in a large scale.
Referring to fig. 4, a video synchronization method according to still another embodiment of the present application is mainly described in the foregoing embodiments, and the method may specifically include the following steps.
At step 310, at least two game videos of the same game captured from different locations are obtained.
Step 310 may refer to corresponding parts of the foregoing embodiments, and will not be described herein.
After the match video is acquired, the match category corresponding to the match video may be determined as a target category.
As an implementation mode, the match category corresponding to the match video can be confirmed manually by staff.
As an implementation mode, one frame of image can be arbitrarily extracted from the game video, the field in the image is identified through a neural network model, and the target category is determined according to the identified field. For example, the identified field is a basketball field, and the target category is determined to be a basketball game.
The game type may be one of a basketball game, a football game, a volleyball game, a badminton game, a table tennis game, a hockey game, and a curling game.
After determining the target category, a target action and a target gesture corresponding to the target category may be queried through an information table. The information table may be preset to be stored in the electronic device, and includes the corresponding relationship between the game category and the action and the gesture. The information table may refer to table 1.
TABLE 1
| Race categories | Movement of | Posture of a person |
| Class 1 | Action 1 | Posture 1 |
| Class 2 | Action 2 | Posture 2 |
If the target type is determined to be type 1, the target action can be determined to be action 1 and the target posture can be determined to be posture 1 through the information table.
The actions and gestures corresponding to the game category will be described in detail below.
When the target category is a basketball game, the target acts as a midcourt service, and the target posture is that the hands of the player contact the basketball for the first time.
When the target category is a football match, the target acts as a midcourt service, and the target posture is that the feet of the players contact the football for the first time.
When the object type is volleyball match, the object movement is used as first serve, and the object posture is that the hands of the player contact volleyball for the first time.
When the target category is a badminton match, the target movement is used as first serve, in the first serve movement, the player swings backward to beat, then swings forward to beat to hit the badminton, and at the moment, when the player swings backward to beat, the posture of the arm at the back is the target posture.
When the target category is a table tennis game, the target action is taken as first serve, and in the first serve action, the player throws the ball upwards with one hand, and at the moment, the posture of the highest point of upward movement of the throwing hand is taken as the target posture.
When the target category is tennis match, the target action is taken as first serve, in the first serve action, the player swings backward first and then swings forward to hit the tennis, and at the moment, when the player swings backward, the posture of the arm at the most backward position is the target posture.
When the target category is ice hockey game, the target action is the first ball contest at the center ball contest point, and the target posture is the first swing posture of the player.
When the target category is curling game, the target action is the posture that both players start curling, and the target posture is the posture that both players stop sliding.
And 340, identifying a target segment corresponding to a target action in each match video, wherein the target segment is composed of images with continuous shooting time, and the target action is taken as a symbolic action for the start of a match.
And step 360, synchronizing the at least two game videos according to the synchronization frame corresponding to each game video.
According to the video synchronization method provided by the embodiment of the application, after the match video is obtained, the match category corresponding to the match video is determined as the target category; determining a target action and a target posture corresponding to the target category according to the target category and a query information table; and then identifying target actions and synchronous frames in the competition video, and carrying out video synchronization based on the synchronous frames. The method comprises the steps of identifying a target segment corresponding to a symbolic action at the beginning of a game from at least two game videos, and then determining a synchronization frame in each game video from the target segment, wherein the actual occurrence time of the synchronization frames is the same no matter how the acquisition modes of the game videos are based on the same target posture corresponding to the symbolic action at the beginning of the game, and the actual acquisition time points of the corresponding synchronization frames are the same.
Referring to fig. 5, an embodiment of the present application provides a video synchronization apparatus 400, where the video synchronization apparatus 400 includes an obtaining module 410, an identifying module 420, a synchronization frame determining module 430, and a synchronizing module 440. The obtaining module 410 is configured to obtain at least two game videos of the same game collected from different locations; the identification module 420 is configured to identify a target segment corresponding to a target action in each of the match videos, where the target segment is composed of images with continuous shooting time, and the target action is a symbolic action of a start of a match; the synchronization frame determining module 430 is configured to determine an image corresponding to a target pose in the target segment, and use the image corresponding to the target pose as a synchronization frame corresponding to each game video; the synchronization module 440 is configured to synchronize the at least two game videos according to a synchronization frame corresponding to each of the game videos.
Further, the video synchronization apparatus 400 further includes a target determination module, where the target determination module is configured to determine a game category corresponding to the game video as a target category; and inquiring an information table according to the target category, and determining a target action and a target gesture corresponding to the target category, wherein the information table comprises the corresponding relation between the race category and the action and gesture.
Further, the identification module 420 is further configured to cut the game video into a plurality of video segments; inputting the video segments into a pre-trained prediction model to obtain candidate video segments; and determining the target segment from the candidate video segments according to the number of the candidate video segments.
Further, the identifying module 420 is further configured to determine that the candidate video segment is the target segment if the number of the candidate video segments is 1; and if the number of the candidate video clips is larger than 1, determining the candidate video clip with the earliest time sequence in the game video as the target clip.
Further, the synchronous frame determining module 430 is further configured to obtain a human skeleton key point and an object key point corresponding to each image in the target segment; and determining the image corresponding to the target posture as the synchronous frame according to the human skeleton key points and the object key points.
Further, when the game category is a basketball game, the target action is a midcourt shot, and the target posture is that the player's hand first contacts the basketball.
Further, when the match category is a volleyball match, the target segment is a first serve, and the target posture is a first contact of the player's hand with the volleyball.
The video synchronization device provided by the embodiment of the application acquires at least two match videos of the same match, which are acquired from different positions; identifying a target segment corresponding to a target action in each match video, wherein the target segment is composed of images with continuous shooting time, and the target action is a symbolic action for starting a match; determining an image corresponding to a target gesture in the target segment, and taking the image corresponding to the target gesture as a synchronous frame corresponding to each competition video; and synchronizing the at least two game videos according to the synchronization frame. The method comprises the steps of identifying a target segment corresponding to a symbolic action at the beginning of a game from at least two game videos, and then determining a synchronization frame in each game video from the target segment, wherein the actual occurrence time of the synchronization frames is the same no matter how the acquisition modes of the game videos are based on the same target posture corresponding to the symbolic action at the beginning of the game, and the actual acquisition time points of the corresponding synchronization frames are the same.
It should be noted that, as will be clear to those skilled in the art, for convenience and brevity of description, the specific working process of the above-described apparatus may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.
Referring to fig. 6, an embodiment of the present application provides a block diagram of an electronic device 500, where the electronic device 500 includes a processor 510, a memory 520, and one or more applications, where the one or more applications are stored in the memory 520 and configured to be executed by the one or more processors 510, and the one or more programs are configured to perform the above-mentioned video synchronization method.
The electronic device 500 may be a terminal device capable of running an application, such as a smart phone or a tablet computer, or may be a server. The electronic device 500 in the present application may include one or more of the following components: a processor 510, a memory 520, and one or more applications, wherein the one or more applications may be stored in the memory 520 and configured to be executed by the one or more processors 510, the one or more programs configured to perform a method as described in the aforementioned method embodiments.
Processor 510 may include one or more processing cores. The processor 510 interfaces with various components throughout the electronic device 500 using various interfaces and circuitry to perform various functions of the electronic device 500 and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 520 and invoking data stored in the memory 520. Alternatively, the processor 510 may be implemented in hardware using at least one of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 510 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 510, but may be implemented by a communication chip.
The Memory 520 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). The memory 520 may be used to store instructions, programs, code sets, or instruction sets. The memory 520 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described below, and the like. The data storage area may also store data created during use by the electronic device 500 (e.g., phone books, audio game videos, chat log data), and so forth.
The electronic equipment provided by the embodiment of the application acquires at least two match videos of the same match, which are acquired from different positions; identifying a target segment corresponding to a target action in each match video, wherein the target segment is composed of images with continuous shooting time, and the target action is a symbolic action for starting a match; determining an image corresponding to a target gesture in the target segment, and taking the image corresponding to the target gesture as a synchronous frame corresponding to each competition video; and synchronizing the at least two game videos according to the synchronization frame. The method comprises the steps of identifying a target segment corresponding to a symbolic action at the beginning of a game from at least two game videos, and then determining a synchronization frame in each game video from the target segment, wherein the actual occurrence time of the synchronization frames is the same no matter how the acquisition modes of the game videos are based on the same target posture corresponding to the symbolic action at the beginning of the game, and the actual acquisition time points of the corresponding synchronization frames are the same.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A method for video synchronization, the method comprising:
acquiring at least two match videos of the same match, which are acquired from different positions;
identifying a target segment corresponding to a target action in each match video, wherein the target segment is composed of images with continuous shooting time, and the target action is a symbolic action for starting a match;
determining an image corresponding to a target gesture in the target segment, and taking the image corresponding to the target gesture as a synchronous frame corresponding to each competition video;
and synchronizing the at least two game videos according to the synchronization frame corresponding to each game video.
2. The method of claim 1, wherein prior to identifying the target segment in each of the game videos, the method further comprises:
determining the match category corresponding to the match video as a target category;
and inquiring an information table according to the target category, and determining a target action and a target gesture corresponding to the target category, wherein the information table comprises the corresponding relation between the race category and the action and gesture.
3. The method of claim 2, wherein the identifying of the target segment corresponding to the target action in each of the game videos comprises:
cutting the game video into a plurality of video segments;
inputting the video segments into a pre-trained prediction model to obtain candidate video segments;
and determining the target segment from the candidate video segments according to the number of the candidate video segments.
4. The method of claim 3, wherein determining the target segment from the candidate video segments according to the number of candidate video segments comprises:
if the number of the candidate video clips is 1, determining the candidate video clips as the target clips;
and if the number of the candidate video clips is larger than 1, determining the candidate video clip with the earliest time sequence in the game video as the target clip.
5. The method of claim 2, wherein determining the image corresponding to the target pose in the target segment as a synchronization frame corresponding to each of the game videos comprises:
acquiring a human skeleton key point and an object key point corresponding to each image in the target fragment;
and determining the image corresponding to the target posture as the synchronous frame according to the human skeleton key points and the object key points.
6. The method of any one of claims 1 to 5, wherein when the game category is a basketball game, the target action is a midcourt shot and the target pose is the first contact of the player's hands with a basketball.
7. The method of any one of claims 1 to 5, wherein when the game category is a volleyball game, the target segment is a first serve and the target pose is a first contact of a player's hand with the volleyball.
8. A video synchronization apparatus, characterized in that the apparatus comprises:
the acquisition module is used for acquiring at least two match videos of the same match, which are acquired from different positions;
the identification module is used for identifying a target segment corresponding to a target action in each competition video, wherein the target segment is composed of images with continuous shooting time, and the target action is used as a symbolic action for the beginning of a competition;
a synchronous frame determining module, configured to determine an image corresponding to a target pose in the target segment, and use the image corresponding to the target pose as a synchronous frame corresponding to each match video;
and the synchronization module is used for synchronizing the at least two match videos according to the synchronization frame corresponding to each match video.
9. An electronic device, characterized in that the electronic device comprises:
one or more processors;
a memory electrically connected with the one or more processors;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the method of any of claims 1-7.
10. A computer-readable storage medium, having stored thereon program code that can be invoked by a processor to perform the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110625325.9A CN113542774B (en) | 2021-06-04 | 2021-06-04 | Video synchronization method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110625325.9A CN113542774B (en) | 2021-06-04 | 2021-06-04 | Video synchronization method, device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113542774A true CN113542774A (en) | 2021-10-22 |
| CN113542774B CN113542774B (en) | 2023-10-20 |
Family
ID=78095154
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110625325.9A Active CN113542774B (en) | 2021-06-04 | 2021-06-04 | Video synchronization method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113542774B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101646022A (en) * | 2009-09-04 | 2010-02-10 | 深圳华为通信技术有限公司 | Image splicing method and system thereof |
| CN106464848A (en) * | 2014-04-21 | 2017-02-22 | 博拉斯特运动有限公司 | Motion event recognition and video synchronization system and method |
| CN107534789A (en) * | 2015-06-25 | 2018-01-02 | 松下知识产权经营株式会社 | Image synchronization device and image synchronous method |
| US20180218243A1 (en) * | 2017-01-31 | 2018-08-02 | Stats Llc | System and method for predictive sports analytics using body-pose information |
| CN110087116A (en) * | 2019-03-14 | 2019-08-02 | 咪咕视讯科技有限公司 | Multi code Rate of Chinese character live video stream clipping method, device, terminal and storage medium |
| CN110516572A (en) * | 2019-08-16 | 2019-11-29 | 咪咕文化科技有限公司 | A kind of method, electronic equipment and storage medium identifying competitive sports video clip |
| CN110765896A (en) * | 2019-10-08 | 2020-02-07 | 维沃移动通信有限公司 | Video processing method and device |
| CN111507219A (en) * | 2020-04-08 | 2020-08-07 | 广东工业大学 | Action recognition method and device, electronic equipment and storage medium |
| CN111680562A (en) * | 2020-05-09 | 2020-09-18 | 北京中广上洋科技股份有限公司 | Human body posture identification method and device based on skeleton key points, storage medium and terminal |
| US20210065452A1 (en) * | 2019-08-30 | 2021-03-04 | Sprongo, LLC | Instant technique analysis for sports |
-
2021
- 2021-06-04 CN CN202110625325.9A patent/CN113542774B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101646022A (en) * | 2009-09-04 | 2010-02-10 | 深圳华为通信技术有限公司 | Image splicing method and system thereof |
| CN106464848A (en) * | 2014-04-21 | 2017-02-22 | 博拉斯特运动有限公司 | Motion event recognition and video synchronization system and method |
| CN107534789A (en) * | 2015-06-25 | 2018-01-02 | 松下知识产权经营株式会社 | Image synchronization device and image synchronous method |
| US20180218243A1 (en) * | 2017-01-31 | 2018-08-02 | Stats Llc | System and method for predictive sports analytics using body-pose information |
| CN110087116A (en) * | 2019-03-14 | 2019-08-02 | 咪咕视讯科技有限公司 | Multi code Rate of Chinese character live video stream clipping method, device, terminal and storage medium |
| CN110516572A (en) * | 2019-08-16 | 2019-11-29 | 咪咕文化科技有限公司 | A kind of method, electronic equipment and storage medium identifying competitive sports video clip |
| US20210065452A1 (en) * | 2019-08-30 | 2021-03-04 | Sprongo, LLC | Instant technique analysis for sports |
| CN110765896A (en) * | 2019-10-08 | 2020-02-07 | 维沃移动通信有限公司 | Video processing method and device |
| CN111507219A (en) * | 2020-04-08 | 2020-08-07 | 广东工业大学 | Action recognition method and device, electronic equipment and storage medium |
| CN111680562A (en) * | 2020-05-09 | 2020-09-18 | 北京中广上洋科技股份有限公司 | Human body posture identification method and device based on skeleton key points, storage medium and terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113542774B (en) | 2023-10-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108712661B (en) | Live video processing method, device, equipment and storage medium | |
| US10713543B1 (en) | Enhanced training of machine learning systems based on automatically generated realistic gameplay information | |
| JP7033587B2 (en) | How and system to automatically create video highlights | |
| US20200320304A1 (en) | Non-transitory computer readable recording medium, extraction method, and information processing apparatus | |
| CN110298220B (en) | Action video live broadcast method, system, electronic equipment and storage medium | |
| CN109819342A (en) | Barrage contents controlling method, device, computer equipment and storage medium | |
| CN109145809B (en) | Notation processing method and device and computer readable storage medium | |
| CN110505519A (en) | A kind of video clipping method, electronic equipment and storage medium | |
| CN108337573A (en) | A kind of implementation method that race explains in real time and medium | |
| CN114651284A (en) | Lightweight multi-branch and multi-scale heavy person identification | |
| US11819734B2 (en) | Video-based motion counting and analysis systems and methods for virtual fitness application | |
| CN101807393A (en) | KTV system, implement method thereof and TV set | |
| US20220366653A1 (en) | Full Body Virtual Reality Utilizing Computer Vision From a Single Camera and Associated Systems and Methods | |
| CN112560605A (en) | Interaction method, device, terminal, server and storage medium | |
| CN107273857B (en) | Motion action recognition method and device and electronic equipment | |
| CN115331314A (en) | Exercise effect evaluation method and system based on APP screening function | |
| CN111741325A (en) | Video playing method and device, electronic equipment and computer readable storage medium | |
| CN114513694A (en) | Scoring determination method and device, electronic equipment and storage medium | |
| CN114140721A (en) | Archery posture evaluation method and device, edge calculation server and storage medium | |
| CN113542774B (en) | Video synchronization method, device, electronic equipment and storage medium | |
| CN115475373A (en) | Motion data display method and device, storage medium and electronic device | |
| CN110969133B (en) | Intelligent data acquisition method for table tennis game video | |
| CN114425162A (en) | Video processing method and related device | |
| CN114302234A (en) | Air skill rapid packaging method | |
| CN110853130B (en) | Three-dimensional image generation method, electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |