CN112019880B - Video file splicing method with any frame rate - Google Patents

Video file splicing method with any frame rate Download PDF

Info

Publication number
CN112019880B
CN112019880B CN202011187951.6A CN202011187951A CN112019880B CN 112019880 B CN112019880 B CN 112019880B CN 202011187951 A CN202011187951 A CN 202011187951A CN 112019880 B CN112019880 B CN 112019880B
Authority
CN
China
Prior art keywords
frame
video
frame rate
video file
splicing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011187951.6A
Other languages
Chinese (zh)
Other versions
CN112019880A (en
Inventor
胡一凡
张宇
殷力
李晓聪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South Sagittarius Integration Co Ltd
Original Assignee
South Sagittarius Integration Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South Sagittarius Integration Co Ltd filed Critical South Sagittarius Integration Co Ltd
Priority to CN202011187951.6A priority Critical patent/CN112019880B/en
Publication of CN112019880A publication Critical patent/CN112019880A/en
Application granted granted Critical
Publication of CN112019880B publication Critical patent/CN112019880B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23424Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/854Content authoring
    • H04N21/85406Content authoring involving a specific file format, e.g. MP4 format
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/854Content authoring
    • H04N21/8547Content authoring involving timestamps for synchronizing content

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

A video file splicing method with any frame rate comprises the following steps: acquiring video files to be spliced, and determining the splicing sequence of the video files to be spliced; respectively acquiring frame rates of video files to be spliced, and judging the formats and data correctness of the video files; generating a spliced target video file according to a frame rate of a video file to be spliced and a first rule preset by a splicing device; and after the target video file is generated, updating file header information of the target file, writing the file header information into a file tail, and completing video file splicing. According to the invention, by searching the minimum video frame rate, the files participating in splicing take the minimum video frame rate as a reference, and the video frame rates of the files participating in splicing are taken as a period, frames are dropped uniformly, the frame rates of the spliced videos are uniform after splicing, the playing time lengths of the videos are unchanged, and the videos are synchronized, so that the overall performance of the video files generated by splicing is ensured to be good. The invention has simple logic and easy realization. The splicing effect is good, and the occupied space of the magnetic disk is small. High feasibility and reliability and wide application range.

Description

Video file splicing method with any frame rate
Technical Field
The invention relates to the technical field of video splicing, in particular to a video file splicing method at any frame rate.
Background
With the rapid development of network communication technology, people have more and more diversified entertainment modes, and the fast fashion of live online broadcasting, short videos, dramas, network movies and the like based on streaming media is more a fierce network nowadays. In order to enhance the video interest, the user can operate the playing speed of the video file, add background music to the video file, clip the video file, splice any video file and the like.
Video splicing, in which two or more video files are connected into one video file according to a specified sequence, and if the video is provided with audio, the audio is spliced together. Each video file participating in splicing contains the attributes of the playing time length, the audio and video number, the video frame rate and the like, and the video file generated by splicing also contains the attributes. The video splicing device not only needs to realize the video splicing function, but also needs to ensure the splicing performance, if the playing time length before and after splicing is unchanged, the spliced video file can be normally spliced again, the occupied disk space is not obviously increased, the playing picture is smooth, the audio and video are synchronous, and the like. The video frame rate refers to the number of pictures displayed in each second of the video. In the prior art, if all video file frame rates participating in splicing are the same, the splicing performance is easy to guarantee, and if all video file frame rates participating in splicing are not completely the same, the splicing performance is difficult to guarantee.
Disclosure of Invention
In view of the above problems, the present invention has been made to provide a method for splicing video files at an arbitrary frame rate, which overcomes or at least partially solves the above problems.
A video file splicing method with any frame rate is characterized by comprising the following steps:
s100, acquiring video files to be spliced, and determining the splicing sequence of the video files to be spliced;
s200, respectively acquiring frame rates of video files to be spliced, and judging video file formats and data correctness;
s300, according to the frame rate of the video files to be spliced, generating spliced target video files according to a first rule preset by a splicing device;
and S400, after the target video file is generated, updating file header information of the target file, writing the file header information into a file tail, and completing video file splicing.
Further, in S100, the format of the video file to be spliced at least includes: MP4, FLV, WMV, AVI, and MPEG.
Further, in S200, the method for determining the video file format and the data correctness is as follows: the method comprises the steps of obtaining a video file to be spliced, obtaining audio and video stream information and a video frame rate of the video file to be spliced, creating a corresponding coder-decoder, and judging that the format and the data of the current video file to be spliced are correct if the corresponding coder-decoder can be created.
Further, in S300, the splicing preset first rule includes: the method comprises the steps of obtaining frame rates of video files to be spliced, wherein the frame rates are respectively a first frame rate and a second frame rate, comparing the first frame rate with the second frame rate, assigning the current frame rate to the frame rate of a target video file when the first frame rate is equal to the second frame rate, starting to encode and decode audio and video data in the video files to be spliced, assigning a timestamp for an audio and video generated by encoding, writing the audio and video data into the target video file, and assigning the timestamp for the video frame again to depend on the frame rate of the target video file.
Further, in S300, the splicing preset first rule further includes: acquiring frame rates of video files to be spliced, namely a first frame rate and a second frame rate respectively, comparing the first frame rate with the second frame rate, acquiring the frame rate assigned to a target video file by the smaller frame rate when the first frame rate is not equal to the second frame rate, encoding and decoding the video files to be spliced with the smaller frame rate, assigning timestamps again, and writing the timestamps into the target video file;
further, in S300, the splicing preset first rule further includes: the method comprises the steps of coding and decoding an audio in a video file with a high frame rate and reassigning a timestamp, decoding a video in the video file with the high frame rate to generate original YUV data, performing frame loss judgment on the generated original YUV data according to a preset second rule by a frame loss module, judging whether the original YUV data is reserved or discarded, if the YUV data is reserved, coding the YUV data to generate a video frame, reassigning the timestamp for the video frame generated by coding, writing an audio and video frame after assigning the timestamp into a target video file, if the YUV data is discarded, terminating the processing of the frame data, and reading the next frame of audio and video data of the file.
Further, the frame loss presetting second rule comprises: calculating the frame rate of a video file with a larger frame rate and the frame rate of a target video file, respectively obtaining a quotient value QuotientValue and a residual value ResidualValue after quotient operation of the two, when the residual value ResidualValue is 0, continuously losing frames according to the rule that 1 frame is taken by each QuotientValue frame, when a video file needing frame loss decodes one frame of video, a splicing device calls a frame loss module, counting the called times Count inside the module, when the value obtained by the Count for the QuotientValue is 1, the frame loss module informs the splicing device of frame preservation, otherwise, informs the splicing device of frame loss, when the value of the Count is equal to the frame rate of the video file with the larger frame rate, the Count is 0, and a next frame loss period is prepared.
Further, the frame loss presetting second rule further comprises: when the residual value ResidualValue is not 0, the frame loss module continuously takes 1 frame according to each QuotientValue frame, and takes 1 frame regular frame loss every (QuotientValue + 1) frame, namely, takes 2 frames regular frame loss every (QuotientValue + 2 + 1) frame, and when a video file needing frame loss decodes a frame, the splicing device calls the frame loss module once, the internal statistics of the called times Count and the retained video frame number Num are carried out, if the retained video Num plus the frame number of the maximum frame number RemainNum value which can be retained by the residual frames in the period is less than the frame rate of the target video file, the splicing device is informed to retain the current frame, otherwise, the splicing device is informed to discard the current frame; when the Count value is equal to the frame rate of the video file with the larger frame rate, setting both the Count value and the Num value to be 0, and preparing for the next frame loss period to arrive.
Further, the maximum number RemainNum of frames that can be reserved for remaining frames in the period is represented by the following formula:
Figure 100002_DEST_PATH_IMAGE001
the SrcFps represents the frame rate of a video file with a larger frame rate of a video file to be spliced, the Count represents the called times of a frame loss module, and the QuotientValue represents a quotient value obtained by performing quotient operation on the frame rate of the video file with the larger frame rate and the frame rate of a target video file.
Further, the frame loss presetting second rule further comprises: for the first frame video data of each period, the frame loss module directly informs the splicing device to reserve, and from the second frame in the period, the splicing device calculates the rest frames in the period and continuously adopts the rule that every QuotientValue frame takes 1 frame, and every (QuotientValue + 1) frame takes 1 frame to perform frame loss.
The technical scheme provided by the embodiment of the invention has the beneficial effects that at least:
the invention discloses a video file splicing method with any frame rate, which comprises the steps of obtaining video files to be spliced, determining the splicing sequence of the video files to be spliced, and judging the format and the data correctness of the video files; if the format and the data of the video file to be spliced are correct, respectively acquiring the frame rate of the video file to be spliced; generating a spliced target video file according to a frame rate of a video file to be spliced and a first rule preset by a splicing device; and after the target video file is generated, updating file header information of the target file, writing the file header information into a file tail, and completing video file splicing. According to the invention, by searching the minimum video frame rate, the files participating in splicing take the minimum video frame rate as a reference, and the video frame rates of the files participating in splicing are taken as a period, frames are dropped uniformly, the frame rates of the spliced videos are uniform after splicing, the playing time lengths of the videos are unchanged, and the videos are synchronized, so that the overall performance of the video files generated by splicing is ensured to be good. The invention has simple logic and easy realization. The splicing effect is good, and the occupied space of the magnetic disk is small. High feasibility and reliability and wide application range.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
fig. 1 is a flowchart of a video file splicing method with an arbitrary frame rate according to embodiment 1 of the present invention;
FIG. 2 is a flow chart of the frame loss video file splicing in embodiment 2 of the present invention;
FIG. 3 is a flowchart illustrating a processing procedure of a ResidualValue non-0 frame loss module according to embodiment 2 of the present invention;
fig. 4 is a flowchart of the processing of the frame loss module with a residual value of 0 in embodiment 2 of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In order to solve the problems that frame rates of spliced video files are not identical and video splicing effects are difficult to guarantee in the prior art, the embodiment of the invention provides a video file splicing method with any frame rate.
Example 1
Fig. 1 is a flowchart of a video file stitching method with an arbitrary frame rate, including:
s100, video files to be spliced are obtained, and the splicing sequence of the video files to be spliced is determined.
In this embodiment, in S100, the format of the video file to be spliced at least includes: MP4, FLV, WMV, AVI, and MPEG. The number of audio and video data streams in the video file is not limited, and the audio and video data streams can only contain video, audio or audio and video. For the media files only containing audio, the video frame rate comparison is not involved, the splicing device can also successfully realize splicing, and the performance is good. In addition, the splicing device processes three or more video files, the more the video files participating in splicing are, but the longer the splicing processing process is, the overall logic is completely the same as the logic for processing two video files, and the embodiment takes the splicing of two video files as an example for introduction.
S200, respectively obtaining the frame rates of the video files to be spliced, and judging the video file formats and the data correctness. In this embodiment, the method for determining the format and data correctness of the video file is as follows: the method comprises the steps of obtaining a video file to be spliced, obtaining audio and video stream information and a video frame rate of the video file to be spliced, creating a corresponding coder-decoder, and judging that the format and the data of the current video file to be spliced are correct if the corresponding coder-decoder can be created.
S300, according to the frame rate of the video files to be spliced, according to a first rule preset by a splicing device, generating spliced target video files.
In this embodiment, the splicing preset first rule includes: the method comprises the steps of obtaining frame rates of video files to be spliced, wherein the frame rates are respectively a first frame rate and a second frame rate, comparing the first frame rate with the second frame rate, assigning the current frame rate to the frame rate of a target video file when the first frame rate is equal to the second frame rate, starting to encode and decode audio and video data in the video files to be spliced, assigning a timestamp for an audio and video generated by encoding, writing the audio and video data into the target video file, and assigning the timestamp for the video frame again to depend on the frame rate of the target video file.
In some preferred embodiments, the stitching preset first rule further comprises: the method comprises the steps of obtaining frame rates of video files to be spliced, wherein the frame rates are respectively a first frame rate and a second frame rate, comparing the first frame rate with the second frame rate, obtaining the frame rate assigned to a target video file by the lower frame rate of the first frame rate and the second frame rate when the first frame rate is not equal to the second frame rate, coding and decoding the video files to be spliced with the lower frame rate, assigning timestamps again, and writing the timestamps into the target video file.
In some preferred embodiments, the stitching preset first rule further comprises: the method comprises the steps of coding and decoding an audio in a video file with a high frame rate and reassigning a timestamp, decoding a video in the video file with the high frame rate to generate original YUV data, performing frame loss judgment on the generated original YUV data according to a preset second rule by a frame loss module, judging whether the original YUV data is reserved or discarded, if the YUV data is reserved, coding the YUV data to generate a video frame, reassigning the timestamp for the video frame generated by coding, writing an audio and video frame after assigning the timestamp into a target video file, if the YUV data is discarded, terminating the processing of the frame data, and reading the next frame of audio and video data of the file.
In some preferred embodiments, the frame loss preset second rule comprises: calculating the frame rate of a video file with a larger frame rate and the frame rate of a target video file, respectively obtaining a quotient value QuotientValue and a residual value ResidualValue after quotient operation of the two, when the residual value ResidualValue is 0, continuously losing frames according to the rule that 1 frame is taken by each QuotientValue frame, when a video file needing frame loss decodes one frame of video, a splicing device calls a frame loss module, counting the called times Count inside the module, when the value obtained by the Count for the QuotientValue is 1, the frame loss module informs the splicing device of frame preservation, otherwise, informs the splicing device of frame loss, when the value of the Count is equal to the frame rate of the video file with the larger frame rate, the Count is 0, and a next frame loss period is prepared.
In some preferred embodiments, the frame loss preset second rule further comprises: when the residual value ResidualValue is not 0, continuously taking 1 frame by the frame dropping module according to each QuotientValue frame, regularly dropping 1 frame by each (QuotientValue + 1) frame, calling the frame dropping module once by the splicing device when a video file needing frame dropping decodes a frame, counting the called times Count and the reserved video frame number Num in the frame dropping module, if the sum of the reserved video frame number Num and the maximum frame number RemainNum value which can be reserved by the residual frame in the period is less than the frame rate of the target video file, informing the splicing device to reserve the current frame, otherwise, informing the splicing device to drop the current frame; when the Count value is equal to the frame rate of the video file with the larger frame rate, setting both the Count value and the Num value to be 0, and preparing for the next frame loss period to arrive.
In some preferred embodiments, the maximum number of frames RemainNum that can be retained for the remaining frames in the period is given by the following formula:
Figure DEST_PATH_IMAGE002
the SrcFps represents the frame rate of a video file with a larger frame rate of a video file to be spliced, the Count represents the called times of a frame loss module, and the QuotientValue represents a quotient value obtained by performing quotient operation on the frame rate of the video file with the larger frame rate and the frame rate of a target video file.
In some preferred embodiments, the frame loss preset second rule further comprises: for the first frame video data of each period, the frame loss module directly informs the splicing device to reserve, and from the second frame in the period, the splicing device calculates the rest frames in the period and continuously adopts the rule that every QuotientValue frame takes 1 frame, and every (QuotientValue + 1) frame takes 1 frame to perform frame loss.
And S400, after the target video file is generated, updating file header information of the target file, writing the file header information into a file tail, and completing video file splicing. In this embodiment, the streaming media files with different formats have different requirements for the head and tail information of the file, the head and tail information is missing or abnormal, and the player cannot normally play the file.
According to the method for splicing the video files with any frame rate, the video files to be spliced are obtained, the splicing sequence of the video files to be spliced is determined, and the format and the data correctness of the video files are judged; if the format and the data of the video file to be spliced are correct, respectively acquiring the frame rate of the video file to be spliced; generating a spliced target video file according to a frame rate of a video file to be spliced and a first rule preset by a splicing device; and after the target video file is generated, updating file header information of the target file, writing the file header information into a file tail, and completing video file splicing. According to the invention, by searching the minimum video frame rate, the files participating in splicing take the minimum video frame rate as a reference, and the video frame rates of the files participating in splicing are taken as a period, frames are dropped uniformly, the frame rates of the spliced videos are uniform after splicing, the playing time lengths of the videos are unchanged, and the videos are synchronized, so that the overall performance of the video files generated by splicing is ensured to be good. The invention has simple logic and easy realization. The splicing effect is good, and the occupied space of the magnetic disk is small. High feasibility and reliability and wide application range.
Example 2
In order to better understand the present invention, the present embodiment describes the present invention in detail with more specific examples. A flow chart of a video file splicing method at any frame rate comprises the following steps:
s100, video files to be spliced are obtained, and the splicing sequence of the video files to be spliced is determined. Specifically, a user designates media files participating in splicing and a splicing sequence to a splicing device, formats of the media files participating in splicing are not limited, common formats include MP4, FLV, WMV, AVI, MPEG and the like, the number of audio and video data streams in the files is not limited, and the files can only contain video, audio or audio and video. For the media files only containing audio, the video frame rate comparison is not involved, the splicing device can also successfully realize splicing, and the performance is good. In addition, the splicing device processes three or more video files, the more the video files participating in splicing are, but the longer the splicing processing process is, the overall logic is completely the same as the logic for processing two video files, the embodiment takes the splicing of two video files as an example for introduction, and it is assumed that the video files specified by the user are srcfile1.mp4 and srcfile2.flv, and the splicing sequence is srcfile1.mp4 before and srcfile2.flv after.
S200, respectively obtaining the frame rates of the video files to be spliced, and judging the video file formats and the data correctness. Specifically, the splicing device opens srcfile1.mp4 and srcfile2.flv specified by the user, reads audio and video attributes in the file, mainly including the encoding format of audio and video data and video frame rates SrcFps1 and SrcFps2, if the two steps of operations are successful, the video file specified by the user is normal, otherwise, the splicing device stops working, and the user is warned that the video file is abnormal. After audio and video attributes of SrcFile1.mp4 and SrcFile2.flv are successfully acquired, decoders corresponding to audio and video formats are respectively created, a splicing generated file DstFile. mp4 is created, the audio and video formats in the DstFile. mp4 are specified, if the video is in an H264 coding format, the audio is in an AAC coding format, and audio and video encoders corresponding to the formats are created.
S300, according to the frame rate of the video files to be spliced, according to a first rule preset by a splicing device, generating spliced target video files.
In this embodiment, the splicing preset first rule includes: the method comprises the steps of obtaining frame rates of video files to be spliced, wherein the frame rates are respectively a first frame rate and a second frame rate, comparing the first frame rate with the second frame rate, assigning the current frame rate to the frame rate of a target video file when the first frame rate is equal to the second frame rate, starting to encode and decode audio and video data in the video files to be spliced, assigning a timestamp for an audio and video generated by encoding, writing the audio and video data into the target video file, and assigning the timestamp for the video frame again to depend on the frame rate of the target video file.
Specifically, after acquiring video frame rates of srcfile1.mp4 and srcfile2.flv, the splicing device compares the sizes of SrcFps1 and SrcFps2, if SrcFps1 and SrcFps2 are equal, the video file participating in splicing does not need to lose frames, and the video frame rate DstFps of the file dstfile. mp4 generated by splicing is SrcFps 1; if SrcFps1 and SrcFps2 are unequal, the video file marked with the small frame rate value does not need to lose frames, the video file marked with the large frame rate value needs to lose video frames, and the video frame rate DstFps of the spliced generated file DstFile. mp4 is smaller than the values of SrcFps1 and SrcFps 2. When SrcFps1 and SrcFps2 are equal, the splicing device decodes and codes SrcFile1.mp4 and SrcFile2.flv audio and video data, time stamps are printed on the audio and video data generated by coding, and the audio and video data generated by coding are written into DstFile. mp4 according to the splicing sequence.
In some preferred embodiments, the stitching preset first rule further comprises: the method comprises the steps of coding and decoding an audio in a video file with a high frame rate and reassigning a timestamp, decoding a video in the video file with the high frame rate to generate original YUV data, performing frame loss judgment on the generated original YUV data according to a preset second rule by a frame loss module, judging whether the original YUV data is reserved or discarded, if the YUV data is reserved, coding the YUV data to generate a video frame, reassigning the timestamp for the video frame generated by coding, writing an audio and video frame after assigning the timestamp into a target video file, if the YUV data is discarded, terminating the processing of the frame data, and reading the next frame of audio and video data of the file.
Specifically, as shown in fig. 2, the splicing apparatus has the following working procedures: the splicing device opens video files SrcFile1.mp4 and SrcFile2.flv which participate in splicing, reads the attribute of audio and video stream in the video file, and assumes that the video frame rate of SrcFile1.mp4 is 45 frames per second, the video frame rate of SrcFile2.flv is 60 frames per second, both SrcFile1.mp4 and SrcFile2.flv contain audio stream and video stream, the video coding format in SrcFile1.mp4 is H265, the audio coding format is AAC, the video coding format in SrcFile2.flv is H264, the audio coding format is AAC, the splicing device respectively creates an audio and video decoder with corresponding formats for SrcFile1.mp4 and SrcFile2.flv, meanwhile, the splicing device creates a splicing generation file DstFile.mp4, selects the video format of DstFile.mp4 as H264, the audio format as AAC, and creates a corresponding AAC encoder.
The frame rate of splicing device comparison SrcFile1.mp4 and SrcFile2.flv, 45 is less than 60, splicing device marks SrcFile1.mp4 as not needing to drop frame, and SrcFile2.flv as needing to drop frame. The splicing device processes SrcFile1.mp4 and SrcFile2.flv according to a splicing sequence specified by a user, when SrcFile1.mp4 without frame loss is processed, the splicing device calls an AAC audio decoder and an H265 video decoder corresponding to the SrcFile1.mp4 to decode audio and video to generate audio and video original data, then calls an AAC encoder and an H264 encoder corresponding to the DstFile. mp4 to encode the audio and video original data, assigns a timestamp to audio and video frames generated by encoding, assigns the timestamp to the audio and video frames based on a frame rate of 45 frames per second, and finally writes the audio and video frames assigned with the timestamp into the DstFile. mp 4.
The splicing device processes video files SrcFile2.flv which need frame loss, the SrcFile2.flv simultaneously contains video data in an H264 format and audio data in an AAC format, the splicing device reads one frame of complete audio data or video data each time, the read audio frame is directly decoded, encoded and encoded, a time stamp is assigned to a frame, the audio frame after the time stamp is assigned is written into DstFile.mp4, and the read video frame needs frame loss processing.
The splicing device calls an H264 video decoder corresponding to SrcFile2.flv to decode video to generate original YUV data of the video, and when one frame of video frame is decoded, the splicing device calls a frame dropping module once, a set of frame dropping logic is arranged in the frame dropping module, and the frame dropping module informs the splicing device whether the current original YUV data is reserved or dropped.
And if the frame dropping module informs the splicing device to drop the current original YUV data, the splicing device stops the current YUV data processing and reads the next frame of media data in SrcFile2.flv, which may be an audio frame or a video frame. If the frame loss module informs the splicing device to reserve the current original YUV data, the splicing device calls an H264 encoder corresponding to DstFile. mp4 to encode the current original YUV data, a timestamp is assigned to a video frame generated by encoding, the video frame assignment timestamp is based on a frame rate of 45 frames per second, and finally the video frame assigned with the timestamp is written into DstFile. mp 4.
In some preferred embodiments, the frame loss preset second rule comprises: calculating the frame rate of a video file with a larger frame rate and the frame rate of a target video file, respectively obtaining a quotient value QuotientValue and a residual value ResidualValue after quotient operation of the two, when the residual value ResidualValue is 0, continuously losing frames according to the rule that 1 frame is taken by each QuotientValue frame, when a video file needing frame loss decodes one frame of video, a splicing device calls a frame loss module, counting the called times Count inside the module, when the value obtained by the Count for the QuotientValue is 1, the frame loss module informs the splicing device of frame preservation, otherwise, informs the splicing device of frame loss, when the value of the Count is equal to the frame rate of the video file with the larger frame rate, the Count is 0, and a next frame loss period is prepared.
In some preferred embodiments, the frame loss preset second rule further comprises: when the residual value ResidualValue is not 0, continuously taking 1 frame by the frame dropping module according to each QuotientValue frame, regularly dropping 1 frame by each (QuotientValue + 1) frame, calling the frame dropping module once by the splicing device when a video file needing frame dropping decodes a frame, counting the called times Count and the reserved video frame number Num in the frame dropping module, if the sum of the reserved video frame number Num and the maximum frame number RemainNum value which can be reserved by the residual frame in the period is less than the frame rate of the target video file, informing the splicing device to reserve the current frame, otherwise, informing the splicing device to drop the current frame; when the Count value is equal to the frame rate of the video file with the larger frame rate, setting both the Count value and the Num value to be 0, and preparing for the next frame loss period to arrive.
In some preferred embodiments, the maximum number of frames RemainNum that can be retained for the remaining frames in the period is given by the following formula:
Figure 474804DEST_PATH_IMAGE001
the SrcFps represents the frame rate of a video file with a larger frame rate of a video file to be spliced, the Count represents the called times of a frame loss module, and the QuotientValue represents a quotient value obtained by performing quotient operation on the frame rate of the video file with the larger frame rate and the frame rate of a target video file.
In some preferred embodiments, the frame loss preset second rule further comprises: for the first frame video data of each period, the frame loss module directly informs the splicing device to reserve, and from the second frame in the period, the splicing device calculates the rest frames in the period and continuously adopts the rule that every QuotientValue frame takes 1 frame, and every (QuotientValue + 1) frame takes 1 frame to perform frame loss.
Specifically, as shown in fig. 3, when the residual value residalvalue is not 0, it is assumed that the frame loss module obtains the minimum frame rate of 45 frames per second for the video splicing and the video file srcfile2.flv frame rate value 60 frames per second for frame loss from the splicing device, and calculates the residual value resialvalue of the quotient between the two frame rate values and the integer part value QuotientValue of the quotient, where the resialvalue is not 0 and is equal to 15, and the QuotientValue is equal to 1.
The ResidualValue value is not 0, the frame dropping module can Count the called times Count and the number Num of the reserved video frames, the initial values of Count and Num are 0, the Count value is added with 1 every time the frame dropping module is called, and the frame dropping module informs the splicing device to reserve the current video data once every time the splicing device reserves the current video data, and the Num value is added with 1. And the frame loss module periodically counts the Count and Num values by taking the video file SrcFile2.flv frame rate value 60 needing frame loss as a period, and when the Count is equal to 60, the Count indicates that one counting period is finished, and the Count and Num values are set to be 0, and waiting for the start of the next counting period. In order to ensure that the cover of the DstFile. mp4 video file generated by splicing is the same as the cover of the first video file participating in splicing, when the Count value of the frame loss module is equal to 1, the splicing device is directly informed to keep the current frame, and simultaneously the Num value is added with 1.
When the frame dropping module is called and the Count value is not equal to 1, it is assumed that Count is equal to 2, that is, the frame dropping module is called for the 2 nd time in the period, contains the current 2 nd frame, and has 59 frames of original video data left in the period, the maximum frame number RemainNum that can be retained by the frame dropping module according to the formula (3) is 59 × 2/(2 × 1 + 1), which is equal to 39, and the frame dropping module has retained the video frame number Num equal to 1.
When the frame dropping module is called for the 2 nd time in the period, RemainNum is equal to 39, Num is equal to 1, RemainNum plus Num is equal to 40 and is smaller than the minimum frame rate 45, the frame dropping module judges that the current original video data is reserved and informs the splicing device of the judgment result.
When the frame dropping module is called for the 31 st time in the period, the RemainNum is calculated to be equal to 20 through a formula, the Num counted by the frame dropping module is equal to 25 at the moment, the RemainNum plus Num is equal to 45 and is not less than the minimum frame rate 45, the frame dropping module judges that the current original video data are dropped, and informs the judgment result to the splicing device. It should be noted that the frame loss module has a condition that the sum of RemainNum and Num is less than 45 and cannot be less than or equal to 45, for example, when the frame loss module is called for the last time in a period, Num is equal to 45, RemainNum is equal to 0, and the sum of Num and RemainNum is equal to 45, the frame loss module notifies the splicing apparatus to retain the last frame, and the Num value will become 46, that is, the splicing apparatus retains 46 frames of video data in each period instead of the desired 45 frames of video data.
As shown in fig. 4, when the residual value residalvalue is 0, it is assumed that the video frame rates of the video files srcfile3.mp4 and srcfile4.flv participating in splicing are respectively 30 frames per second and 90 frames per second, other streaming media attributes are the same as srcfile1.mp4 and srcfile2.flv in S201, and the frame dropping module respectively calculates residalvalue equal to 0 and QuotientValue equal to 3 according to the quotient formula. It can be found that 90 is exactly an integral multiple of 30, and based on the characteristic of uniform frame loss of the frame loss module, the video data of SrcFile4.flv can take 1 frame every other QuotientValue frame, i.e. take 1 frame every other 3 frames of video data. In order to ensure that a cover of a DstFile. mp4 video file generated by splicing is the same as a cover of a first video file participating in splicing, a splicing device needs to keep first frame video data of SrcFile4.flv, so a frame dropping module adopts a uniform frame dropping strategy of keeping one frame of video data every other QuotientValue frame from the first frame, namely 3 frames.
The ResidualValue value is equal to 0, the frame dropping module counts the called times, the Count initial value is 0, and the Count value is added with 1 every time the frame dropping module is called. And the frame loss module takes the video file SrcFile4.flv frame rate value 90 needing frame loss as a period, periodically counts the value, when the Count is equal to 90, the counting indicates that one counting period is finished, and the value is set to be 0, and waits for the start of the next counting period.
And adding 1 to the Count value once the frame loss module is called, and calculating the remainder of the quotient of the Count and the QuotientValue.
When the frame loss module is called for the 7 th time, the Count is equal to 7, the QuotientValue is equal to 3, and the 7 takes the remainder of the 3, namely 7% of the 3 is equal to 1, the frame loss module informs the splicing device to retain the current video data.
When the frame loss module is called for the 8 th time, the Count is equal to 8, the QuotientValue is equal to 3, the 8 takes the rest of the 3, namely 8% of the 3 is equal to 2, and the frame loss module informs the splicing device to discard the current video data.
According to the method for splicing the video files with any frame rate, the video files to be spliced are obtained, the splicing sequence of the video files to be spliced is determined, and the format and the data correctness of the video files are judged; if the format and the data of the video file to be spliced are correct, respectively acquiring the frame rate of the video file to be spliced; generating a spliced target video file according to a frame rate of a video file to be spliced and a first rule preset by a splicing device; and after the target video file is generated, updating file header information of the target file, writing the file header information into a file tail, and completing video file splicing. According to the invention, by searching the minimum video frame rate, the files participating in splicing take the minimum video frame rate as a reference, and the video frame rates of the files participating in splicing are taken as a period, frames are dropped uniformly, the frame rates of the spliced videos are uniform after splicing, the playing time lengths of the videos are unchanged, and the videos are synchronized, so that the overall performance of the video files generated by splicing is ensured to be good. The invention has simple logic and easy realization. The splicing effect is good, and the occupied space of the magnetic disk is small. High feasibility and reliability and wide application range.
It should be understood that the specific order or hierarchy of steps in the processes disclosed is an example of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not intended to be limited to the specific order or hierarchy presented.
In the foregoing detailed description, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments of the subject matter require more features than are expressly recited in each claim. Rather, as the following claims reflect, invention lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. Of course, the processor and the storage medium may reside as discrete components in a user terminal.
For a software implementation, the techniques described herein may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes may be stored in memory units and executed by processors. The memory unit may be implemented within the processor or external to the processor, in which case it can be communicatively coupled to the processor via various means as is known in the art.
What has been described above includes examples of one or more embodiments. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the aforementioned embodiments, but one of ordinary skill in the art may recognize that many further combinations and permutations of various embodiments are possible. Accordingly, the embodiments described herein are intended to embrace all such alterations, modifications and variations that fall within the scope of the appended claims. Furthermore, to the extent that the term "includes" is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term "comprising" as "comprising" is interpreted when employed as a transitional word in a claim. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or".

Claims (5)

1. A video file splicing method with any frame rate is characterized by comprising the following steps:
s100, acquiring video files to be spliced, and determining the splicing sequence of the video files to be spliced;
s200, respectively acquiring frame rates of video files to be spliced, and judging video file formats and data correctness;
s300, according to the frame rate of the video files to be spliced, generating spliced target video files according to a first rule preset by a splicing device;
specifically, in S300, the preset first rule of the splicing device includes: acquiring frame rates of video files to be spliced, namely a first frame rate and a second frame rate respectively, comparing the first frame rate with the second frame rate, acquiring the frame rate assigned to a target video file by the smaller frame rate when the first frame rate is not equal to the second frame rate, encoding and decoding the video files to be spliced with the smaller frame rate, assigning timestamps again, and writing the timestamps into the target video file; the splicing device presetting the first rule further comprises: coding and decoding the audio in the video file with the higher frame rate and reassigning a timestamp, decoding the video in the video file with the higher frame rate to generate original YUV data, performing frame dropping judgment on the generated original YUV data according to a preset second rule by a frame dropping module, judging whether the original YUV data is reserved or discarded, if the YUV data is reserved, coding the YUV data to generate a video frame, reassigning the timestamp for the video frame generated by coding, writing the audio and video frame after assigning the timestamp into a target video file, if the YUV data is discarded, terminating the processing of the frame data, and reading the next frame of audio and video data of the file;
specifically, the frame loss module performs frame loss judgment on the generated original YUV data according to a preset second rule, and the frame loss judgment comprises: calculating the frame rate of a video file with a larger frame rate and the frame rate of a target video file, respectively obtaining a quotient QuotientValue and a residual value ResidualValue after quotient calculation, when the residual value ResidualValue is 0, continuously dropping frames according to the rule that each QuotientValue frame takes 1 frame, when the video file needing to drop frames decodes one frame of video, a splicing device calls a frame dropping module once, the number of times of call is counted in the module, when the value obtained by the Count for the QuotientValue is 1, the frame dropping module informs the splicing device of keeping the frame, otherwise, the splicing device is informed of dropping the frame, when the value of the Count is equal to the frame rate of the video file with the larger frame rate, the Count is 0, and a next frame dropping period is prepared;
the frame loss module judges frame loss of the generated original YUV data according to a preset second rule, and the method further comprises the following steps:
when the residual value ResidualValue is not 0, the frame loss module continuously and alternately judges whether the current frame read by the splicing device is lost or not according to the mode that each QuotientValue frame takes 1 frame and each (QuotientValue + 1) frame takes 1 frame;
after a video file needing frame loss is decoded into a frame of video, the splicing device calls a frame loss module, the frame loss module internally counts the called times Count and the number Num of the reserved video frames, if the sum of the number Num of the reserved video frames and the number RemainNum of the maximum number RemainNum which can be reserved by the remaining frames in the period is less than the frame rate of the target video file, the splicing device is informed to reserve the current frame, otherwise, the splicing device is informed to discard the current frame; when the Count value is equal to the frame rate of the video file with the larger frame rate, setting both the Count value and the Num value as 0, and preparing for the coming of the next frame loss period;
the maximum frame number RemainNum value formula that the remaining frames can keep in the period is as follows:
Figure DEST_PATH_IMAGE001
the method comprises the steps that SrcFps represents the frame rate of a video file with a larger frame rate of a video file to be spliced, Count represents the called times of a frame loss module, and QuotientValue represents a quotient value obtained after quotient operation is carried out on the frame rate of the video file with the larger frame rate and the frame rate of a target video file;
and S400, after the target video file is generated, updating file header information of the target file, writing file tail, and splicing the video file.
2. The method for video splicing at any frame rate according to claim 1, wherein in S100, the format of the video file to be spliced at least comprises: one of MP4, FLV, WMV, AVI, and MPEG.
3. The method for video splicing at any frame rate according to claim 1, wherein in S200, the method for determining the video file format and the data correctness comprises: the method comprises the steps of obtaining a video file to be spliced, obtaining audio and video stream information and a video frame rate of the video file to be spliced, creating a corresponding coder-decoder, and judging that the format and the data of the current video file to be spliced are correct if the corresponding coder-decoder can be created.
4. The method of splicing videos with arbitrary frame rates as claimed in claim 1, wherein the step S300 of presetting the first rule by the splicing apparatus further comprises: the method comprises the steps of obtaining frame rates of video files to be spliced, wherein the frame rates are respectively a first frame rate and a second frame rate, comparing the first frame rate with the second frame rate, assigning the current frame rate to the frame rate of a target video file when the first frame rate is equal to the second frame rate, starting to encode and decode audio and video data in the video files to be spliced, assigning a timestamp for an audio and video generated by encoding, writing the audio and video data into the target video file, and assigning the timestamp for the video frame again to depend on the frame rate of the target video file.
5. The method of video stitching at any frame rate as recited in claim 1, wherein the default second rule for frame loss further comprises: for the first frame video data of each period, the frame loss module directly informs the splicing device to keep, starting from the second frame in the period, the splicing device reads the remaining frames in the period frame by frame from the video file needing frame loss, and each frame in the remaining frames is continuously and alternately judged to be lost or kept by the frame loss module in a mode that each QuotientValue frame takes 1 frame and each (QuotientValue + 1) frame takes 1 frame.
CN202011187951.6A 2020-10-30 2020-10-30 Video file splicing method with any frame rate Active CN112019880B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011187951.6A CN112019880B (en) 2020-10-30 2020-10-30 Video file splicing method with any frame rate

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011187951.6A CN112019880B (en) 2020-10-30 2020-10-30 Video file splicing method with any frame rate

Publications (2)

Publication Number Publication Date
CN112019880A CN112019880A (en) 2020-12-01
CN112019880B true CN112019880B (en) 2021-02-12

Family

ID=73527727

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011187951.6A Active CN112019880B (en) 2020-10-30 2020-10-30 Video file splicing method with any frame rate

Country Status (1)

Country Link
CN (1) CN112019880B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113612922A (en) * 2021-07-29 2021-11-05 重庆赛迪奇智人工智能科技有限公司 Video processing method and device, electronic equipment and computer readable storage medium
CN114390314B (en) * 2021-12-30 2024-06-18 咪咕文化科技有限公司 Variable frame rate audio/video processing method, equipment and storage medium
CN114390343B (en) * 2021-12-31 2024-06-14 深圳市闪剪智能科技有限公司 Multi-video playing method and related equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101217657A (en) * 2008-01-08 2008-07-09 北京中星微电子有限公司 A frame rate control method and device
US8903223B1 (en) * 2010-06-29 2014-12-02 Samsung Electronics Co., Ltd. Video driver over a network
CN104363508A (en) * 2014-11-21 2015-02-18 浙江宇视科技有限公司 Image stitching method and device for preventing video rollback
CN105704508A (en) * 2016-01-06 2016-06-22 无锡天脉聚源传媒科技有限公司 Video merging method and device
CN106658037A (en) * 2016-12-13 2017-05-10 天脉聚源(北京)传媒科技有限公司 Live video method and apparatus of multiple video streams

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101448162B (en) * 2001-12-17 2013-01-02 微软公司 Method for processing video image
US20120169883A1 (en) * 2010-12-31 2012-07-05 Avermedia Information, Inc. Multi-stream video system, video monitoring device and multi-stream video transmission method
CN111355989B (en) * 2018-12-21 2023-08-08 深圳市中兴微电子技术有限公司 Frame rate control method and related equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101217657A (en) * 2008-01-08 2008-07-09 北京中星微电子有限公司 A frame rate control method and device
US8903223B1 (en) * 2010-06-29 2014-12-02 Samsung Electronics Co., Ltd. Video driver over a network
CN104363508A (en) * 2014-11-21 2015-02-18 浙江宇视科技有限公司 Image stitching method and device for preventing video rollback
CN105704508A (en) * 2016-01-06 2016-06-22 无锡天脉聚源传媒科技有限公司 Video merging method and device
CN106658037A (en) * 2016-12-13 2017-05-10 天脉聚源(北京)传媒科技有限公司 Live video method and apparatus of multiple video streams

Also Published As

Publication number Publication date
CN112019880A (en) 2020-12-01

Similar Documents

Publication Publication Date Title
CN112019880B (en) Video file splicing method with any frame rate
US20080063387A1 (en) Data Processor
WO2020215453A1 (en) Video recording method and system
CN106131550B (en) Method and device for playing multimedia file
EP2114076B1 (en) Apparatus and method for composing scenes using rich media contents
US20170180746A1 (en) Video transcoding method and electronic apparatus
CN112584087B (en) Video conference recording method, electronic device and storage medium
CN112073810B (en) Multi-layout cloud conference recording method and system and readable storage medium
CN101206899A (en) Method of and apparatus for encoding/decoding multimedia data with preview function
CN111918121B (en) Accurate editing method for streaming media file
CN114363648A (en) Method, equipment and storage medium for audio and video alignment in mixed flow process of live broadcast system
KR101199166B1 (en) A apparatus generating interpolated frames
JP2001157204A (en) Moving picture decoding method and device
US9338468B2 (en) Latency aware priority based multi-channel decoding
CN111741376B (en) Method for synchronizing audio and video lip sounds of multimedia file splicing
CN113873176B (en) Media file merging method and device
CN111279694A (en) GDR code stream encoding method, terminal device and machine readable storage medium
CN113423011B (en) Transcoding method for video file with any format
CN113965786A (en) Method for accurately controlling video output and playing
CN111726683B (en) Media playing method and device, electronic equipment and storage medium
TWI439137B (en) A method and apparatus for restructuring a group of pictures to provide for random access into the group of pictures
US7423652B2 (en) Apparatus and method for digital video decoding
CN113261283B (en) Video processing method, apparatus and computer readable storage medium
CN112738571B (en) Method and device for determining streaming media parameters
WO2021114305A1 (en) Video processing method and apparatus, and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant