CN116233411A - Method, device, equipment and computer storage medium for audio and video synchronous test - Google Patents
Method, device, equipment and computer storage medium for audio and video synchronous test Download PDFInfo
- Publication number
- CN116233411A CN116233411A CN202310001594.7A CN202310001594A CN116233411A CN 116233411 A CN116233411 A CN 116233411A CN 202310001594 A CN202310001594 A CN 202310001594A CN 116233411 A CN116233411 A CN 116233411A
- Authority
- CN
- China
- Prior art keywords
- video
- audio
- preset
- frame
- frames
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
- H04N21/43072—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
Abstract
The embodiment of the invention relates to the technical field of multimedia and discloses a method for synchronously testing audio and video, which comprises the following steps: inputting the first video data into a video playing system to be tested to obtain second video data broadcasted by the video playing system to be tested; the first video data comprises a plurality of first video frames and a plurality of first audio frames, the second video data comprises a plurality of second video frames and a plurality of second audio frames, and the first video frames and the first audio frames with consistent playing time respectively carry preset marks; determining a first second audio frame carrying the preset mark in the second video data; and determining the audio-video synchronization time delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark. The embodiment of the invention realizes the audio and video synchronization test in the live broadcast field system, and obtains more accurate audio and video time delay.
Description
Technical Field
The embodiment of the invention relates to the technical field of multimedia, in particular to a method, a device, equipment and a computer storage medium for audio and video synchronous test.
Background
At present, an audio and video synchronization test plays a standard audio and video synchronization signal through auxiliary equipment, and determines whether audio and video are synchronized by taking a voltage delay difference value of a video signal and an audio signal as an audio and video synchronization parameter value; or performing audio and video synchronization test in a one-to-one correspondence manner of the video frames and the audio frames.
In carrying out embodiments of the present invention, the inventors found that: in the existing audio and video synchronization test method, by using a scheme of voltage delay difference values, instead of using delay difference values of audio signals and delay difference values of video signals and video signals as audio and video synchronization parameter values, the calculated voltage delay difference values are not reasonable enough, and by adopting a scheme of one-to-one correspondence of video frames and audio frames, the problem that the duration of video frames is inconsistent with the duration of audio frames is ignored, and the audio and video synchronization test is not performed according to the characteristics of video data, so that the obtained audio and video synchronization test results are not accurate.
Disclosure of Invention
In view of the above problems, embodiments of the present invention provide a method, an apparatus, a device, and a computer storage medium for audio and video synchronization test, which are used to solve the problem in the prior art that the audio and video synchronization test result is not accurate enough.
According to an aspect of an embodiment of the present invention, there is provided a method for audio/video synchronization test, the method including:
inputting the first video data into a video playing system to be tested to obtain second video data broadcasted by the video playing system to be tested; the first video data comprises a plurality of first video frames and a plurality of first audio frames, the second video data comprises a plurality of second video frames and a plurality of second audio frames, and the first video frames and the first audio frames with consistent playing time respectively carry preset marks;
determining a first second audio frame carrying the preset mark in the second video data;
and determining the audio-video synchronization time delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark.
In an alternative manner, each of the first video frames is marked with a frame number; the determining the audio-video synchronization delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark includes:
Determining a preset playing time of a first second audio frame carrying the preset mark in the second video data according to the frame sequence number of the first second video frame in the second video data;
determining the actual playing time of the first second audio frame carrying the preset mark;
and determining the audio and video synchronization time delay of the video playing system to be tested according to the preset playing time and the actual playing time.
In an optional manner, the determining, according to the frame number of the first second video frame in the second video data, the preset playing time of the first second audio frame carrying the preset mark in the second video data includes:
acquiring a frame sequence number of a second video frame carrying the preset mark corresponding to a first second audio frame carrying the preset mark in the second video data;
and determining the preset playing time of the first second audio frame carrying the preset mark in the second video data according to the frame number of the first second video frame in the second video data and the frame number of the second video frame carrying the preset mark corresponding to the first second audio frame carrying the preset mark in the second video data.
In an optional manner, before the first video data is input to the video playing system to be tested to obtain the second video data played by the video playing system to be tested, the method includes:
determining a mark period of the preset mark according to preset video parameters and preset audio parameters;
and marking the preset marks on the first video frame and the first audio frame with consistent playing time according to the marking period so as to generate the first video data.
In an optional manner, the determining the marking period of the preset mark according to the preset video parameter and the preset audio parameter includes:
calculating the least common multiple of the duration of the audio frame and the duration of the video frame according to the preset video parameters and the preset audio parameters;
and determining the marking period of the preset marking according to the least common multiple.
In an optional manner, the preset mark carried by the first audio frame is a volume mark; the marking the preset mark on the first video frame and the first audio frame with consistent playing time according to the marking period to generate the first video data comprises the following steps:
the volume of each first audio frame is marked according to the preset period, and a plurality of first audio frames carrying the preset marks are obtained;
Adding the preset marks to the first video frames with consistent playing time according to the preset period according to the plurality of first audio frames with the preset marks, so as to obtain a plurality of first video frames with the preset marks;
and generating the first video data according to the plurality of first audio frames carrying the preset marks and the plurality of first video frames carrying the preset marks.
In an optional manner, the marking the volume of each first audio frame according to the preset period to obtain a plurality of first audio frames carrying the preset mark includes:
setting the volume of the nth target audio frame to Ndb and the volume of the remaining audio frames to 0db; the target audio frame is determined from a plurality of first audio frames according to the preset period, and the residual audio frames are first audio frames except the target audio frame in the plurality of first audio frames;
the adding the preset mark to the first video frames with consistent playing time according to the preset period according to the plurality of first audio frames with the preset mark, to obtain a plurality of first video frames with the preset mark, includes:
Adding a ringing mark on the picture of the Nth target video frame; the target video frames are determined from a plurality of first video frames according to the preset period; the nth target video frame corresponds to the nth target audio frame, and N is an integer greater than 0.
According to another aspect of the embodiment of the present invention, there is provided an audio/video synchronization test apparatus, including:
the input module is used for inputting the first video data to the video playing system to be tested to obtain second video data broadcasted by the video playing system to be tested; the first video data comprises a plurality of first video frames and a plurality of first audio frames, the second video data comprises a plurality of second video frames and a plurality of second audio frames, and the first video frames and the first audio frames with consistent playing time respectively carry preset marks;
a first determining module, configured to determine a first second audio frame carrying the preset mark in the second video data;
and the second determining module is used for determining the audio-video synchronization time delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark.
According to another aspect of the embodiment of the present invention, there is provided an audio/video synchronization test apparatus, including: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation of the audio and video synchronization test method.
According to still another aspect of the embodiments of the present invention, there is provided a computer readable storage medium having stored therein at least one executable instruction that, when executed on an audio and video synchronization test apparatus, causes the audio and video synchronization test apparatus to perform the operations of the audio and video synchronization test method.
According to the embodiment of the invention, the first video data is input to the video playing system to be tested, so that the second video data played by the video playing system to be tested is obtained; determining a first second audio frame carrying the preset mark in the second video data; and determining the audio-video synchronization time delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark. The time delay between the audio and video frames can be obtained by comparing the relation of the video marks before and after the test of the video through the preset marks carried on the first video frame and the first audio frame respectively, wherein the playing time of the first video data is consistent, so that an accurate audio and video synchronous test result is obtained.
The foregoing description is only an overview of the technical solutions of the embodiments of the present invention, and may be implemented according to the content of the specification, so that the technical means of the embodiments of the present invention can be more clearly understood, and the following specific embodiments of the present invention are given for clarity and understanding.
Drawings
The drawings are only for purposes of illustrating embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
fig. 1 shows a flow chart of an audio and video synchronization test method according to an embodiment of the present invention;
fig. 2 is a schematic diagram illustrating adding a preset mark to a first audio frame in an audio-video synchronization test method according to an embodiment of the present invention;
fig. 3 is a schematic diagram illustrating adding a preset mark to a first video frame in the audio and video synchronization test method according to the embodiment of the present invention;
fig. 4 shows a schematic structural diagram of an audio and video synchronization test device according to an embodiment of the present invention;
fig. 5 shows a schematic structural diagram of an audio and video synchronization test device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Fig. 1 shows a flowchart of an audio and video synchronization test method provided by an embodiment of the present invention, where the method is performed by an audio and video synchronization test device. The computer device may be a server device, desktop computer, tablet computer, intelligent terminal device, etc. As shown in fig. 1, the method comprises the steps of:
Step 110: and inputting the first video data into a video playing system to be tested to obtain second video data broadcasted by the video playing system to be tested.
The first video data comprises a plurality of first video frames and a plurality of first audio frames, the second video data comprises a plurality of second video frames and a plurality of second audio frames, and the first video frames and the first audio frames with consistent playing time respectively carry preset marks. The first video data refers to video data used for testing a video playing system to be tested.
In the embodiment of the present invention, the first video data needs to be pre-constructed, including:
step 001: and determining the marking period of the preset mark according to the preset video parameter and the preset audio parameter.
In the embodiment of the invention, the time delay test of the video playing system to be tested is realized by adding the preset marks on the first video frame and the first audio frame in the first video data. The preset mark marks the first video frame and the first audio frame in the first video data according to a preset mark period. Specifically, according to preset video parameters and preset audio parameters, calculating the least common multiple of the audio frame duration and the video frame duration; and determining the marking period of the preset marking according to the least common multiple. In the embodiment of the invention, the preset video parameters and the preset audio parameters can be determined according to the video parameters and the audio parameters of the output video of the video playing system to be tested; the settings may also be based on user input. And then, according to the video parameters and the audio parameters of the output video of the video playing system to be tested, calculating to obtain the least common multiple of the audio frame duration and the video frame duration. Wherein, the period duration of the preset period can be determined by determining the multiple of the least common multiple closest to 1 second, and can be expressed as: maxf (n), wherein f (n) = minimum common multiple is less than or equal to 1 second, and n is an integer. The common video frame rates in video are 25fps, 30fps, 60fps, etc., which means that there are 25 frames, 30 frames, or 60 frames of pictures every 1 second, that is, there is one video frame to render every 40ms, 100/3ms, 50/3ms, respectively. Thus, the time stamps for each video frame will be accumulated at 40ms, 100/3ms, 50/3ms, respectively. Common audio codes have mp3 or aac, and each audio frame contains a sample number 1152 or 1024, respectively. If the sampling rate is 48000Hz, 1 audio frame needs to be rendered every 1152×1000/48000ms or 1024×1000/48000 ms. The time stamp for each audio frame will be accumulated at 24ms or 64/3 ms. Therefore, if the sampling rate is 44100Hz, the time stamp accumulation duration of each audio frame can be obtained according to the calculation method. In the embodiment of the invention, the streaming video frame rate of the video playing system to be tested is assumed to be 25fps, the audio coding format is aac, and the audio sampling rate is assumed to be 48000Hz. According to the video parameters and the audio parameters of the output video of the video playing system to be tested, the least common multiple of the audio frame duration and the video frame duration is calculated to be 40ms and the least common multiple of 64/3ms is calculated to be 320ms. The marking period of the preset mark may be an integer multiple of the least common multiple. For example, an integer multiple of 320ms, such as 960ms (approaching 1 second), may be arbitrarily taken as a period duration adopted by the preset marking, and the first audio frame and the first video frame with consistent playing time are marked according to the selected period duration.
Step 002: and marking the preset marks on the first video frame and the first audio frame with consistent playing time according to the marking period so as to generate the first video data.
Wherein the preset mark of the first audio frame is a volume mark. When a preset mark is added to the first audio frames, the volume of each first audio frame is marked according to the preset period, and a plurality of first audio frames carrying the preset mark are obtained. The preset marking of the first audio frames may be to process the audio values of each first audio frame into audio values of different audio sizes according to a preset period. For example, the audio values of the respective audio frames may be sequentially increased or decreased according to a preset period, and may be specifically generated by the video generating apparatus. In one embodiment of the present invention, specifically, the volume of the nth target audio frame is set to Ndb, and the volume of the remaining audio frames is set to 0db; the target audio frame is determined from a plurality of first audio frames according to the preset period, and the residual audio frames are first audio frames except the target audio frame in the plurality of first audio frames.
After marking the first audio frames, adding the preset marks to the first video frames with consistent playing time according to the preset period according to the first audio frames carrying the preset marks, so as to obtain the first video frames carrying the preset marks. Wherein the preset mark of the first video frame is a ringing mark. Specifically, a ringing mark is added on the picture of the Nth target video frame; the target video frames are determined from a plurality of first video frames according to the preset period; the nth target video frame corresponds to the nth target audio frame, and N is an integer greater than 0. In the embodiment of the invention, in order to facilitate the subsequent identification of each video frame, the frame sequence number of the first video frame is also marked in each first video frame. In the above manner, the first video data is generated.
For example, referring to fig. 2 and 3, for a video playing system to be tested with an streaming video frame rate of 25fps, an audio encoding format aac, and an audio sampling rate of 48000Hz, when a preset period is 960ms, a volume dB of a 1 st first audio frame is set to be 1dB in a time of 0-64/3ms when a preset mark is generated to the first audio frame and the first video frame. The audio walks through 44 more frames in the 64/3ms-960ms time. dB of the 44 first audio frames is set to 0dB. The volume dB of the 46 th first audio frame is set to 2dB for a time of 960ms- (960+64/3) ms. In the 91 st frame thereafter, the 136 th frame … is set to 3db,4db, etc., respectively. When 100dB is reached, reset from 1dB, making a loop from 1dB to 100 dB. The volume of the other first audio frames is set to 0dB.
According to the preset period 960ms, the number of frames of the first video frame in one preset period can be obtained as 24 frames, 24 frames means 24 video pictures. The video frame may be marked by performing a cycle of image marking according to a preset period, so that the video frame in each preset period is cycled, for example, the 1 st frame, that is, 0-40ms, is set as the 1 st frame video frame P1, the video is required to play 23 frames in 40-960ms, and the remaining 23 frames are respectively set as P2, P3 … … P24. The video picture of frame 1, i.e., frame 25 960-1000ms, at the beginning of the second period is also set to P1. In the embodiment of the invention, besides processing the video frames of the first video frame, the embodiment of the invention marks the frame number corresponding to the video frame at the right lower corner of each video frame. That is, the image and frame number of the video frame picture are marked from the first frame of the first preset period, the video picture generated in the 1 st frame, i.e., the 25 th frame 960-1000ms, of the second preset period is also set to P1, and the lower right corner is marked with 25 (indicating that the sample video frame number is 25), so that the sample video frames in each period are marked in turn. In the embodiment of the present invention, the image mark of each video frame may be set as a plurality of regularly arranged squares on the video frame, where the squares on each sample video frame sequentially and cyclically lighten according to a preset period, for example, in a first preset period, the squares at the first position on the video frame of the 1 st sample video frame lighten, the squares at the 2 nd position on the video frame of the 2 nd sample video frame lighten, … …, and the squares at the 24 th position on the video frame of the 24 th sample video frame lighten; the first square on the video frame of the 1 st sample video frame (i.e., the 25 th frame of the sample video) within the second preset period is illuminated, and the second square on the video frame of the 2 nd sample video frame (i.e., the 26 th frame of the sample video) is illuminated … …. In another embodiment of the present invention, a small square may be disposed in the middle of the video frame of the first video frame, where the number is identified in the small square, and the number is determined according to the number of the video frame, for example, the small square identification number of the video frame P1 is 1, the small square identification number of the video frame P2 is 2 … …, and the small square identification number of the video frame P24 is 24. In order to facilitate the identification of the period, a ringing flag is set in the video picture P1, and when the ringing flag is combined with audio data, a ringing sound is generated to prompt the start of a new period. Wherein, in the period of 0-40ms, the frame picture P1 of the first video frame of the first frame is set to be P2, P3 … P24 respectively, and in the period of 960ms-1000ms, the 25 th frame picture is basically the same as P1 (the lower right corner label of each picture is the frame of the number of frames). Wherein, each of the pictures from P1 to P24 has 24 small squares, the figures are marked in the small squares, the P1 picture small square 1 is lightened, the P2 picture small square 2 is lightened, and the P3 picture small square 3 is lightened …. The P1 picture is added with a ringing mark than other pictures. In the 49 th frame thereafter, the 73 rd frame … and the like are set to P1, and the other frames are set to P2, P3 … P24 in this order. The first video data will generate a sound bell in frame 1, frame 25 and frame …, and there will be a bell mark in the frame. Only small blocks are lit up in other frames. After the first video data is obtained, the first video data marked with regularity can be used for testing the audio and video synchronization of the video playing system to be tested. The video system to be tested can be a live broadcast system and the like. And inputting the first video data into a video playing system to be tested to obtain second video data broadcasted by the video playing system to be tested, wherein the second video data comprises a plurality of second video frames and a plurality of second audio frames. In the same embodiment of the invention, the second video data can be streamed after the first video data is input into the video playing system to be tested by recording with ffmpeg (a set of open source computer programs which can be used for recording, converting digital audio and video and converting the digital audio and video into streams).
Step 120: and determining a first second audio frame carrying the preset mark in the second video data.
In the embodiment of the invention, a plurality of video frames and audio data in a plurality of second video data are separated through an audio/video extraction technology. The audio data may be in wav format. And extracting the audio frame from the audio data to obtain a first second audio frame carrying the preset mark.
And obtaining a first second video frame in the second video data according to the frame picture in the second video frame and the frame sequence number of the lower right corner of the frame picture. A start time and an end time for recording the second video data may be calculated. Wherein each second video frame in the second video data is in one-to-one correspondence with each first video frame in the first video data.
Step 130: and determining the audio-video synchronization time delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark.
Because the corresponding relation exists between each first video frame in the first video data and the preset mark in the first audio frame, if the video playing system to be tested can cause the asynchronism of the audio and the video, the corresponding relation between each second video frame in the output second video data and the second audio frame can be changed. Therefore, according to the first second video frame in the second video data and the first second audio frame carrying the preset mark, determining the audio-video synchronization time delay of the video playing system to be tested.
The audio and video synchronization delay process of the video playing system to be tested comprises the following steps:
301: and determining the preset playing time of the first second audio frame carrying the preset mark in the second video data according to the frame sequence number of the first second video frame in the second video data.
Specifically, a frame sequence number of a second video frame carrying the preset mark corresponding to a first second audio frame carrying the preset mark in the second video data is obtained. And determining the preset playing time of the first second audio frame carrying the preset mark in the second video data according to the frame number of the first second video frame in the second video data and the frame number of the second video frame carrying the preset mark corresponding to the first second audio frame carrying the preset mark in the second video data. The method and the device for determining the actual start time of the first second video frame in the second video data are characterized in that the start time of the second video data is not necessarily the time of the first second video frame due to the problems of time delay and the like of the video playing system to be tested; according to the starting time of the first video frame and the second video frame, the preset starting time of the first video frame in the first video data is obtained; calculating the play offset according to the actual start time and the preset start time; and determining the preset playing time of the first audio frame carrying the preset mark in the second video data according to the playing offset and the preset starting time of the first audio frame carrying the preset mark in the first video data. Assuming that the number of frames in the lower right corner is found to be from 65 to 320 (the number is arbitrary) in the pictures separated from the recorded video, the time period for recording the video is 2560ms to 12800ms. It is known that the preset generates 3dB, 4dB, 5dB … 12dB, 13dB of audio in the time periods of 2880ms- (2880+64/3) ms, 3840ms- (3840+64/3) ms, 4800ms- (4800+64/3) ms … 11520ms- (11520+64/3) ms, 12480ms- (12480+64/3) ms respectively. Since video starts from 2560ms, 3dB, 4dB, 5dB … dB, 13dB audio presets should be at 320ms- (320+64/3) ms, 1280ms- (1280+64/3) ms, 2240ms- (2240+64/3) ms … 8960ms- (8960+64/3) ms, 9920ms- (9920+64/3) ms, with 2560ms offsets removed.
Step 302: and determining the actual playing time of the first second audio frame carrying the preset mark. Specifically, the audio generation time and the volume of the audio waveform corresponding to the audio data are checked through Adobe audio, and the audio frame generation time with the volume greater than or equal to 1dB is found, so that the first second audio frame carrying the preset mark is determined to be the actual playing time.
Step 303: and determining the audio and video synchronization time delay of the video playing system to be tested according to the preset playing time and the actual playing time.
Assuming that the number of frames in the lower right corner is found to be 320 (the number is arbitrary) from 65 in the picture separated from the recorded video, the time period of the recorded video is 2560ms to 12800ms. It is known that the preset generates 3dB, 4dB, 5dB … 12dB, 13dB of audio in the time periods of 2880ms- (2880+64/3) ms, 3840ms- (3840+64/3) ms, 4800ms- (4800+64/3) ms … 11520ms- (11520+64/3) ms, 12480ms- (12480+64/3) ms respectively. Since the video starts from 2560ms, the 2560ms offset is removed. The 3dB, 4dB, 5dB … dB, 13dB audio presets should look at the audio waveform of the streaming audio track b.wav with Adobe audio at 320ms- (320+64/3) ms, 1280ms- (1280+64/3) ms, 2240ms- (2240+64/3) ms … 8960ms- (8960+64/3) ms, 9920ms- (9920+64/3) ms, looking at the actual generation time of 3dB, 4dB, 5dB … dB, 13dB audio. The time difference between the audio frame generation time with the actual volume being more than or equal to 1dB and the audio frame generation time with the preset volume being more than or equal to 1dB is the audio-video synchronization time delay.
According to the embodiment of the invention, the first video data is input to the video playing system to be tested, so that the second video data played by the video playing system to be tested is obtained; determining a first second audio frame carrying the preset mark in the second video data; and determining the audio-video synchronization time delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark. The time delay between the audio and video frames can be obtained by comparing the relation of the video marks before and after the test of the video through the preset marks carried on the first video frame and the first audio frame respectively, wherein the playing time of the first video data is consistent, so that an accurate audio and video synchronous test result is obtained.
Fig. 4 shows a schematic structural diagram of an audio and video synchronization test device according to an embodiment of the present invention. As shown in fig. 4, the apparatus 200 includes:
the input module 210 is configured to input the first video data to a video playing system to be tested, so as to obtain second video data played by the video playing system to be tested; the first video data comprises a plurality of first video frames and a plurality of first audio frames, the second video data comprises a plurality of second video frames and a plurality of second audio frames, and the first video frames and the first audio frames with consistent playing time respectively carry preset marks;
A first determining module 220, configured to determine, in the second video data, a first second audio frame carrying the preset mark;
a second determining module 230, configured to determine an audio-video synchronization delay of the video playing system to be tested according to a first second video frame in the second video data and the first second audio frame carrying the preset mark.
In an alternative manner, each of the first video frames is marked with a frame number; the determining the audio-video synchronization delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark includes:
determining a preset playing time of a first second audio frame carrying the preset mark in the second video data according to the frame sequence number of the first second video frame in the second video data;
determining the actual playing time of the first second audio frame carrying the preset mark;
and determining the audio and video synchronization time delay of the video playing system to be tested according to the preset playing time and the actual playing time.
In an optional manner, the determining, according to the frame number of the first second video frame in the second video data, the preset playing time of the first second audio frame carrying the preset mark in the second video data includes:
acquiring a frame sequence number of a second video frame carrying the preset mark corresponding to a first second audio frame carrying the preset mark in the second video data;
and determining the preset playing time of the first second audio frame carrying the preset mark in the second video data according to the frame number of the first second video frame in the second video data and the frame number of the second video frame carrying the preset mark corresponding to the first second audio frame carrying the preset mark in the second video data.
In an optional manner, before the first video data is input to the video playing system to be tested to obtain the second video data played by the video playing system to be tested, the method includes:
determining a mark period of the preset mark according to preset video parameters and preset audio parameters;
and marking the preset marks on the first video frame and the first audio frame with consistent playing time according to the marking period so as to generate the first video data.
In an optional manner, the determining the marking period of the preset mark according to the preset video parameter and the preset audio parameter includes:
calculating the least common multiple of the duration of the audio frame and the duration of the video frame according to the preset video parameters and the preset audio parameters;
and determining the marking period of the preset marking according to the least common multiple.
In an optional manner, the preset mark carried by the first audio frame is a volume mark; the marking the preset mark on the first video frame and the first audio frame with consistent playing time according to the marking period to generate the first video data comprises the following steps:
the volume of each first audio frame is marked according to the preset period, and a plurality of first audio frames carrying the preset marks are obtained;
adding the preset marks to the first video frames with consistent playing time according to the preset period according to the plurality of first audio frames with the preset marks, so as to obtain a plurality of first video frames with the preset marks;
and generating the first video data according to the plurality of first audio frames carrying the preset marks and the plurality of first video frames carrying the preset marks.
In an optional manner, the marking the volume of each first audio frame according to the preset period to obtain a plurality of first audio frames carrying the preset mark includes:
setting the volume of the nth target audio frame to Ndb and the volume of the remaining audio frames to 0db; the target audio frame is determined from a plurality of first audio frames according to the preset period, and the residual audio frames are first audio frames except the target audio frame in the plurality of first audio frames;
the adding the preset mark to the first video frames with consistent playing time according to the preset period according to the plurality of first audio frames with the preset mark, to obtain a plurality of first video frames with the preset mark, includes: adding a ringing mark on the picture of the Nth target video frame; the target video frames are determined from a plurality of first video frames according to the preset period; the nth target video frame corresponds to the nth target audio frame, and N is an integer greater than 0.
Fig. 5 shows a schematic structural diagram of an audio and video synchronization test device according to an embodiment of the present invention, and the specific embodiment of the present invention does not limit the specific implementation of the audio and video synchronization test device.
As shown in fig. 5, the audio and video synchronization test apparatus may include: a processor (processor) 302, a communication interface (Communications Interface) 304, a memory (memory) 306, and a communication bus 308.
Wherein: processor 302, communication interface 304, and memory 306 perform communication with each other via communication bus 308. A communication interface 304 for communicating with network elements of other devices, such as clients or other servers. The processor 302 is configured to execute the program 310, and may specifically perform the relevant steps in the above-described embodiment of the audio/video synchronization test method.
In particular, program 310 may include program code comprising computer-executable instructions.
The processor 302 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in the audio/video synchronization test device may be the same type of processor, such as one or more CPUs; but may also be different types of processors such as one or more CPUs and one or more ASICs.
Memory 306 for storing programs 310. Memory 306 may comprise high-speed RAM memory or may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
inputting the first video data into a video playing system to be tested to obtain second video data broadcasted by the video playing system to be tested; the first video data comprises a plurality of first video frames and a plurality of first audio frames, the second video data comprises a plurality of second video frames and a plurality of second audio frames, and the first video frames and the first audio frames with consistent playing time respectively carry preset marks;
determining a first second audio frame carrying the preset mark in the second video data;
and determining the audio-video synchronization time delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark.
In an alternative manner, each of the first video frames is marked with a frame number; the determining the audio-video synchronization delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark includes:
Determining a preset playing time of a first second audio frame carrying the preset mark in the second video data according to the frame sequence number of the first second video frame in the second video data;
determining the actual playing time of the first second audio frame carrying the preset mark;
and determining the audio and video synchronization time delay of the video playing system to be tested according to the preset playing time and the actual playing time.
In an optional manner, the determining, according to the frame number of the first second video frame in the second video data, the preset playing time of the first second audio frame carrying the preset mark in the second video data includes:
acquiring a frame sequence number of a second video frame carrying the preset mark corresponding to a first second audio frame carrying the preset mark in the second video data;
and determining the preset playing time of the first second audio frame carrying the preset mark in the second video data according to the frame number of the first second video frame in the second video data and the frame number of the second video frame carrying the preset mark corresponding to the first second audio frame carrying the preset mark in the second video data.
In an optional manner, before the first video data is input to the video playing system to be tested to obtain the second video data played by the video playing system to be tested, the method includes:
determining a mark period of the preset mark according to preset video parameters and preset audio parameters;
and marking the preset marks on the first video frame and the first audio frame with consistent playing time according to the marking period so as to generate the first video data.
In an optional manner, the determining the marking period of the preset mark according to the preset video parameter and the preset audio parameter includes:
calculating the least common multiple of the duration of the audio frame and the duration of the video frame according to the preset video parameters and the preset audio parameters;
and determining the marking period of the preset marking according to the least common multiple.
In an optional manner, the preset mark carried by the first audio frame is a volume mark; the marking the preset mark on the first video frame and the first audio frame with consistent playing time according to the marking period to generate the first video data comprises the following steps:
the volume of each first audio frame is marked according to the preset period, and a plurality of first audio frames carrying the preset marks are obtained;
Adding the preset marks to the first video frames with consistent playing time according to the preset period according to the plurality of first audio frames with the preset marks, so as to obtain a plurality of first video frames with the preset marks;
and generating the first video data according to the plurality of first audio frames carrying the preset marks and the plurality of first video frames carrying the preset marks.
In an optional manner, the marking the volume of each first audio frame according to the preset period to obtain a plurality of first audio frames carrying the preset mark includes:
setting the volume of the nth target audio frame to Ndb and the volume of the remaining audio frames to 0db; the target audio frame is determined from a plurality of first audio frames according to the preset period, and the residual audio frames are first audio frames except the target audio frame in the plurality of first audio frames;
the adding the preset mark to the first video frames with consistent playing time according to the preset period according to the plurality of first audio frames with the preset mark, to obtain a plurality of first video frames with the preset mark, includes:
Adding a ringing mark on the picture of the Nth target video frame; the target video frames are determined from a plurality of first video frames according to the preset period; the nth target video frame corresponds to the nth target audio frame, and N is an integer greater than 0.
According to still another aspect of the embodiments of the present invention, there is provided a computer readable storage medium having stored therein at least one executable instruction that, when executed on an audio and video synchronization test apparatus, causes the audio and video synchronization test apparatus to perform the operations of the audio and video synchronization test method.
The computer readable storage medium is invoked by the processor to cause the audio video synchronization test device to:
inputting the first video data into a video playing system to be tested to obtain second video data broadcasted by the video playing system to be tested; the first video data comprises a plurality of first video frames and a plurality of first audio frames, the second video data comprises a plurality of second video frames and a plurality of second audio frames, and the first video frames and the first audio frames with consistent playing time respectively carry preset marks;
determining a first second audio frame carrying the preset mark in the second video data;
And determining the audio-video synchronization time delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark.
In an alternative manner, each of the first video frames is marked with a frame number; the determining the audio-video synchronization delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark includes:
determining a preset playing time of a first second audio frame carrying the preset mark in the second video data according to the frame sequence number of the first second video frame in the second video data;
determining the actual playing time of the first second audio frame carrying the preset mark;
and determining the audio and video synchronization time delay of the video playing system to be tested according to the preset playing time and the actual playing time.
In an optional manner, the determining, according to the frame number of the first second video frame in the second video data, the preset playing time of the first second audio frame carrying the preset mark in the second video data includes:
Acquiring a frame sequence number of a second video frame carrying the preset mark corresponding to a first second audio frame carrying the preset mark in the second video data;
and determining the preset playing time of the first second audio frame carrying the preset mark in the second video data according to the frame number of the first second video frame in the second video data and the frame number of the second video frame carrying the preset mark corresponding to the first second audio frame carrying the preset mark in the second video data.
In an optional manner, before the first video data is input to the video playing system to be tested to obtain the second video data played by the video playing system to be tested, the method includes:
determining a mark period of the preset mark according to preset video parameters and preset audio parameters;
and marking the preset marks on the first video frame and the first audio frame with consistent playing time according to the marking period so as to generate the first video data.
In an optional manner, the determining the marking period of the preset mark according to the preset video parameter and the preset audio parameter includes:
Calculating the least common multiple of the duration of the audio frame and the duration of the video frame according to the preset video parameters and the preset audio parameters;
and determining the marking period of the preset marking according to the least common multiple.
In an optional manner, the preset mark carried by the first audio frame is a volume mark; the marking the preset mark on the first video frame and the first audio frame with consistent playing time according to the marking period to generate the first video data comprises the following steps:
the volume of each first audio frame is marked according to the preset period, and a plurality of first audio frames carrying the preset marks are obtained;
adding the preset marks to the first video frames with consistent playing time according to the preset period according to the plurality of first audio frames with the preset marks, so as to obtain a plurality of first video frames with the preset marks;
and generating the first video data according to the plurality of first audio frames carrying the preset marks and the plurality of first video frames carrying the preset marks.
In an optional manner, the marking the volume of each first audio frame according to the preset period to obtain a plurality of first audio frames carrying the preset mark includes:
Setting the volume of the nth target audio frame to Ndb and the volume of the remaining audio frames to 0db; the target audio frame is determined from a plurality of first audio frames according to the preset period, and the residual audio frames are first audio frames except the target audio frame in the plurality of first audio frames;
the adding the preset mark to the first video frames with consistent playing time according to the preset period according to the plurality of first audio frames with the preset mark, to obtain a plurality of first video frames with the preset mark, includes:
adding a ringing mark on the picture of the Nth target video frame; the target video frames are determined from a plurality of first video frames according to the preset period; the nth target video frame corresponds to the nth target audio frame, and N is an integer greater than 0.
The embodiment of the invention provides a computer program which can be called by a processor to enable audio and video synchronous test equipment to execute the audio and video synchronous test method in any method embodiment.
An embodiment of the present invention provides a computer program product, where the computer program product includes a computer program stored on a computer readable storage medium, where the computer program includes program instructions, when the program instructions are executed on a computer, cause the computer to execute the audio/video synchronization test method in any of the above method embodiments.
The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with the teachings herein. The required structure for a construction of such a system is apparent from the description above. In addition, embodiments of the present invention are not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the above description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component, and they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names. The steps in the above embodiments should not be construed as limiting the order of execution unless specifically stated.
Claims (10)
1. A method for audio and video synchronization testing, the method comprising:
inputting the first video data into a video playing system to be tested to obtain second video data broadcasted by the video playing system to be tested; the first video data comprises a plurality of first video frames and a plurality of first audio frames, the second video data comprises a plurality of second video frames and a plurality of second audio frames, and the first video frames and the first audio frames with consistent playing time respectively carry preset marks;
determining a first second audio frame carrying the preset mark in the second video data;
and determining the audio-video synchronization time delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark.
2. The method of audio video synchronization testing according to claim 1, wherein each of the first video frames is marked with a frame number; the determining the audio-video synchronization delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark includes:
Determining a preset playing time of a first second audio frame carrying the preset mark in the second video data according to the frame sequence number of the first second video frame in the second video data;
determining the actual playing time of the first second audio frame carrying the preset mark;
and determining the audio and video synchronization time delay of the video playing system to be tested according to the preset playing time and the actual playing time.
3. The method according to claim 2, wherein determining the preset play time of the first second audio frame carrying the preset mark in the second video data according to the frame number of the first second video frame in the second video data comprises:
acquiring a frame sequence number of a second video frame carrying the preset mark corresponding to a first second audio frame carrying the preset mark in the second video data;
and determining the preset playing time of the first second audio frame carrying the preset mark in the second video data according to the frame number of the first second video frame in the second video data and the frame number of the second video frame carrying the preset mark corresponding to the first second audio frame carrying the preset mark in the second video data.
4. The method according to claim 1, wherein before the first video data is input to a video playing system to be tested to obtain the second video data played by the video playing system to be tested, the method comprises:
determining a mark period of the preset mark according to preset video parameters and preset audio parameters;
and marking the preset marks on the first video frame and the first audio frame with consistent playing time according to the marking period so as to generate the first video data.
5. The method of claim 4, wherein determining the mark period of the preset mark according to the preset video parameter and the preset audio parameter comprises:
calculating the least common multiple of the duration of the audio frame and the duration of the video frame according to the preset video parameters and the preset audio parameters;
and determining the marking period of the preset marking according to the least common multiple.
6. The method of claim 4, wherein the preset mark carried by the first audio frame is a volume mark; the marking the preset mark on the first video frame and the first audio frame with consistent playing time according to the marking period to generate the first video data comprises the following steps:
The volume of each first audio frame is marked according to the preset period, and a plurality of first audio frames carrying the preset marks are obtained;
adding the preset marks to the first video frames with consistent playing time according to the preset period according to the plurality of first audio frames with the preset marks, so as to obtain a plurality of first video frames with the preset marks;
and generating the first video data according to the plurality of first audio frames carrying the preset marks and the plurality of first video frames carrying the preset marks.
7. The method of claim 6, wherein the marking the volume of each first audio frame according to the preset period to obtain a plurality of first audio frames carrying the preset mark comprises:
setting the volume of the nth target audio frame to Ndb and the volume of the remaining audio frames to 0db; the target audio frame is determined from a plurality of first audio frames according to the preset period, and the residual audio frames are first audio frames except the target audio frame in the plurality of first audio frames;
the adding the preset mark to the first video frames with consistent playing time according to the preset period according to the plurality of first audio frames with the preset mark, to obtain a plurality of first video frames with the preset mark, includes:
Adding a ringing mark on the picture of the Nth target video frame; the target video frames are determined from a plurality of first video frames according to the preset period; the nth target video frame corresponds to the nth target audio frame, and N is an integer greater than 0.
8. An audio and video synchronization test device, the device comprising:
the input module is used for inputting the first video data to the video playing system to be tested to obtain second video data broadcasted by the video playing system to be tested; the first video data comprises a plurality of first video frames and a plurality of first audio frames, the second video data comprises a plurality of second video frames and a plurality of second audio frames, and the first video frames and the first audio frames with consistent playing time respectively carry preset marks;
a first determining module, configured to determine a first second audio frame carrying the preset mark in the second video data;
and the second determining module is used for determining the audio-video synchronization time delay of the video playing system to be tested according to the first second video frame in the second video data and the first second audio frame carrying the preset mark.
9. An audio and video synchronization test apparatus, comprising: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus;
the memory is configured to store at least one executable instruction that causes the processor to perform the operations of the audio video synchronization test method according to any one of claims 1-7.
10. A computer readable storage medium, wherein at least one executable instruction is stored in the storage medium, which when run on an audio-video synchronization test device causes the audio-video synchronization test device to perform the operations of the audio-video synchronization test method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310001594.7A CN116233411A (en) | 2023-01-03 | 2023-01-03 | Method, device, equipment and computer storage medium for audio and video synchronous test |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310001594.7A CN116233411A (en) | 2023-01-03 | 2023-01-03 | Method, device, equipment and computer storage medium for audio and video synchronous test |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116233411A true CN116233411A (en) | 2023-06-06 |
Family
ID=86577792
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310001594.7A Pending CN116233411A (en) | 2023-01-03 | 2023-01-03 | Method, device, equipment and computer storage medium for audio and video synchronous test |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116233411A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117082281A (en) * | 2023-10-17 | 2023-11-17 | 苏州元脑智能科技有限公司 | Audio and video data synchronous processing method, system, equipment and medium |
-
2023
- 2023-01-03 CN CN202310001594.7A patent/CN116233411A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117082281A (en) * | 2023-10-17 | 2023-11-17 | 苏州元脑智能科技有限公司 | Audio and video data synchronous processing method, system, equipment and medium |
| CN117082281B (en) * | 2023-10-17 | 2024-02-23 | 苏州元脑智能科技有限公司 | Audio and video data synchronous processing method, system, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8379735B2 (en) | Automatic video glitch detection and audio-video synchronization assessment | |
| CN106993239B (en) | Information display method in live broadcast process | |
| CN109842795B (en) | Audio and video synchronization performance testing method and device, electronic equipment and storage medium | |
| US11595731B2 (en) | Implementation method and system of real-time subtitle in live broadcast and device | |
| CN108063970A (en) | A kind of method and apparatus for handling live TV stream | |
| CN104902317A (en) | Audio video synchronization method and device | |
| CN111050201A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN111383656B (en) | Voiceprint live broadcast method, voiceprint live broadcast device, server, client device and storage medium | |
| CN112954434B (en) | Subtitle processing method, system, electronic device and storage medium | |
| CN112511818B (en) | Video playing quality detection method and device | |
| CN116233411A (en) | Method, device, equipment and computer storage medium for audio and video synchronous test | |
| CN108174264B (en) | Synchronous lyric display method, system, device, medium and equipment | |
| CN114040255A (en) | Live caption generating method, system, equipment and storage medium | |
| CN106331820B (en) | Audio and video synchronization processing method and device | |
| CN103581741A (en) | Special-effect playing device and method | |
| CN109040818A (en) | Audio and video synchronization method, storage medium, electronic equipment and system when live streaming | |
| CN113596545A (en) | Audio track switching method and device, media playing device and computer storage medium | |
| CN109218849B (en) | Live data processing method, device, equipment and storage medium | |
| CN113630620A (en) | Multimedia file playing system, related method, device and equipment | |
| CN106303754A (en) | A kind of audio data play method and device | |
| CN114697712B (en) | Method, device and equipment for downloading media stream and storage medium | |
| CN116437068A (en) | Lip synchronization test method and device, electronic equipment and storage medium | |
| CN109600571B (en) | Multimedia resource transmission test system and multimedia resource transmission test method | |
| CN113784094A (en) | Video data processing method, gateway, terminal device and storage medium | |
| CN112073791A (en) | Playing synchronization method and device, storage medium and user side |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |