WO2017071670A1

WO2017071670A1 - Audio and video synchronization method, device and system

Info

Publication number: WO2017071670A1
Application number: PCT/CN2016/104028
Authority: WO
Inventors: 刘成刚; 易鹤声; 陈洲; 曹珈; 范旭彤; 尤洪涛; 田智平; 王芳
Original assignee: 中兴通讯股份有限公司
Priority date: 2015-10-30
Filing date: 2016-10-31
Publication date: 2017-05-04
Also published as: CN106658065A; CN106658065B

Abstract

Provided in embodiments of the present invention are an audio and video synchronization method, device and system. The method comprises: a first device performs a difference reducing processing on differences of presentation time stamps (PTSs) of an audio packet and a video packet in a source code stream to obtain a restructured code stream, wherein the restructured code stream is used by a set-top box to perform an audio and video synchronization processing; and the first device sends the restructured code stream to the set-top box. The embodiment of the present invention solves the problem in the related art in which the audio and video code stream synchronization has poor effect when an STB switches channels, thereby enabling fast audio and video synchronization during channel switching without having to modify the set-top box, thus greatly improving user experience.

Description

Audio and video synchronization method, device and system

Technical field

The present invention relates to the field of communications, and in particular to an audio and video synchronization method, apparatus, and system.

Background technique

With the growing maturity of STB (Set Top Box) technology, how to improve the user experience has become an urgent problem to be solved. Generally, when the STB switches channels, in order to realize synchronous output of video and audio, the following two synchronization mechanisms are generally used:

(1) Slow synchronization: When the STB receives the video, it starts to output the video (not synchronized with the audio). During the playback, the video is synchronized with the audio by playing back or dropping the frame. The advantage of this method is that the video broadcast speed is fast, the channel switching speed is fast, but the slow motion phenomenon is seen in the first few seconds of cutting in the channel, thereby affecting the user experience.

(2) Fast synchronization: The STB outputs the video picture after the video and audio are synchronized. In this way, the user does not see the process of video and audio synchronization, but since there is video output after synchronization, the channel switching time is long, which in turn affects the user experience.

Therefore, no matter which of the above schemes is adopted, there is no influence that the audio stream is not synchronized.

In view of the problem that the audio and video code stream synchronization effect is poor when the STB switches channels in the related art, an effective solution has not been proposed yet.

Summary of the invention

The embodiment of the invention provides a method, a device and a system for synchronizing audio and video, so as to solve at least the problem that the audio and video code stream synchronization effect is poor when the STB switches channels in the related art.

According to an embodiment of the present invention, there is provided a method for synchronizing audio and video, comprising: acquiring, by a first device, an original code stream sent by an encoder; and displaying, by a first device, a time stamp PTS of an audio packet and a video packet in the original code stream The difference is subjected to a reduction process to obtain a recombined code stream, wherein the reassembled code stream is used for the set top box to perform audio and video synchronization processing; and the first device transmits the reassembled code stream to the set top box.

Optionally, performing, by the first device, a reduction process on the difference between the audio packet in the original code stream and the display time label PTS of the video packet includes: detecting, by the first device, a difference between the audio packet in the original code stream and the PTS of the video packet. Whether it is more than 200 ms; if it is detected that the difference between the audio packet of the original stream and the PTS of the video packet exceeds 200 ms, the first device performs a reduction process on the difference between the audio packet of the original stream and the PTS of the video packet.

Optionally, after the first device sends the reassembled code stream to the set top box, the method further includes: the set top box performs audio and video synchronization processing according to the reassembled code stream, and detects whether the synchronization is successful; if the synchronization is successful, the set top box is first. The device sends a termination instruction, wherein the termination instruction is used to instruct the first device to stop transmitting the reassembled code stream; and/or if the first pre- If the synchronization is not detected within the duration, the set top box sends a termination command to the first device.

Optionally, after the first device sends the reassembled code stream to the set top box, the method further includes: after receiving the reassembled code stream for the second preset duration, the set top box starts to receive the original code stream, and the time length of receiving the original code stream is reached. After the third preset duration, the receiving of the reassembled code stream is stopped, wherein the set top box performs a fusion process on the reassembled code stream and the original code stream within a third preset duration of the set top box receiving the original code stream.

Optionally, performing, by the set-top box, the merging process on the reassembled code stream and the original code stream, the set-top box performs a fusion process on the recombined code stream and the original code stream according to the real-time transport protocol RTP packet sequence number respectively carried by the reassembled code stream and the original code stream.

Optionally, the acquiring, by the first device, the original code stream sent by the encoder comprises: acquiring, by the first device, the original code stream sent by the encoder via the content distribution network CDN device.

Optionally, the sending, by the first device, the reassembled code stream to the set top box includes: sending, by the first device, the reassembled code stream to the set top box via the CDN device.

Optionally, before the first device sends the reassembled code stream to the set top box, the method further includes: the set top box receives the channel switching instruction; and according to the channel switching instruction, the set top box sends a request message to the first device, where the request message is used for the request A device sends a reassembled code stream.

Optionally, the first device is a CDN device, wherein the CDN device comprises one of the following: a backbone CDN device, an edge CDN device.

According to another embodiment of the present invention, there is provided an audio and video synchronization apparatus, comprising: an acquisition module configured to acquire an original code stream sent by an encoder; and a processing module configured to match an audio package and a video package in the original code stream The difference of the display time label PTS performs a reduction process to obtain a recombined code stream, wherein the recombined code stream is used for the set top box to perform audio and video synchronization processing; and the sending module is configured to send the recombined code stream to the set top box.

According to another embodiment of the present invention, an audio and video synchronization system is provided, including: an encoder configured to transmit a source code stream; and a first device configured to acquire an original code stream sent by the encoder, in the original code stream The difference between the display time stamp PTS of the audio pack and the video pack performs a reduction process, obtains a recombined code stream, and sends the reassembled code stream to the set top box, wherein the reassembled code stream is used for performing set-top box to perform audio and video synchronization processing; the set top box is set to receive The code stream is reorganized, and audio and video synchronization processing is performed according to the recombined code stream.

In the embodiment of the present invention, the original code stream sent by the encoder is obtained by the first device; the first device performs a reduction process on the difference between the audio packet in the original code stream and the display time label PTS of the video packet to obtain a recombined code stream, where The recombined code stream is used for performing the audio and video synchronization processing on the set top box; the first device sends the recombined code stream to the set top box, which solves the problem that the audio and video code stream synchronization effect is poor when the STB switches channels in the related art, and thus does not need to do the set top box. Any modification can quickly realize audio and video synchronization when the channel is switched, which greatly improves the user experience.

DRAWINGS

The drawings described herein are provided to provide a further understanding of the invention, which form a part of this application, The illustrative embodiments and the description thereof are illustrative of the invention and are not to be construed as limiting the invention. In the drawing:

1 is a flowchart of an audio and video synchronization method according to a first embodiment of the present invention;

2 is a schematic diagram of a topology structure of an audio and video transmission network;

3 is a schematic diagram of an audio and video synchronization method according to a second embodiment of the present invention;

4 is a schematic diagram of an audio and video synchronization method according to a third embodiment of the present invention;

FIG. 5 is a schematic diagram of an audio and video synchronization method according to a fourth embodiment of the present invention; FIG.

6 is a schematic diagram of an audio and video synchronization method according to a fifth embodiment of the present invention;

7 is a schematic diagram of an audio and video synchronization apparatus according to an embodiment of the present invention;

8 is a schematic diagram of an audio and video synchronization system in accordance with an embodiment of the present invention.

detailed description

The invention will be described in detail below with reference to the drawings in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict.

It is to be understood that the terms "first", "second" and the like in the specification and claims of the present invention are used to distinguish similar objects, and are not necessarily used to describe a particular order or order.

In this embodiment, an audio and video synchronization method is provided. FIG. 1 is a flowchart of an audio and video synchronization method according to a first embodiment of the present invention. As shown in FIG. 1, the flow includes the following steps:

Step S102: The first device acquires a source code stream sent by the encoder.

In this step, the first device may be directly connected to the encoder, that is, directly receive the original code stream sent by the encoder, or the first device may also be indirectly connected to the encoder, for example, between the first device and the encoder. There are other devices, such as CDN devices, so that the first device can indirectly receive the original stream sent by the encoder.

Step S104: The first device performs a reduction process on the difference between the audio packet in the original code stream and the display time label PTS of the video packet to obtain a recombined code stream, where the reassembled code stream is used in the set top box to perform audio and video synchronization processing.

The PTS is a time stamp indicating the time at which the audio and video are displayed. The display time stamp PTS may be carried in the PES header information of the original code stream for determining the order in which the audio and video are displayed.

In this embodiment, the difference in display time stamp PTS of the original code stream generated by the encoder may be small or large. Optionally, a reference threshold may be preset, and when the difference of the PTS of the original code stream exceeds the reference threshold, the difference of the PTS of the original code stream is considered to be large, and vice versa, the difference of the PTS of the original code stream is considered. The value is small. Performing a reduction process on the difference between the audio packet of the original code stream and the display time label PTS of the video packet by the first device includes: performing, by the first device, a difference between the audio packet in the original code stream and the display time label PTS of the video packet Reduce the processing so that the difference in PTS does not exceed The above reference threshold also results in a recombination code stream. It should be noted that the recombined code stream has, but is not limited to, the following characteristics: the difference between the PTS of the audio packet and the video packet does not exceed the above reference threshold.

In order to enable the reassembled code stream to be more accurately applied to the set top box to perform audio and video synchronization, optionally, the above reference threshold may be set to 200 ms, and the first device displays the time stamp of the audio pack and the video pack in the original stream. Performing the reduction processing of the difference of the PTS includes: the first device detects whether the difference between the audio packet of the original code stream and the PTS of the video packet exceeds 200 ms; if the difference between the PTS of the audio packet and the video packet in the original code stream is detected After more than 200 ms, the first device performs a reduction process on the difference between the audio packets in the original stream and the PTS of the video packets.

It should be noted that the first device is disposed between the encoder and the set top box, and may be a device introduced outside the audio and video transmission network, or may be a device role concurrently owned by a certain device in the audio and video transmission network. For example, the first device may be a CDN device in a transport network, the CDN device being different from the conventional CDN device in that a processing module configured to perform a PTS difference reduction process is provided in the CDN device. Therefore, in the present invention, the physical form of the first device is not specifically limited, but the first device needs to have the following functions:

After receiving the code stream of the original encoder, extracting and analyzing the audio and video content to determine whether the video and the audio PTS are too different (for example, if the PTS value of the video packet and the audio packet received by the STB at the same time exceeds 200 ms, It will have a certain impact on the audio and video synchronization. If it is, the content of the code stream will be recombined twice. The recombined code stream needs to make the PTS difference between the audio package and the video packet small enough (for example, less than 200ms).

In addition, it should be noted that, in the present invention, there are many specific reduction methods that can be used for the reduction of the difference between the PTS of the audio pack and the video pack, and the present invention does not specifically limit this.

Step S106, the first device sends the reassembled code stream to the set top box.

In this step, the first device may directly send the recombined code stream to the set top box, that is, the first device and the set top box are directly connected, or another device may be disposed between the first device and the set top box, for example, the first device. A CDN device is disposed between the set top box and the CDN device, and the CDN device can transmit the recombined code stream generated by the first device to the set top box through the CDN device.

Here, after receiving the recombined code stream sent by the first device, the set top box performs synchronization of the audio and video based on the recombined code stream. Since the difference between the PTS of the audio stream of the reassembled code stream and the PTS of the video packet is reduced compared to the original code stream, after the set-top box receives the original code stream (recombined code stream) with the PTS difference reduced, the reassembly can be performed according to the reassembly. The code stream quickly realizes audio and video synchronization, thereby shortening the time of synchronization processing on the set top box side, and improving the user experience of the channel switching user.

It should be noted that the process of performing audio and video synchronization of the set top box can be performed based on the recombined code stream, that is, in the process of controlling the playing of audio and video by the set top box, the recombined code stream is continuously generated and sent to the set top box; or, When the audio and video synchronization is initially performed, by means of the recombined code stream, after the audio and video synchronization is successful, the original code stream is received instead, thereby realizing the synchronized broadcast of the audio and video.

In this embodiment, the original code stream sent by the encoder is obtained by the first device; the first device performs a reduction process on the difference between the audio time packet of the original code stream and the display time label PTS of the video packet to obtain a recombined code stream, where The recombined code stream is used for performing audio and video synchronization processing on the set top box; the first device sends the reassembled code stream to the set top box, and solves the related art in the STB cut When changing channels, the problem of poor synchronization of audio and video code streams, and thus without any modification to the set-top box, can quickly achieve audio and video synchronization during channel switching, greatly improving the user experience.

Optionally, after the first device sends the reassembled code stream to the set top box, the method further includes: the set top box performs audio and video synchronization processing according to the reassembled code stream, and detects whether the synchronization is successful; if the synchronization is successful, the set top box is first. The device sends a termination instruction, wherein the termination instruction is used to instruct the first device to stop transmitting the reassembled code stream; and/or, if the synchronization is not detected within the first preset duration, the set top box sends a termination instruction to the first device.

In order to improve the efficiency of the audio and video synchronization, optionally, the synchronization of the audio and video synchronization may be performed. If the synchronization is successful, the set top box instructs the first device to stop transmitting the recombined code stream, that is, to quickly realize the audio and video synchronization. After that, the PTS difference reduction processing is not performed by using the first device; or, the timer may be set in advance for the audio and video synchronization process of the set top box, and the timer starts to perform timing in the set top box, and is preset. Stop timing after the duration. At the same time, detecting whether the audio and video synchronization is successful during the timing, if the set top box does not detect the audio and video synchronization success within the preset time period, when the timing is terminated, the termination instruction is sent to the first device; if the timing is detected within the time duration If the audio and video synchronization is successful, a termination command is sent to the first device when the synchronization is successfully detected.

It should be noted that during the process of playing audio and video in the set top box, the recombined code stream can be utilized in the whole process. Alternatively, the original code stream sent by the encoder may be started to be received by the original code stream and the recombined code stream, and the original code stream is synchronized, so as to follow the preset time length before the first device stops transmitting the recombined code stream. The original code stream can be received only, and the reassembled code stream is no longer needed to be received; that is, the reassembled code stream lays a foundation for audio and video synchronization of the original code stream only for a short period of time, and is subsequently utilized in audio and video playback. Still the original stream.

In order to improve the performance of the audio and video synchronization, the method further includes: after the first device sends the reassembled code stream to the set top box, the method further includes: the set top box starts to receive the original code stream after receiving the reassembled code stream for a second preset duration After receiving the original code stream for a third preset duration, stopping receiving the reassembled code stream, wherein the set top box performs a fusion process on the reassembled code stream and the original code stream within a third preset duration of the set top box receiving the original code stream. .

In this embodiment, the set top box initially receives only the reassembled code stream, and after receiving the reassembled code stream for a second preset duration, begins to receive the reassembled code stream and the original code stream sent by the encoder (wherein the first device can be timed) After the second preset duration is reached, the set top box is notified to receive the original code stream sent by the encoder. The set top box performs fusion processing on the two streams. Optionally, the set-top box performs the fusion processing on the recombined code stream and the original code stream by performing the following manner: the set-top box performs the fusion on the recombined code stream and the original code stream according to the real-time transport protocol RTP packet sequence number carried by the reassembled code stream and the original code stream respectively. deal with. Specifically, the set top box performs the fusion of the two code streams according to the received original code stream and the RTP header sequence of the previously received recombined code stream. When the packet sequence number of the original code stream catches up with the packet sequence number of the reassembled code stream, the A device stops transmitting the reassembled code stream. A timer may be set in the set-top box. If the synchronization time of the timer is not successfully achieved, the first device may be instructed to terminate the transmission of the reassembled code stream, and only the original code stream is received.

It should be noted that, for the fusion processing of the recombined code stream and the original code stream in the present invention, a specific fusion method may be adopted, and no specific limitation is made herein.

In this embodiment, the first device can be directly connected to the set top box to send the reassembled code stream to the set top box; or the first device can be connected to the set top box via other devices.

In this embodiment, the first device may be directly connected to the encoder to receive the original code stream transmitted by the encoder; or the first device may be connected to the encoder via other devices.

In order to perform effective control on the audio and video synchronization, the channel switching instruction (for example, the channel switching instruction input by the user through the remote control device) may be used as a trigger signal, and when the trigger signal is received, the set top box automatically returns to the first The device sends a request message for reassembling the code stream, and the first device sends the reassembled code stream to the set top box according to the request message.

In the above embodiment, by introducing a device between the encoder and the set top box (the device is deployed on a node server in the code stream transmission network, or a server additionally introduced in the code stream transmission network), The device inputs the original code stream output by the encoder (first extracts the PTS information of the audio and video, and determines whether the PTS difference is too large, for example, the PTS value of the video packet and the audio packet received by the terminal at the same time is too large, if it is too large, if it is too large That is, the PTS difference of the audio and video packets is subjected to a reduction process, and the code stream recombined by the audio and video PTS is output, and the set top box receives the recombined code stream and quickly realizes audio and video synchronization. The above embodiment does not require any improvement of the set-top box, and can quickly realize audio and video synchronization when switching channels, which greatly improves the user's viewing experience.

2 is a schematic diagram of a topology of an audiovisual transmission network. As shown in FIG. 2, the topology includes: the encoder transmits the original code stream to the backbone CDN device, and the backbone CDN device sends the original code stream to the set top box terminal via the edge CDN device, and the set top box terminal performs corresponding audio and video synchronization control.

3 is a schematic diagram of an audio and video synchronization method according to a second embodiment of the present invention. As shown in FIG. 3, a PTS reassembly server is added based on the topology of the audio and video transmission network. The PTS reassembly server is disposed between the CDN device and the set top box terminal. The encoder sends the original code stream to the PTS reassembly server via the CDN device, and the PTS reassembly server analyzes the difference between the PTS of the video packet and the audio packet in the original code stream, and when the difference exceeds the reference threshold, the PTS performs the reduction. Processing, and acquiring the PTS recombination code stream (that is, the recombined code stream in the above embodiment). The PTS reassembly server sends the PTS recombination code stream to the set-top box terminal, and the set-top box terminal quickly implements audio and video synchronization according to the received PTS recombination code stream, and after the synchronization succeeds, instructs the PTS reassembly server to stop transmitting the PTS recombination code stream, and then receives the The original code stream generated by the encoder forwarded by the CDN device combines the original code stream with the PTS recombination code stream to realize synchronous output of audio and video.

Specifically, the process mainly includes:

Step S31, deploying a PTS reassembly server in the transmission network, the function of the PTS reassembly server is to receive the code stream of the original encoder, perform caching, analyze the audio and video information, and rearrange the difference of the audio and video PTS, and the difference is The value is reduced.

Step S32, when the channel is switched, the set top box (STB) first communicates with the PTS reassembly server, requests the PTS to reassemble the code stream, and the code stream received by the PTS reassembly server is originated from the CDN central node, and is cached for a short time. This buffer is enough for the STB to quickly synchronize audio and video.

Step S33: After receiving the request command of the STB, the PTS reassembly server sends the reassembled code stream to the STB.

Step S34, after receiving the code stream of the PTS reassembly server, the STB quickly performs audio and video synchronization.

Step S35: After the STB synchronization succeeds, the PTS reassembly server immediately sends a command to stop the reassembly of the code stream (that is, the termination instruction described above), and then accepts the CDN source code stream, and performs fusion of the two streams.

Step S36, if the STB times out without synchronization success, the receiving recombination code stream is immediately stopped, and the CDN source code stream is received instead.

With this embodiment, by setting a PTS reassembly server between the CDN device and the set top box terminal, the reassembled code stream is sent to the set top box terminal, and the set top box terminal can quickly realize the synchronization of audio and video without any improvement on the set top box terminal. It can quickly realize audio and video synchronization when switching channels, which greatly improves the user's viewing experience.

4 is a schematic diagram of an audio and video synchronization method according to a third embodiment of the present invention. As shown in FIG. 4, a PTS reassembly server is added based on the topology of the audio and video transmission network. The PTS reassembly server is disposed between the encoder and the backbone CDN device, and the PTS reassembly server receives the original code stream sent by the encoder, and performs PTS difference reduction processing on the original code stream to obtain a PTS recombination code stream, and the PTS recombination code is obtained. The stream is sent to the backbone CDN device, and the backbone CDN device sends the PTS reassembly code stream to the set top box terminal via the edge CDN device.

Specifically, the process mainly includes:

In step S41, a PTS reassembly server is deployed at the back end of the encoder.

In step S42, the PTS reassembly server accepts the original code stream from the encoder, analyzes the code stream, and reassembles the audio and video PTS.

In step S43, the reassembled code stream is transmitted to the STB through the CDN device.

In step S44, the STB can rely on the recombined code stream for fast synchronization.

With this embodiment, a PTS reassembly server is set up between the CDN device and the encoder to send the recombined code stream to the set-top box terminal through the CDN device, and the set-top box terminal can quickly realize the synchronization of audio and video, without the need for the set-top box terminal. Any improvement can quickly realize audio and video synchronization when switching channels, greatly improving the user's viewing experience.

FIG. 5 is a schematic diagram of an audio and video synchronization method according to a fourth embodiment of the present invention. As shown in FIG. 5, on the basis of the topology of the audio and video transmission network, a PTS reassembly code stream module is deployed on the backbone CDN device for performing a reduction process on the difference of the PTS of the original code stream. The encoder sends the original code stream to the backbone CDN device, and the backbone CDN device passes. The self-deployed PTS recombination code stream module performs a reduction process on the difference of the PTS of the original code stream to obtain a PTS recombination code stream, and sends the PTS recombination code stream to the set top box terminal via the edge CDN, and the set top box terminal reassembles the code stream based on the PTS. Realize audio and video synchronization.

Specifically, the process mainly includes:

In step S51, the PTS reassembly code stream module is directly deployed on a backbone CDN device.

Step S52, the PTS recombination code stream module analyzes the video and audio information, and reorganizes the audio and video PTS.

In step S53, the reassembled code stream is sent through the backbone CDN device.

Step S54: The STB receives the PTS recombination code stream of the backbone CDN device through the edge CDN device to implement fast synchronization of audio and video.

In this embodiment, by setting a PTS reassembly code stream module in the backbone CDN device, the recombined code stream is sent to the set top box terminal through the CDN device, and the set top box terminal can quickly realize audio and video synchronization, without any need for the set top box terminal. Improved, you can quickly achieve audio and video synchronization when switching channels, greatly improving the user's look and feel experience.

FIG. 6 is a schematic diagram of an audio and video synchronization method according to a fifth embodiment of the present invention. As shown in FIG. 6, on the basis of the topology of the audio and video transmission network, a PTS reassembly code stream module is deployed on the edge CDN device, and is set to perform a reduction process on the difference of the PTS of the original code stream. The encoder generates the original code stream, which is sent to the edge CDN device by the backbone CDN device, and the PTS recombination code stream module is deployed in the edge CDN device, and the module performs a reduction process on the difference of the PTS of the original code stream to obtain a reduction process. The PTS reassembles the code stream and sends the PTS reassembly code stream to the set top box terminal. The set top box terminal performs audio and video synchronization based on the PTS reassembly code stream.

Specifically, the process mainly includes:

S61. Deploy a PTS reassembly code stream module on an edge CDN device.

S62: The PTS recombination code stream module analyzes the video and audio information, and reorganizes the audio and video PTS.

S63, the recombined code stream is sent through the edge CDN device.

S64. The STB receives the PTS reassembly code stream of the edge CDN device to implement fast synchronization of audio and video.

With this embodiment, by setting a PTS reassembly code stream module in the edge CDN device, the recombination code stream is sent to the set top box terminal through the CDN device, and the set top box terminal can quickly realize the synchronization of audio and video, thereby reducing the reorganization range, and not Any improvement to the set-top box terminal is required to quickly synchronize audio and video when switching channels, which greatly enhances the user's viewing experience.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases, the former is A better implementation. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, CD-ROM, including a number of instructions to make a terminal device (can be a mobile phone, a computer, The server, or network device, etc.) performs the methods described in various embodiments of the present invention.

An audio and video synchronization device is also provided in this embodiment, and the device is configured to implement the foregoing embodiments and preferred embodiments, and details are not described herein. As used below, the term "module" may implement a combination of software and/or hardware of a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware, is also possible and contemplated.

FIG. 7 is a schematic diagram of an audio and video synchronization apparatus according to an embodiment of the present invention. As shown in FIG. 7, the apparatus includes an acquisition module 70, a processing module 72, and a transmission module 74.

The obtaining module 70 is configured to obtain a source code stream sent by the encoder.

The processing module 72 is configured to perform a reduction process on the difference between the audio packet in the original code stream and the display time label PTS of the video packet to obtain a recombined code stream, where the recombined code stream is used in the set top box to perform audio and video synchronization processing.

The sending module 74 is configured to send the reassembled code stream to the set top box.

In this embodiment, the original code stream sent by the encoder is obtained by the obtaining module 70; the processing module 72 performs a reduction process on the difference between the audio packet in the original code stream and the display time label PTS of the video packet to obtain a recombined code stream, where The reassembled code stream is used for performing the audio and video synchronization processing on the set top box; the sending module 74 sends the recombined code stream to the set top box, which solves the problem that the audio and video code stream synchronization effect is poor when the STB switches channels in the related art, and thus does not need to do any work on the set top box. The modification can quickly realize the audio and video synchronization when the channel is switched, which greatly improves the user experience.

It should be noted that each of the above modules may be implemented by software or hardware. For the latter, the foregoing may be implemented by, but not limited to, the foregoing modules are all located in the same processor; or, the modules are located in multiple In the processor.

An audio-video synchronization system is also provided in this embodiment. The system is configured to implement the foregoing embodiments and preferred embodiments, and details are not described herein.

8 is a schematic diagram of an audio and video synchronization system in accordance with an embodiment of the present invention. As shown in FIG. 8, the system includes an encoder 80, a first device 82, and a set top box 84.

Encoder 80 is arranged to transmit the original stream.

The first device 82 is configured to obtain an original code stream sent by the encoder, perform a reduction process on the difference between the audio time packet of the original code stream and the display time label PTS of the video packet, obtain a recombined code stream, and send the recombined code stream. To the set top box, wherein the reassembled code stream is used for the set top box to perform audio and video synchronization processing.

The set top box 84 is configured to receive the reassembled code stream and perform audio and video synchronization processing according to the reassembled code stream.

In this embodiment, the original code stream is sent by the encoder 80. The first device 82 obtains the original code stream sent by the encoder, and performs a reduction process on the difference between the audio packet in the original code stream and the display time label PTS of the video packet. And reassembling the code stream, and sending the recombined code stream to the set top box, wherein the recombined code stream is used for the set top box to perform audio and video synchronization processing; the set top box 84 receives the recombined code stream, and performs audio and video synchronization processing according to the reassembled code stream, and solves the related art in the related art. When the STB switches channels, The problem of poor synchronization of audio and video code streams, and thus without any modification to the set-top box, can quickly realize audio and video synchronization during channel switching, greatly improving the user experience.

Embodiments of the present invention also provide a storage medium. Optionally, in the embodiment, the foregoing storage medium may be configured to store program code for performing the following steps:

S1: Obtain the original code stream sent by the encoder.

S2, performing a reduction process on the difference between the audio time packet of the original code stream and the display time label PTS of the video packet to obtain a recombined code stream, where the recombined code stream is used for the set top box to perform audio and video synchronization processing.

S3, the reassembled code stream is sent to the set top box.

Optionally, in this embodiment, the foregoing storage medium may include, but not limited to, a USB flash drive, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, and a magnetic memory. A variety of media that can store program code, such as a disc or a disc.

It will be apparent to those skilled in the art that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein. The steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps thereof are fabricated as a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Industrial applicability

As described above, the audio-video synchronization method, device, and system provided by the embodiments of the present invention have the following beneficial effects: the problem that the audio and video code stream synchronization effect is poor when the STB switches channels in the related art is solved, and thus the problem is not needed. By making any modifications to the set-top box, you can quickly synchronize audio and video during channel switching, greatly improving the user experience.

Claims

An audio and video synchronization method includes:

The first device acquires a source code stream sent by the encoder;

And performing, by the first device, a reduction process on a difference between a display time stamp PTS of the audio packet and the video packet in the original code stream, to obtain a recombined code stream, where the recombined code stream is used to perform audio and video synchronization processing on the set top box;

The first device sends the reassembled code stream to the set top box.
The method according to claim 1, wherein the performing, by the first device, a reduction process on a difference value of a display time stamp PTS of the audio packet and the video packet in the original code stream comprises:

The first device detects whether a difference between a PTS of the audio packet and the video packet in the original code stream exceeds 200 ms;

If it is detected that the difference between the audio packets of the original code stream and the PTS of the video packet exceeds 200 ms, the first device performs a reduction process on the difference between the audio packets of the original code stream and the PTS of the video packets. .
The method of claim 1, wherein after the first device sends the reassembled code stream to the set top box, the method further comprises:

The set top box performs audio and video synchronization processing according to the reassembled code stream, and detects whether the synchronization is successful;

If the synchronization is successful, the set top box sends a termination instruction to the first device, where the termination instruction is used to instruct the first device to stop sending the reassembled code stream; and/or,

If the synchronization is not detected within the first preset duration, the set top box sends the termination instruction to the first device.
The method of claim 1, wherein after the first device sends the reassembled code stream to the set top box, the method further comprises:

After receiving the reassembled code stream for a second preset duration, the set-top box starts to receive the original code stream, and stops receiving the reassembled code stream after receiving the original code stream for a third preset duration. The set top box performs a fusion process on the reassembled code stream and the original code stream within a third preset duration in which the set top box receives the original code stream.
The method according to claim 4, wherein said set top box performs a fusion process on said reassembled code stream and said original code stream comprises:

The set top box performs a fusion process on the reassembled code stream and the original code stream according to the real-time transport protocol RTP packet sequence number respectively carried by the reassembled code stream and the original code stream.
The method according to claim 1, wherein the acquiring, by the first device, the original code stream sent by the encoder comprises:

The first device acquires a source code stream sent by the encoder via a content distribution network CDN device.
The method of claim 1 wherein said first device transmits said reassembled code stream to said set top box include:

The first device transmits the reassembled code stream to the set top box via a CDN device.
The method of claim 1, wherein before the first device sends the reassembled code stream to the set top box, the method further comprises:

The set top box receives a channel switching instruction;

And the set top box sends a request message to the first device according to the channel switching instruction, where the request message is used to request the first device to send the reassembled code stream.
The method of claim 1, wherein the first device is a CDN device, wherein the CDN device comprises one of: a backbone CDN device, an edge CDN device.
An audio and video synchronization device includes:

Obtaining a module, configured to obtain a source code stream sent by the encoder;

a processing module, configured to perform a reduction process on a difference between a display time stamp PTS of the audio packet and the video packet in the original code stream, to obtain a recombined code stream, where the reassembled code stream is used to perform audio and video synchronization processing on the set top box;

And a sending module, configured to send the reassembled code stream to the set top box.
An audio and video synchronization system includes:

An encoder configured to transmit a source stream;

a first device, configured to acquire an original code stream sent by the encoder, perform a reduction process on a difference between a display time stamp PTS of the audio packet and the video packet in the original code stream, obtain a recombined code stream, and Transmitting the reassembled code stream to the set top box, wherein the reassembled code stream is used to perform audio and video synchronization processing on the set top box;

The set top box is configured to receive the reassembled code stream, and perform the audio and video synchronization process according to the reassembled code stream.
A computer storage medium arranged to store a computer program for performing the audio-video synchronization method according to any one of claims 1 to 9.