CN113452946A - Audio and video transmission method and system for video telephone - Google Patents
Audio and video transmission method and system for video telephone Download PDFInfo
- Publication number
- CN113452946A CN113452946A CN202110721246.8A CN202110721246A CN113452946A CN 113452946 A CN113452946 A CN 113452946A CN 202110721246 A CN202110721246 A CN 202110721246A CN 113452946 A CN113452946 A CN 113452946A
- Authority
- CN
- China
- Prior art keywords
- video
- audio
- signal
- frame
- transmission
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 74
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000005236 sound signal Effects 0.000 claims abstract description 97
- 230000001360 synchronised effect Effects 0.000 claims abstract description 35
- 238000004891 communication Methods 0.000 claims abstract description 11
- 230000006870 function Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 7
- 230000006835 compression Effects 0.000 description 5
- 238000007906 compression Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/233—Processing of audio elementary streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/23602—Multiplexing isochronously with the video sync, e.g. according to bit-parallel or bit-serial interface formats, as SDI
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/2368—Multiplexing of audio and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
Abstract
The invention provides an audio and video transmission method and system of a video telephone, and relates to the field of audio and video communication. An audio and video transmission method for video telephony comprising: receiving an audio signal and a video signal, and coding the audio signal and the video signal to obtain a code rate value; generating a synchronous signal, adding the synchronous signal into the received audio signal and video signal, and respectively configuring a frame queue priority sending queue of audio and video according to the synchronous signal; and transmitting the plurality of frame packets and the plurality of audio and video frames according to a preset transmission rule from a first frame packet corresponding to the first audio and video in the sending queue and a first audio and video frame in the plurality of audio and video frames. The video signal and the audio signal can be merged according to the synchronous signal, and the synchronism of the video signal and the audio signal in the video and audio file after subsequent merging is ensured. The invention also provides an audio and video transmission system of the video telephone.
Description
Technical Field
The invention relates to the field of audio and video communication, in particular to an audio and video transmission method and system of a video telephone.
Background
With the development of communication network technology, video and audio become very popular in life, and the requirements of users on the visual and auditory experiences are higher and higher. Therefore, it is very critical to achieve the synchronous transmission of video and audio, and the synchronization of video picture and audio lip sound.
In the prior art, when audio and video synchronization is implemented, video and audio generated by the same device are input to the same encoder (or soft encoding) for synchronous encoding, so as to form a Real Time Streaming Protocol (RTSP) stream, and then the RTSP stream is transmitted through a network. At the end, the video is output to the video output device and the audio output device in synchronization with the audio decoding by the same decoder (or soft decoding).
In the existing communication application, when a terminal device transmits audio and video, a video code stream is transmitted to the terminal device, the terminal device obtains the audio code stream by decoding the video code stream, and then sends the audio code stream to an earphone through a network, so that a user perceives that sound and pictures are not synchronous and experience feeling is poor.
Therefore, how to transmit audio and video is a problem to be solved urgently.
Disclosure of Invention
The invention aims to provide an audio and video transmission method of a video telephone, which can ensure that a video signal and an audio signal received at the same time have the same synchronous signal, can combine the video signal and the audio signal according to the synchronous signal when combining, and ensure the synchronism of the video signal and the audio signal in a video and audio file after subsequent combination, thereby meeting the use of a user.
Another object of the present invention is to provide an audio and video transmission system of a video phone capable of operating an audio and video transmission method of a video phone.
The embodiment of the invention is realized by the following steps:
in a first aspect, an embodiment of the present application provides an audio and video transmission method for a video phone, which includes receiving an audio signal and a video signal, and encoding the audio signal and the video signal to obtain a code rate value; generating a synchronous signal, adding the synchronous signal into the received audio signal and video signal, and respectively configuring a frame queue priority sending queue of audio and video according to the synchronous signal; and transmitting the plurality of frame packets and the plurality of audio and video frames according to a preset transmission rule from a first frame packet corresponding to the first audio and video in the sending queue and a first audio and video frame in the plurality of audio and video frames.
In some embodiments of the present invention, the receiving an audio signal and a video signal, and encoding the audio signal and the video signal to obtain a bitrate value includes: and obtaining the size of each frame when the audio signal and the video signal reach a preset frame rate after being coded according to the size and the transmission time of the audio signal and the video signal, and obtaining a code rate value according to the size of each frame.
In some embodiments of the present invention, the above further includes: the audio and video code rate values output after the audio and video are coded are processed through a video callback function and an audio callback function, the video callback function distinguishes video key frames and video non-key frames according to frame types returned by a coding library, and the audio callback function puts the audio into an audio frame sending queue.
In some embodiments of the present invention, the generating the synchronization signal to be added to the received audio signal and video signal, and configuring the frame queue priority transmission queue of the audio and video respectively according to the synchronization signal includes: and adding the generated synchronous signals into the received audio signals and video signals, and combining the received audio signals, video signals and synchronous signals according to the synchronous signals added in the audio signals and video signals to obtain combined audio signals and video signals.
In some embodiments of the present invention, the above further includes: and according to the synchronous signal added in the audio signal and the video signal and reserving the synchronous signal, combining the received audio signal and the video signal to obtain a combined audio signal and video signal and reserving the synchronous signal, and combining the combined audio signal and the video signal according to the reserved synchronous signal in the combined audio signal and video signal.
In some embodiments of the present invention, the transmitting the plurality of frame packets and the plurality of audio and video frames according to the preset transmission rule from the first frame packet corresponding to the first audio and video in the transmission queue and from the first audio and video frame in the plurality of audio and video frames comprises: and recording the packet loss number and the continuous packet loss time of the transmission queue, and if the transmission queue reaches the packet loss condition, discarding the subsequent frame packets of the audio and video of the priority transmission queue and actively reducing the transmission flow of the audio and video.
In some embodiments of the present invention, the above further includes: and if the frame packet transmission lasts without packet loss for a preset time, gradually recovering the audio signal and video signal coding types and the coding rates of the audio signal and the video signal according to the strategy of each priority sending queue of the priority sequence.
In a second aspect, an embodiment of the present application provides an audio and video transmission system for a video phone, which includes a receiving module, configured to receive an audio signal and a video signal, and encode the audio signal and the video signal to obtain a code rate value;
the synchronization module is used for generating synchronization signals, adding the synchronization signals into the received audio signals and video signals, and respectively configuring frame queue priority sending queues of audio and video according to the synchronization signals;
and the sending module is used for transmitting the plurality of frame packets and the plurality of audio and video frames from the first frame packet corresponding to the first audio and video in the sending queue and the first audio and video frame in the plurality of audio and video frames according to a preset transmission rule.
In some embodiments of the invention, the above includes: at least one memory for storing computer instructions; at least one processor in communication with the memory, wherein the at least one processor, when executing the computer instructions, causes the system to: the device comprises a receiving module, a synchronization module and a sending module.
In a third aspect, embodiments of the present application provide a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method such as any one of audio and video transmission methods for video telephony.
Compared with the prior art, the embodiment of the invention has at least the following advantages or beneficial effects:
by adding a synchronizing signal to the video signal and the audio signal and combining the audio signal and the video signal based on the synchronizing signal at the time of combining, thereby ensuring that the video signal and the audio signal received at the same time have the same synchronous signal and real-time transmission rate to obtain the code rate value of the code, so as to encode the transmitted video and audio according to the obtained bit rate value to obtain a plurality of video and audio frames, then dividing each video and audio frame to obtain a plurality of frame packets, so as to mix and transmit the plurality of frame packets, combine the transmission rate and ensure the transmission stability, when the video signals and the audio signals are combined, the video signals and the audio signals can be combined according to the synchronous signals, and the synchronism of the video signals and the audio signals in the video and audio files after subsequent combination is guaranteed, so that the use of a user is met.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a schematic diagram illustrating steps of an audio and video transmission method for a video phone according to an embodiment of the present invention;
FIG. 2 is a detailed step diagram of an audio and video transmission method for video telephony according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an audio and video transmission system module of a video telephone according to an embodiment of the present invention;
fig. 4 is an electronic device according to an embodiment of the present invention.
Icon: 10-a receiving module; 20-a synchronization module; 30-a sending module; 101-a memory; 102-a processor; 103-communication interface.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
It is to be noted that the term "comprises," "comprising," or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the individual features of the embodiments can be combined with one another without conflict.
Example 1
Referring to fig. 1, fig. 1 is a schematic diagram of steps of an audio and video transmission method of a video phone according to an embodiment of the present invention, which is shown as follows:
step S100, receiving an audio signal and a video signal, and encoding the audio signal and the video signal to obtain a code rate value;
in some embodiments, the audio signal and the video signal are acquired by a terminal having a video signal collecting and transmitting function, such as a mobile terminal or a fixed-line telephone. Or the collection of the audio signal and the video signal is realized through a camera, and the terminal sends the video information of the remote user collected by the camera to the terminal where the local user is located. And a process of obtaining a code rate value used when the video signal and the audio signal are coded according to the actual transmission rate of the terminal equipment, so that when the subsequent video signal and the audio signal are coded, the code rate value is used for coding the video data, and the size of the video frame transmitted at the actual transmission rate is obtained.
Step S110, generating a synchronous signal to be added into the received audio signal and video signal, and respectively configuring a frame queue priority sending queue of the audio and video according to the synchronous signal;
in some embodiments, when the priority transmission queue reaches a packet loss condition, all data packets in the secondary priority transmission queue are directly discarded, and no data in the secondary priority transmission queue is transmitted; simultaneously implementing the first change of the audio and video coding types, firstly obtaining the preset audio and video coding types of a coding library, then obtaining all the audio and video coding types supported by an audio and video module, setting the audio and video coding types with higher compression ratio than the preset audio and video coding types as new coding types to the coding library, and after the setting is successful, outputting audio and video streams by the coding library according to the audio and video coding types with higher compression ratio than the preset audio and video coding types;
and if the audio and video coding type with the higher compression ratio is the audio coding type with the highest compression ratio supported by the audio and video module, keeping outputting the audio and video streams with the audio and video coding type with the highest compression ratio.
If the packet loss condition is not met, judging whether the network recovery condition of the priority queue is met, and gradually recovering the descending audio and video flow strategies of the priority sending queue into the preset audio and video coding types; if yes, restoring the audio coding type and the video coding type to the preset audio and video coding type step by step; if not, ending the priority queue data packet transmission processing flow.
Step S120, a plurality of frame packets and a plurality of audio and video frames are transmitted according to a preset transmission rule from a first frame packet corresponding to a first audio and video in the transmission queue and from a first audio and video frame in the plurality of audio and video frames.
In some embodiments, when the audio signal and the video signal are combined, the audio signal and the video signal are combined according to a synchronization signal added to the audio signal and the video signal to obtain a combined audio signal and video signal, and the synchronization signal is retained in the combined audio signal and video signal. The combined audio and video signals are then recorded.
Specifically, when the audio signal and the video signal are combined, since the video signal is received in units of frames, and the width and the height of each frame are known, when the combined video signal is obtained by combining, the length or the width of each frame may be set to be twice of the maximum value of the length or the width in the video signal. For example, when the width of each frame of the combined video signal is twice the width of each frame of the video signal, the left and right sides in each frame of the combined video signal are the corresponding image frames, respectively.
And the terminal where the local user is located aligns the combined video signal and the combined audio signal and video signal which are the same as the synchronous signal according to the recorded combined audio signal and video signal and the synchronous signal reserved by the combined audio signal and video signal, and combines the aligned video signal and audio signal to generate the audio signal and video signal.
Example 2
Referring to fig. 2, fig. 2 is a detailed step diagram of an audio and video transmission method of a video phone according to an embodiment of the present invention, which is shown as follows:
step S200, according to the size and the transmission time of the audio signal and the video signal, the size of each frame when the audio signal and the video signal reach the preset frame rate after being coded is obtained, and the code rate value is obtained according to the size of each frame.
And step S210, the audio and video code rate values output after the audio and video are coded are processed through a video callback function and an audio callback function, the video callback function distinguishes video key frames and video non-key frames according to the frame types returned by the coding library, and the audio callback function puts the audio into an audio frame sending queue.
Step S220, adding the generated synchronization signal to the received audio signal and video signal, and combining the received audio signal and video signal with the synchronization signal according to the synchronization signal added to the audio signal and video signal to obtain a combined audio signal and video signal.
And step S230, according to the synchronization signal added to the audio signal and the video signal and reserving the synchronization signal, combining the received audio signal and the video signal to obtain a combined audio signal and video signal and reserving the synchronization signal, and according to the synchronization signal reserved in the combined audio signal and video signal, combining the combined audio signal and video signal.
Step S240, recording the packet loss number and the continuous packet loss time of the transmission queue, and if the transmission queue reaches the packet loss condition, discarding the subsequent frame packets of the audio and video of the priority transmission queue, and actively reducing the transmission flow of the audio and video.
Step S250, if the frame packet transmission continues without packet loss for a preset time, the encoding types of the audio signal and the video signal and the encoding rates of the audio signal and the video signal are gradually recovered according to the policy of each priority transmission queue of the priority order.
In some embodiments, the received audio and video is encoded and output to obtain audio frames, video key frames and video non-key frames; the streaming media control is used for creating three sending queue packet loss detections for accommodating audio frames, video key frames and video non-key frames respectively, and detecting the packet loss number of each sending queue; carrying out priority sequencing on each sending queue, and presetting packet loss conditions and corresponding processing modes according to the packet loss number of each sending queue;
the method is used for processing each sending queue according to the processing mode and comprises the following steps: and if a certain priority sending queue reaches a packet loss condition, discarding the video stream of the subsequent priority sending queue, and actively reducing the transmission flow of the audio stream or the video stream.
Example 3
Referring to fig. 3, fig. 3 is a schematic diagram of an audio and video transmission system module of a video phone according to an embodiment of the present invention, which is shown as follows:
the receiving module 10 is configured to receive an audio signal and a video signal, and encode the audio signal and the video signal to obtain a code rate value;
a synchronization module 20, configured to generate a synchronization signal, add the synchronization signal to the received audio signal and video signal, and configure frame queue priority sending queues for audio and video according to the synchronization signal;
and a sending module 30, configured to transmit the plurality of frame packets and the plurality of audio and video frames according to a preset transmission rule from a first frame packet corresponding to a first audio and video in the sending queue and from a first audio and video frame in the plurality of audio and video frames.
As shown in fig. 4, an embodiment of the present application provides an electronic device, which includes a memory 101 for storing one or more programs; a processor 102. The one or more programs, when executed by the processor 102, implement the method of any of the first aspects as described above.
Also included is a communication interface 103, and the memory 101, processor 102 and communication interface 103 are electrically connected to each other, directly or indirectly, to enable transfer or interaction of data. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The memory 101 may be used to store software programs and modules, and the processor 102 executes the software programs and modules stored in the memory 101 to thereby execute various functional applications and data processing. The communication interface 103 may be used for communicating signaling or data with other node devices.
The Memory 101 may be, but is not limited to, a Random Access Memory 101 (RAM), a Read Only Memory 101 (ROM), a Programmable Read Only Memory 101 (PROM), an Erasable Read Only Memory 101 (EPROM), an electrically Erasable Read Only Memory 101 (EEPROM), and the like.
The processor 102 may be an integrated circuit chip having signal processing capabilities. The Processor 102 may be a general-purpose Processor 102, including a Central Processing Unit (CPU) 102, a Network Processor 102 (NP), and the like; but may also be a Digital Signal processor 102 (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware components.
In the embodiments provided in the present application, it should be understood that the disclosed method and system and method can be implemented in other ways. The method and system embodiments described above are merely illustrative, for example, the flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
In another aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, which, when executed by the processor 102, implements the method according to any one of the first aspect described above. The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory 101 (ROM), a Random Access Memory 101 (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In summary, the audio and video transmission method and system for video telephony provided by the embodiments of the present application add the synchronization signal to the video signal and the audio signal, and combine the audio signal and the video signal according to the synchronization signal during combining, so as to ensure that the video signal and the audio signal received at the same time have the same synchronization signal and real-time transmission rate, obtain a coded code rate value, so as to encode the transmitted video and audio according to the obtained code rate value, obtain a plurality of video and audio frames, divide each video and audio frame, obtain a plurality of frame packets, so as to mix and transmit the plurality of frame packets, combine the transmission rate and ensure the transmission stability, and during combining, combine the video signal and the audio signal according to the synchronization signal, so as to ensure the synchronization of the video signal and the audio signal in the subsequent combined video and audio file, thereby satisfying the use of the user.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (10)
1. A method for audio and video transmission for video telephony, comprising:
receiving an audio signal and a video signal, and coding the audio signal and the video signal to obtain a code rate value;
generating a synchronous signal, adding the synchronous signal into the received audio signal and video signal, and respectively configuring a frame queue priority sending queue of audio and video according to the synchronous signal;
and transmitting the plurality of frame packets and the plurality of audio and video frames according to a preset transmission rule from a first frame packet corresponding to the first audio and video in the sending queue and a first audio and video frame in the plurality of audio and video frames.
2. The method of claim 1, wherein the receiving the audio signal and the video signal and encoding the audio signal and the video signal to obtain the bitrate value comprises:
and obtaining the size of each frame when the audio signal and the video signal reach a preset frame rate after being coded according to the size and the transmission time of the audio signal and the video signal, and obtaining a code rate value according to the size of each frame.
3. The audio and video transmission method of a video phone according to claim 2, further comprising:
the audio and video code rate values output after the audio and video are coded are processed through a video callback function and an audio callback function, the video callback function distinguishes video key frames and video non-key frames according to frame types returned by a coding library, and the audio callback function puts the audio into an audio frame sending queue.
4. The audio and video transmission method of a video phone according to claim 1, wherein said generating a synchronization signal to be added to the received audio signal and video signal and configuring the frame queue priority transmission queues for audio and video, respectively, according to the synchronization signal comprises:
and adding the generated synchronous signals into the received audio signals and video signals, and combining the received audio signals, video signals and synchronous signals according to the synchronous signals added in the audio signals and video signals to obtain combined audio signals and video signals.
5. The audio and video transmission method of a video phone according to claim 4, further comprising:
and according to the synchronous signal added in the audio signal and the video signal and reserving the synchronous signal, combining the received audio signal and the video signal to obtain a combined audio signal and video signal and reserving the synchronous signal, and combining the combined audio signal and the video signal according to the reserved synchronous signal in the combined audio signal and video signal.
6. The audio and video transmission method of a video phone according to claim 1, wherein said transmitting a plurality of frame packets and a plurality of audio and video frames according to a predetermined transmission rule from a first frame packet corresponding to a first audio and video in the transmission queue and a first audio and video frame in the plurality of audio and video frames comprises:
and recording the packet loss number and the continuous packet loss time of the transmission queue, and if the transmission queue reaches the packet loss condition, discarding the subsequent frame packets of the audio and video of the priority transmission queue and actively reducing the transmission flow of the audio and video.
7. The audio and video transmission method of a video phone according to claim 6, further comprising:
and if the frame packet transmission lasts without packet loss for a preset time, gradually recovering the audio signal and video signal coding types and the coding rates of the audio signal and the video signal according to the strategy of each priority sending queue of the priority sequence.
8. An audio and video transmission system for video telephony, comprising:
the receiving module is used for receiving the audio signal and the video signal and coding the audio signal and the video signal to obtain a code rate value;
the synchronization module is used for generating synchronization signals, adding the synchronization signals into the received audio signals and video signals, and respectively configuring frame queue priority sending queues of audio and video according to the synchronization signals;
and the sending module is used for transmitting the plurality of frame packets and the plurality of audio and video frames from the first frame packet corresponding to the first audio and video in the sending queue and the first audio and video frame in the plurality of audio and video frames according to a preset transmission rule.
9. The audio and video transmission system for video telephony of claim 8, comprising:
at least one memory for storing computer instructions;
at least one processor in communication with the memory, wherein the at least one processor, when executing the computer instructions, causes the system to perform: the device comprises a receiving module, a synchronization module and a sending module.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110721246.8A CN113452946A (en) | 2021-06-28 | 2021-06-28 | Audio and video transmission method and system for video telephone |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110721246.8A CN113452946A (en) | 2021-06-28 | 2021-06-28 | Audio and video transmission method and system for video telephone |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113452946A true CN113452946A (en) | 2021-09-28 |
Family
ID=77813526
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110721246.8A Pending CN113452946A (en) | 2021-06-28 | 2021-06-28 | Audio and video transmission method and system for video telephone |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113452946A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116233024A (en) * | 2022-12-22 | 2023-06-06 | 中国联合网络通信集团有限公司 | Data packet determining method, device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102724559A (en) * | 2012-06-13 | 2012-10-10 | 天脉聚源(北京)传媒科技有限公司 | Method and system for synchronizing encoding of videos and audios |
| CN102932673A (en) * | 2011-08-11 | 2013-02-13 | 阿里巴巴集团控股有限公司 | Video signal and audio signal transmission and synthesis method, system and device |
| CN105847182A (en) * | 2016-04-18 | 2016-08-10 | 武汉烽火众智数字技术有限责任公司 | Method and system thereof for preferentially transmitting audio in audio and video system |
| CN109756789A (en) * | 2018-12-28 | 2019-05-14 | 视联动力信息技术股份有限公司 | A kind of loss treating method and system of audio, video data packet |
| CN112272313A (en) * | 2020-12-23 | 2021-01-26 | 深圳乐播科技有限公司 | HID (high intensity discharge) -based audio and video transmission method and device and computer readable storage medium |
-
2021
- 2021-06-28 CN CN202110721246.8A patent/CN113452946A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102932673A (en) * | 2011-08-11 | 2013-02-13 | 阿里巴巴集团控股有限公司 | Video signal and audio signal transmission and synthesis method, system and device |
| CN102724559A (en) * | 2012-06-13 | 2012-10-10 | 天脉聚源(北京)传媒科技有限公司 | Method and system for synchronizing encoding of videos and audios |
| CN105847182A (en) * | 2016-04-18 | 2016-08-10 | 武汉烽火众智数字技术有限责任公司 | Method and system thereof for preferentially transmitting audio in audio and video system |
| CN109756789A (en) * | 2018-12-28 | 2019-05-14 | 视联动力信息技术股份有限公司 | A kind of loss treating method and system of audio, video data packet |
| CN112272313A (en) * | 2020-12-23 | 2021-01-26 | 深圳乐播科技有限公司 | HID (high intensity discharge) -based audio and video transmission method and device and computer readable storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116233024A (en) * | 2022-12-22 | 2023-06-06 | 中国联合网络通信集团有限公司 | Data packet determining method, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10819766B2 (en) | Voice encoding and sending method and apparatus | |
| US7734104B2 (en) | Image coding apparatus, image decoding apparatus and image processing system | |
| CN110784745A (en) | Video transmission method, device, system, equipment and storage medium | |
| US6603501B1 (en) | Videoconferencing using distributed processing | |
| CN110784718B (en) | Video data encoding method, apparatus, device and storage medium | |
| CN102984495A (en) | Video image processing method and device | |
| CN111741248B (en) | Data transmission method, device, terminal equipment and storage medium | |
| CN109618170B (en) | D2D real-time video streaming transmission method based on network coding | |
| JP7356581B2 (en) | Information processing methods, devices, equipment and computer readable storage media | |
| CN111641804A (en) | Video data processing method and device, terminal, camera and video conference system | |
| CN113452946A (en) | Audio and video transmission method and system for video telephone | |
| CN111970565A (en) | Video data processing method and device, electronic equipment and storage medium | |
| CN113490055B (en) | Data processing method and device | |
| JP7136084B2 (en) | Communication device, media delivery system, media delivery method and program | |
| CN109660826B (en) | Video transmission method, device, equipment and storage medium | |
| CN112929704A (en) | Data transmission method, device, electronic equipment and storage medium | |
| CN115209189B (en) | Video stream transmission method, system, server and storage medium | |
| US20230156064A1 (en) | Method, system and computer program product for initiating downspeeding in a videoconferencing session | |
| CN113141352B (en) | Multimedia data transmission method and device, computer equipment and storage medium | |
| CN115442615A (en) | Video coding method and device, electronic equipment and storage medium | |
| CN111030985A (en) | Data packet sending method and device | |
| US20030099234A1 (en) | Multi-point communication method | |
| JP2002344937A (en) | Quality control assurance method and quality control assurance device, and network connection device | |
| RU2674463C1 (en) | Digital signals multiplexing method with the information multi-modal representation | |
| CN114245196B (en) | Screen recording and stream pushing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210928 |
|
| RJ01 | Rejection of invention patent application after publication |