CN111641804A - Video data processing method and device, terminal, camera and video conference system - Google Patents
Video data processing method and device, terminal, camera and video conference system Download PDFInfo
- Publication number
- CN111641804A CN111641804A CN202010549862.5A CN202010549862A CN111641804A CN 111641804 A CN111641804 A CN 111641804A CN 202010549862 A CN202010549862 A CN 202010549862A CN 111641804 A CN111641804 A CN 111641804A
- Authority
- CN
- China
- Prior art keywords
- camera
- video stream
- encoding
- coding
- data processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
Abstract
The application provides a video data processing method and device, a terminal, a camera, a video conference system and a computer storage medium, wherein the method comprises the following steps: collecting an encoded video stream from a camera; the coded video stream is a code stream obtained by coding the shot image data by the camera; detecting the encoding parameters of the encoded video stream based on a virtual encoder, and feeding back the encoding parameters to the camera so as to control the encoding process by the camera; and transmitting the coded video stream to a receiving end in a transparent transmission mode. According to the technical scheme, the bandwidth requirement that the camera transmits image data to the terminal equipment is reduced, and meanwhile, the hardware performance requirement on the terminal equipment is lowered, so that some low-performance terminal equipment can support high-definition video data processing, and the popularization and application of a video conference technology are promoted.
Description
Technical Field
The present application relates to the field of internet technologies, and in particular, to a video data processing method and apparatus, a terminal, a camera, a video conference system, and a computer storage medium.
Background
With the development of internet technology, video conference system technology is widely applied to business and social scenes, and taking a common video conference system as an example, the common video conference system is generally formed by combining terminal equipment (such as a flat-panel television all-in-one machine) with a camera, image data is acquired by the camera and transmitted to the terminal equipment, and the terminal equipment carries out video conference through an access network.
In the current video conference system scheme, when a video call is to be established, the processing flow of a general terminal device is as follows: video capture, encoding, transmission, decoding, rendering, etc. In the processing process, a certain requirement is placed on the processing capacity of the terminal device, since the current video conference generally adopts high-definition video data, such as 1080p, a higher bandwidth is required for data transmission of a camera, a higher requirement is placed on the hardware performance of the terminal device, and for some low-performance terminal devices, high-definition video data processing cannot be supported, so that the technical application of the video conference is influenced.
Disclosure of Invention
The present application aims to address the technical defects, and provide a video data processing method, an apparatus, a video conference system, and a storage medium, so as to reduce the bandwidth requirement for data transmission of a camera and reduce the hardware performance requirement for a terminal device.
The application provides a video data processing method, which comprises the following steps:
collecting an encoded video stream from a camera; the coded video stream is a code stream obtained by coding the shot image data by the camera;
detecting the encoding parameters of the encoded video stream based on a virtual encoder, and feeding back the encoding parameters to the camera so as to control the encoding process by the camera;
and transmitting the coded video stream to a receiving end in a transparent transmission mode.
In one embodiment, the step of detecting the encoding parameters of the encoded video stream based on the virtual encoder and feeding back the encoding parameters to the camera so that the camera controls the encoding process includes:
counting the number of protection data packets in the coded video stream within a period of time in the past based on the estimation result of the current network bandwidth by the virtual encoder;
calculating the current coding rate according to the number of the protection data packets;
and feeding back the coding rate to the camera through the virtual encoder, and setting the coding rate.
In one embodiment, the video data processing method further includes:
when the coded video stream is input into the virtual encoder, detecting the current performance parameters of the terminal equipment; and carrying out selective packet loss processing on the coded video stream according to the performance parameters.
In one embodiment, the packet loss processing includes:
if the coded video stream has priority, discarding the video frames with low priority and sending the remaining video frames with high priority;
if the encoded video stream does not have priority, discarding the current video frame and requesting a key video frame from the camera.
In one embodiment, the video data processing method further includes:
and reading the coded video stream collected from the camera, decoding the coded video stream locally by using a decoding module, and rendering video data obtained by decoding to a local display for displaying.
A video data processing apparatus comprising:
the acquisition unit is used for acquiring the coded video stream from the camera; the coded video stream is a code stream obtained by coding the shot image data by the camera;
the first control unit is used for detecting the encoding parameters of the encoded video stream based on a virtual encoder and feeding back the encoding parameters to the camera so as to enable the camera to control the encoding process;
and the transmission unit is used for transmitting the coded video stream to a receiving end in a transparent transmission mode.
A terminal comprising one or more processors, memory, and a display;
the terminal is connected with the camera;
one or more application programs are stored in the memory of the terminal device, wherein the application programs are configured to be executed by the processor, and the one or more programs comprise the steps of the video data processing method when executed.
The application also provides a video data processing method, which comprises the following steps:
coding the shot image data to obtain a coded video stream;
sending the coded video stream to a terminal device, and receiving a coding parameter fed back by the terminal device; the terminal equipment detects the encoding parameters of the encoded video stream based on a virtual encoder and transmits the encoded video stream to a receiving end in a transparent transmission mode;
and controlling the encoding process according to the encoding parameters.
In one embodiment, the step of encoding the captured image data to obtain an encoded video stream includes:
and coding the shot image data through a built-in coding module to obtain a coded video stream with a set format.
In one embodiment, the step of controlling the encoding process according to the encoding parameters comprises:
receiving, by an encoding module, an encoding rate fed back by the virtual encoder; the coding rate is calculated by the virtual encoder according to the estimation result of the current network bandwidth and the number of protection data packets in the coded video stream counted in a past period of time;
and setting the coding rate to a coding module, and controlling the coding rate of the coding module.
In one embodiment, the video data processing method further includes:
and outputting the coded video stream to a decoding module of the terminal equipment, so that the decoding module decodes the coded video stream, and renders video data obtained by decoding to a display of the terminal equipment for display.
The present application also provides a video data processing apparatus, including:
the encoding unit is used for encoding the shot image data to obtain an encoded video stream;
the feedback unit is used for sending the coded video stream to terminal equipment and receiving coding parameters fed back by the terminal equipment; the terminal equipment detects the encoding parameters of the encoded video stream based on a virtual encoder and transmits the encoded video stream to a receiving end in a transparent transmission mode;
and the second control unit is used for controlling the encoding process according to the encoding parameters.
The application also provides a camera which comprises a camera module, one or more processors and a memory;
the camera is connected with the terminal equipment;
one or more applications are stored in the memory of the camera, wherein the applications are configured to be executed by the processor, and the one or more programs when executed comprise the steps of the video data processing method described above.
A video conferencing system, comprising: the camera and the terminal are provided.
A computer device storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements a video data processing method as described above.
According to the video data processing method and device, the terminal, the camera, the video conference system and the computer storage medium, the camera is used for coding the shot image data to obtain a coded video stream, the coded video stream is collected from the camera, the coding parameters of the coded video stream are detected based on the virtual encoder, and the coding parameters are fed back to the camera to control the coding process; and simultaneously transmitting the coded video stream to a receiving end in a transparent transmission mode. According to the technical scheme, the encoding process is set in the camera to be achieved, the bandwidth requirement that the camera transmits image data to the terminal device is reduced, meanwhile, the terminal device directly collects encoded video streams from the camera, the virtual encoder is utilized to achieve the purpose that encoding parameters are fed back to the camera to control the encoding process, and the encoded video streams are transmitted to the receiving end in a transparent mode, so that the hardware performance requirement on the terminal device is lowered, some low-performance terminal devices can support high-definition video data processing, and popularization and application of the video conference technology are promoted.
Furthermore, the current coding rate is calculated according to the pre-estimation of the current network bandwidth, and the coding rate of the coding module of the camera is set by feeding back the coding rate to the camera through the virtual encoder, so that the coding rate of the camera can be controlled according to the transmission network bandwidth, and the stability of the transmission of the coded video stream is ensured.
Furthermore, selective packet loss processing is performed on the coded video stream by detecting the current performance parameters of the terminal device, and the packet loss processing can be performed by combining with the priority; the smoothness and integrity of transmission can be ensured.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a topology diagram of a video conferencing system;
FIG. 2 is a flow diagram of a video data processing method of an embodiment;
FIG. 3 is a schematic diagram of a video data processing process according to an embodiment;
FIG. 4 is a schematic diagram of a video data processing procedure according to another embodiment;
FIG. 5 is a flow diagram of coding rate control for one embodiment;
FIG. 6 is a block diagram of a video data processing apparatus according to an embodiment;
FIG. 7 is a flow chart illustrating a video data processing method according to another embodiment;
FIG. 8 is a block diagram of a video data processing apparatus according to another embodiment;
figure 9 is a schematic diagram of a video conferencing system configuration, according to an embodiment.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.
It will be understood by those within the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The video conference scene applied in the technical scheme of the present application is generated by combining the topological diagram of the video conference system of fig. 1, as shown in the figure, when a video conference is performed, image data is collected by the camera a, the image data is transmitted to the terminal device a, rendering and display can be performed locally through the terminal device a, meanwhile, the terminal device a encodes the image data to obtain a video stream, the video stream is transmitted to the terminal device B through a network, the terminal device B decodes the video stream and renders the video stream on a display, and similarly, the camera B collects corresponding video data, and for the format of the encoded video stream, H264 encoding can be adopted.
Referring to the above topological diagram, in the technical solution of the present application, the camera a implements encoding to obtain an encoded video stream, which is transmitted to the terminal device a and then transmitted to the terminal device B.
Referring to fig. 2, fig. 2 is a schematic flowchart of a video data processing method according to an embodiment, where the flowchart is mainly implemented on a terminal device, and includes the following steps:
s110, collecting coded video stream from a camera; the coded video stream is a code stream obtained by coding the shot image data by the camera.
Referring to fig. 3, fig. 3 is a schematic view of a video data processing process according to an embodiment, taking a terminal a as an example, where an a camera acquires image data, and then encodes the image data to obtain an encoded video stream, and the terminal a acquires the encoded video stream through the a camera. Because the terminal device A collects the coded video stream, the bandwidth requirement for the camera A to transmit the image data to the terminal device is reduced for the transmission between the camera A and the terminal device A. Taking 1080P high definition video data as an example, 1080P resolution image data is transmitted, the requirement on the bandwidth of an a camera is extremely high, and most camera devices are difficult to implement.
And S120, detecting the encoding parameters of the encoded video stream based on a virtual encoder, and feeding back the encoding parameters to the camera so as to control the encoding process by the camera.
Taking the terminal device a as an example, the terminal device a detects the relevant encoding parameters of the encoded video stream through the set virtual encoder, where the virtual encoder performs encoding control and transmission control functions instead of playing a role in encoding, feeds the encoding parameters back to the camera a, and the camera a performs encoding control on the encoding process and adjusts the camera to output the encoded video stream with the corresponding code rate and resolution.
And S130, transmitting the coded video stream to a receiving end in a transparent transmission mode.
As in the above example, the a terminal device transmits the collected encoded video stream to the receiving end in a transparent transmission manner, and by adopting the transparent transmission manner, the a terminal device does not need to encode the image data, so that the hardware performance requirement on the a terminal device can be reduced, the system can process and transmit high-definition videos with various resolutions, such as 1080P high-definition video data, the use threshold is reduced, and most terminal devices can be supported.
According to the technical scheme of the embodiment, the shot image data is coded through the camera to obtain a coded video stream, the coded video stream is collected from the camera, the coding parameters of the coded video stream are detected based on the virtual coder, and the coded video stream is fed back to the camera to control the coding process according to the coding parameters; and simultaneously transmitting the coded video stream to a receiving end in a transparent transmission mode. According to the technical scheme, the encoding process is set in the camera to be achieved, the bandwidth requirement that the camera transmits image data to the terminal device is reduced, meanwhile, the terminal device directly collects encoded video streams from the camera, the virtual encoder is utilized to achieve the purpose that encoding parameters are fed back to the camera to control the encoding process, and the encoded video streams are transmitted to the receiving end in a transparent mode, so that the hardware performance requirement on the terminal device is lowered, some low-performance terminal devices can support high-definition video data processing, and popularization and application of the video conference technology are promoted.
In order to make the technical solutions of the present application clearer, the following describes the solutions of the present application with reference to several embodiments.
Referring to fig. 4, fig. 4 is a schematic diagram illustrating a video data processing procedure according to another embodiment; as an embodiment, a camera is provided with a built-in encoding module, and captured image data is encoded by the encoding module to obtain an encoded video stream with a set format.
The encoding module may be a hardware module or may be based on a software program algorithm module, in the figure, a camera and a terminal a are taken as examples, encoding processing on image data shot by the camera can be realized by setting the encoding module in the camera, and meanwhile, the encoding module may receive a virtual encoder feedback encoding parameter of a terminal device a to realize control on an encoding process.
With continuing reference to fig. 4, in the video conference system, based on the video data processing method provided by the present application, when displaying image data locally, the following technical solution is adopted:
and reading the coded video stream collected from the camera, decoding the coded video stream locally by using a decoding module, and rendering video data obtained by decoding to a local display for displaying.
As shown in the figure, since the encoding process is executed in the a camera, the encoded video stream collected by the a terminal device is locally decoded by the decoder and locally rendered to the display, although the decoding process is added compared with the conventional method, when the a terminal device has high-definition video decoding capability, the decoding parameters can be set according to the actual situation, and therefore the display process of the a terminal device on the image data is not affected.
In one embodiment, as further illustrated, the step of transmitting the encoded video stream to the receiving end in a transparent transmission manner of step S30 may transmit the encoded video stream to the receiving end in a transparent transmission manner through the network; and the receiving end decodes the coded video stream and renders the video data obtained by decoding to a receiving end display for displaying.
As shown in the figure, the virtual encoder of the a terminal device transmits the encoded video stream to the B terminal device in a transparent transmission mode through the transmission module, and the B terminal device decodes the encoded video stream through the decoder and renders the encoded video stream to the display for displaying.
Based on the video transmission method provided by the application, in order to control the transmission process, the application also provides the following embodiments.
In an embodiment, referring to fig. 5, fig. 5 is an encoding rate control flowchart of an embodiment, and the step S120 of detecting an encoding parameter of the encoded video stream based on a virtual encoder and feeding back the encoding parameter to the camera according to the encoding parameter, so that the camera controls an encoding process may specifically include the following steps:
s201, counting the number of protection data packets in the coded video stream within a period of time in the past based on the estimation result of the virtual encoder according to the current network bandwidth;
s202, calculating the current coding rate according to the number of the protection data packets;
s203, feeding the coding rate back to the camera through the virtual encoder, and setting the coding rate.
Taking terminal equipment A as an example, the current network bandwidth can be estimated through a virtual encoder, the transmission matching code rate under the current network bandwidth condition can be calculated by counting the number of protection data packets in the coded video stream in a past period of time, the virtual encoder feeds the calculated coding code rate back to a coding module of a camera, and the coding module codes image data according to the coding code rate and outputs a code stream with the corresponding code rate. By the scheme of the embodiment, the control of the code rate is realized, and the stability of the transmission of the coded video stream can be ensured.
In one embodiment, in order to better match the transmission speed, during the video data processing, the packet loss processing may also be performed on the encoded video stream by the virtual encoder. Specifically, when the encoded video stream is input into the virtual encoder, the current performance parameters of the terminal device are detected; and carrying out selective packet loss processing on the coded video stream according to the performance parameters.
The scheme for packet loss processing may include the following:
if the coded video stream has priority, discarding the video frames with low priority and sending the remaining video frames with high priority; if the encoded video stream does not have priority, discarding the current video frame and requesting a key video frame from the camera.
Taking the terminal device a as an example, when the encoded video stream output by the camera a reaches the virtual encoder, the terminal device a detects the current performance parameters of the terminal device, and performs selective packet loss processing on the encoded video stream as a reference; by adopting the scheme for processing the frame loss through the packet loss processing and the priority thereof, the transmission fluency and integrity can be ensured.
An exemplary embodiment of a video data processing device, which is used in a terminal, is described below.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a video data processing apparatus according to an embodiment, including:
an acquisition unit 110 for acquiring an encoded video stream from a camera; the coded video stream is a code stream obtained by coding the shot image data by the camera;
a first control unit 120, configured to detect an encoding parameter of the encoded video stream based on a virtual encoder, and feed back the encoding parameter to the camera according to the encoding parameter, so that the camera controls an encoding process;
a transmitting unit 130, configured to transmit the encoded video stream to a receiving end in a transparent transmission manner.
According to the video data processing device, a camera is used for coding shot image data to obtain a coded video stream, the acquisition unit 110 acquires the coded video stream from the camera, the first control unit 120 detects coding parameters of the coded video stream based on the virtual encoder and feeds the coding parameters back to the camera to control a coding process; meanwhile, the transmission unit 130 transmits the encoded video stream to the receiving end in a transparent transmission manner. According to the technical scheme, the encoding process is set in the camera to be achieved, the bandwidth requirement that the camera transmits image data to the terminal device is reduced, meanwhile, the terminal device directly collects encoded video streams from the camera, the virtual encoder is utilized to achieve the purpose that encoding parameters are fed back to the camera to control the encoding process, and the encoded video streams are transmitted to the receiving end in a transparent mode, so that the hardware performance requirement on the terminal device is lowered, some low-performance terminal devices can support high-definition video data processing, and popularization and application of the video conference technology are promoted.
The following describes a video data processing method scheme of another embodiment.
Referring to fig. 7, fig. 7 is a schematic flowchart of a video data processing method according to another embodiment, where the flowchart is mainly implemented on a camera device, and includes the following steps:
and S210, coding the shot image data to obtain a coded video stream.
Referring to fig. 3, taking a terminal a as an example, an a camera acquires image data, and then encodes the image data to obtain an encoded video stream, and the terminal a acquires the encoded video stream through the a camera.
The terminal device A acquires the coded video stream, so that the bandwidth requirement for transmitting the image data to the terminal device by the camera A is reduced for the transmission between the camera A and the terminal device A. Taking 1080P high definition video data as an example, 1080P resolution image data is transmitted, the requirement on the bandwidth of an a camera is extremely high, and most camera devices are difficult to implement.
As an embodiment, the captured image data may be encoded by a built-in encoding module to obtain an encoded video stream with a set format.
S220, sending the coded video stream to a terminal device, and receiving a coding parameter fed back by the terminal device; the terminal device detects the encoding parameters of the encoded video stream based on the virtual encoder, and transmits the encoded video stream to a receiving end in a transparent transmission mode.
As an embodiment, as shown in fig. 4, a camera is provided with a built-in encoding module, and the captured image data is encoded by the encoding module to obtain an encoded video stream with a set format.
The encoding module may be a hardware module or may be based on a software program algorithm module, in the figure, a camera and a terminal a are taken as examples, encoding processing on image data shot by the camera can be realized by setting the encoding module in the camera, and meanwhile, the encoding module may receive a virtual encoder feedback encoding parameter of a terminal device a to realize control on an encoding process.
As shown in fig. 4, the a terminal device transmits the collected encoded video stream to the receiving end in a transparent transmission manner, and the a terminal device does not need to encode image data by adopting the transparent transmission manner, so that the hardware performance requirement on the a terminal device can be reduced, the system can process and transmit high-definition videos with various resolutions, such as 1080P high-definition video data, the use threshold is reduced, and most terminal devices can be supported.
And S230, controlling the encoding process according to the encoding parameters.
Taking the terminal device a as an example, the terminal device a detects the relevant encoding parameters of the encoded video stream through the set virtual encoder, where the virtual encoder performs encoding control and transmission control functions instead of playing a role in encoding, feeds the encoding parameters back to the camera a, and the camera a performs encoding control on the encoding process and adjusts the camera to output the encoded video stream with the corresponding code rate and resolution.
In one embodiment, the step of controlling the encoding process according to the encoding parameters in step S230 may include the following steps:
(1) receiving, by an encoding module, an encoding rate fed back by the virtual encoder; and the coding rate is calculated by the virtual encoder according to the estimation result of the current network bandwidth and the number of the protection data packets in the coded video stream counted in a past period of time.
(2) And setting the coding rate to a coding module, and controlling the coding rate of the coding module.
For example, the terminal device a detects the relevant encoding parameters of the encoded video stream through the set virtual encoder, where the virtual encoder performs encoding control and transmission control functions instead of playing a role in encoding, feeds the encoding parameters back to the camera a, and the camera a performs encoding control on the encoding process of the camera a and adjusts the camera to output the encoded video stream corresponding to the code rate and resolution.
According to the technical scheme of the embodiment, the shot image data is coded through the camera to obtain a coded video stream, the terminal equipment collects the coded video stream from the camera, detects the coding parameters of the coded video stream based on the virtual encoder, and feeds the coding parameters back to the camera to control the coding process according to the coding parameters; and simultaneously transmitting the coded video stream to a receiving end in a transparent transmission mode. According to the technical scheme, the encoding process is set in the camera to be achieved, the bandwidth requirement that the camera transmits image data to the terminal device is reduced, meanwhile, the terminal device directly collects encoded video streams from the camera, the virtual encoder is utilized to achieve the purpose that encoding parameters are fed back to the camera to control the encoding process, and the encoded video streams are transmitted to the receiving end in a transparent mode, so that the hardware performance requirement on the terminal device is lowered, some low-performance terminal devices can support high-definition video data processing, and popularization and application of the video conference technology are promoted.
In an embodiment, the video data processing method according to the application may further output the encoded video stream to a decoding module of the terminal device, so that the decoding module decodes the encoded video stream, and renders video data obtained by decoding to a display of the terminal device for display.
An embodiment of the video data processing apparatus is set forth below.
Referring to fig. 8, fig. 8 is a schematic structural diagram of a video data processing apparatus according to an embodiment, including:
an encoding unit 210 for encoding the captured image data to obtain an encoded video stream;
a feedback unit 220, configured to send the encoded video stream to a terminal device, and receive an encoding parameter fed back by the terminal device; the terminal equipment detects the encoding parameters of the encoded video stream based on a virtual encoder and transmits the encoded video stream to a receiving end in a transparent transmission mode;
a second control unit 230, configured to control the encoding process according to the encoding parameter.
Camera embodiments are set forth below.
The camera provided by the application comprises a camera module, one or more processors and a memory; the camera is connected with the terminal equipment; one or more application programs are stored in the memory of the camera, wherein the application programs are configured to be executed by the processor, and the one or more programs when executed comprise the steps of the video data processing method of any of the above embodiments.
The application also provides a camera which comprises a camera module, one or more processors and a memory;
the camera is connected with the terminal equipment;
one or more applications are stored in the memory of the camera, wherein the applications are configured to be executed by the processor, and the one or more programs when executed comprise the steps of the video data processing method described above.
According to the camera, the coding module can be arranged in the camera, the coding function can be realized, so that the camera can be connected with other terminals, a coded video stream is provided for the terminal, and the terminal is not required to be coded, so that the hardware performance requirement on terminal equipment is reduced, some low-performance terminal equipment can support high-definition video data processing, and the popularization and application of a video conference technology are promoted.
An embodiment of a video conferencing system is set forth below.
Referring to fig. 9, fig. 9 is a schematic structural diagram of a video conference system according to an embodiment, including: the camera and the terminal of any embodiment.
Wherein the terminal device comprises one or more processors, memory, and a display; the camera is internally provided with a coding module;
the camera is internally provided with a coding module which is used for coding the shot image data to obtain a coded video stream and outputting the coded video stream to the electronic equipment; one or more applications are stored in the memory of the terminal, wherein the applications are configured to be executed by the processor, and the one or more programs when executed comprise the steps of the video data processing method of any of the above embodiments.
Specifically, as shown in the figure, the hardware part of the video conference system of the present application includes a camera and a terminal device, where the camera may have a built-in encoding module for encoding and outputting image data. The terminal device may include a processor, a memory, and a display, enabling the running of stored applications to enable video data processing.
Embodiments of computer device storage media are set forth below.
A computer device storage medium is provided having a computer program stored thereon, which when executed by a processor implements a video data processing method as in any of the embodiments described above.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.
Claims (15)
1. A method of processing video data, comprising the steps of:
collecting an encoded video stream from a camera; the coded video stream is a code stream obtained by coding the shot image data by the camera;
detecting the encoding parameters of the encoded video stream based on a virtual encoder, and feeding back the encoding parameters to the camera so as to control the encoding process by the camera;
and transmitting the coded video stream to a receiving end in a transparent transmission mode.
2. The method according to claim 1, wherein the step of detecting the encoding parameters of the encoded video stream based on the virtual encoder and feeding back the encoding parameters to the camera so that the camera controls the encoding process comprises:
counting the number of protection data packets in the coded video stream within a period of time in the past based on the estimation result of the current network bandwidth by the virtual encoder;
calculating the current coding rate according to the number of the protection data packets;
and feeding back the coding rate to the camera through the virtual encoder, and setting the coding rate.
3. The video data processing method according to claim 1, further comprising:
when the coded video stream is input into the virtual encoder, detecting the current performance parameters of the terminal equipment; and carrying out selective packet loss processing on the coded video stream according to the performance parameters.
4. The video data processing method according to claim 3, wherein the packet loss processing includes:
if the coded video stream has priority, discarding the video frames with low priority and sending the remaining video frames with high priority;
if the encoded video stream does not have priority, discarding the current video frame and requesting a key video frame from the camera.
5. The video data processing method according to claim 1, further comprising:
and reading the coded video stream collected from the camera, decoding the coded video stream locally by using a decoding module, and rendering video data obtained by decoding to a local display for displaying.
6. A video data processing apparatus, comprising:
the acquisition unit is used for acquiring the coded video stream from the camera; the coded video stream is a code stream obtained by coding the shot image data by the camera;
the first control unit is used for detecting the encoding parameters of the encoded video stream based on a virtual encoder and feeding back the encoding parameters to the camera so as to enable the camera to control the encoding process;
and the transmission unit is used for transmitting the coded video stream to a receiving end in a transparent transmission mode.
7. A terminal comprising one or more processors, memory, and a display;
the terminal is connected with the camera;
one or more applications are stored in a memory of the terminal device, wherein the applications are configured to be executed by the processor, the one or more programs when executed comprising the steps of the video data processing method of any of claims 1-5.
8. A method of processing video data, comprising the steps of:
coding the shot image data to obtain a coded video stream;
sending the coded video stream to a terminal device, and receiving a coding parameter fed back by the terminal device; the terminal equipment detects the encoding parameters of the encoded video stream based on a virtual encoder and transmits the encoded video stream to a receiving end in a transparent transmission mode;
and controlling the encoding process according to the encoding parameters.
9. The video data processing method according to claim 8, wherein said step of encoding the captured image data into an encoded video stream comprises:
and coding the shot image data through a built-in coding module to obtain a coded video stream with a set format.
10. The method of claim 9, wherein the step of controlling the encoding process according to the encoding parameters comprises:
receiving, by an encoding module, an encoding rate fed back by the virtual encoder; the coding rate is calculated by the virtual encoder according to the estimation result of the current network bandwidth and the number of protection data packets in the coded video stream counted in a past period of time;
and setting the coding rate to a coding module, and controlling the coding rate of the coding module.
11. The video data processing method of claim 8, further comprising:
and outputting the coded video stream to a decoding module of the terminal equipment, so that the decoding module decodes the coded video stream, and renders video data obtained by decoding to a display of the terminal equipment for display.
12. A video data processing apparatus, comprising:
the encoding unit is used for encoding the shot image data to obtain an encoded video stream;
the feedback unit is used for sending the coded video stream to terminal equipment and receiving coding parameters fed back by the terminal equipment; the terminal equipment detects the encoding parameters of the encoded video stream based on a virtual encoder and transmits the encoded video stream to a receiving end in a transparent transmission mode;
and the second control unit is used for controlling the encoding process according to the encoding parameters.
13. A camera is characterized by comprising a camera module, one or more processors and a memory;
the camera is connected with the terminal equipment;
one or more applications stored in a memory of the camera, wherein the applications are configured to be executed by the processor, the one or more programs when executed comprising the steps of the video data processing method of any of claims 8-11.
14. A video conferencing system, comprising: the terminal of claim 7 and the camera of claim 13.
15. A computer storage medium having stored thereon a computer program, characterized in that the program, when being executed by a processor, implements a video data processing method according to any one of claims 1-5 or 8-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010549862.5A CN111641804A (en) | 2020-06-16 | 2020-06-16 | Video data processing method and device, terminal, camera and video conference system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010549862.5A CN111641804A (en) | 2020-06-16 | 2020-06-16 | Video data processing method and device, terminal, camera and video conference system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111641804A true CN111641804A (en) | 2020-09-08 |
Family
ID=72330743
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010549862.5A Pending CN111641804A (en) | 2020-06-16 | 2020-06-16 | Video data processing method and device, terminal, camera and video conference system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111641804A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113038062A (en) * | 2021-03-12 | 2021-06-25 | 随锐科技集团股份有限公司 | Video communication method and system |
| CN113038165A (en) * | 2021-03-26 | 2021-06-25 | 腾讯音乐娱乐科技（深圳）有限公司 | Method, apparatus and storage medium for determining a set of coding parameters |
| CN114222169A (en) * | 2021-12-14 | 2022-03-22 | 咪咕互动娱乐有限公司 | Video stream transmission method and device and communication equipment |
| CN114339344A (en) * | 2020-10-10 | 2022-04-12 | 海信视像科技股份有限公司 | Intelligent device and video recording method |
| CN115118601A (en) * | 2022-05-26 | 2022-09-27 | 广东跃昉科技有限公司 | Transmission bandwidth allocation method and device for multi-path USB camera and computer equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080025413A1 (en) * | 2006-07-28 | 2008-01-31 | John Apostolopoulos | Selecting bit rates for encoding multiple data streams |
| CN103607564A (en) * | 2013-11-11 | 2014-02-26 | 天脉聚源(北京)传媒科技有限公司 | Camera-based video acquisition realization method and apparatus |
| CN107087138A (en) * | 2017-04-07 | 2017-08-22 | 华南理工大学 | A kind of video conferencing system based on H.265 video code model |
| CN207869239U (en) * | 2017-12-28 | 2018-09-14 | 天津益华微电子有限公司 | A kind of portable 4K high resolution audio and videos live broadcast system |
| CN109729439A (en) * | 2019-01-11 | 2019-05-07 | 北京三体云联科技有限公司 | Method for real-time video transmission |
-
2020
- 2020-06-16 CN CN202010549862.5A patent/CN111641804A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080025413A1 (en) * | 2006-07-28 | 2008-01-31 | John Apostolopoulos | Selecting bit rates for encoding multiple data streams |
| CN103607564A (en) * | 2013-11-11 | 2014-02-26 | 天脉聚源(北京)传媒科技有限公司 | Camera-based video acquisition realization method and apparatus |
| CN107087138A (en) * | 2017-04-07 | 2017-08-22 | 华南理工大学 | A kind of video conferencing system based on H.265 video code model |
| CN207869239U (en) * | 2017-12-28 | 2018-09-14 | 天津益华微电子有限公司 | A kind of portable 4K high resolution audio and videos live broadcast system |
| CN109729439A (en) * | 2019-01-11 | 2019-05-07 | 北京三体云联科技有限公司 | Method for real-time video transmission |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114339344A (en) * | 2020-10-10 | 2022-04-12 | 海信视像科技股份有限公司 | Intelligent device and video recording method |
| CN114339344B (en) * | 2020-10-10 | 2024-03-05 | 海信视像科技股份有限公司 | Intelligent device and video recording method |
| CN113038062A (en) * | 2021-03-12 | 2021-06-25 | 随锐科技集团股份有限公司 | Video communication method and system |
| CN113038062B (en) * | 2021-03-12 | 2022-09-09 | 随锐科技集团股份有限公司 | Video communication method and system |
| CN113038165A (en) * | 2021-03-26 | 2021-06-25 | 腾讯音乐娱乐科技（深圳）有限公司 | Method, apparatus and storage medium for determining a set of coding parameters |
| CN114222169A (en) * | 2021-12-14 | 2022-03-22 | 咪咕互动娱乐有限公司 | Video stream transmission method and device and communication equipment |
| CN114222169B (en) * | 2021-12-14 | 2023-10-20 | 咪咕互动娱乐有限公司 | Video streaming method, communication device and computer readable storage medium |
| CN115118601A (en) * | 2022-05-26 | 2022-09-27 | 广东跃昉科技有限公司 | Transmission bandwidth allocation method and device for multi-path USB camera and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111641804A (en) | Video data processing method and device, terminal, camera and video conference system | |
| US8774283B2 (en) | Information processing device and method | |
| US9628834B2 (en) | Method and system for playback of motion video | |
| CN101742289B (en) | Method, system and device for compressing video code stream | |
| CN113766317A (en) | Video transmission method, video transmission device, electronic equipment and storage medium | |
| CN110113610B (en) | Data transmission method and device | |
| WO2021057697A1 (en) | Video encoding and decoding methods and apparatuses, storage medium, and electronic device | |
| CN106993199A (en) | A kind of method and system of real-time video transcoding | |
| US9264737B2 (en) | Error resilient transmission of random access frames and global coding parameters | |
| WO2021057477A1 (en) | Video encoding and decoding method and related device | |
| CN110996122B (en) | Video frame transmission method, device, computer equipment and storage medium | |
| CN113794903A (en) | Video image processing method and device and server | |
| CN113259671B (en) | Loop filtering method, device, equipment and storage medium in video coding and decoding | |
| US7403566B2 (en) | System, computer program product, and method for transmitting compressed screen images from one computer to another or many computers | |
| TWI519131B (en) | Video transmission system and transmitting device and receiving device thereof | |
| CN103929682A (en) | Method and device for setting key frames in video live broadcast system | |
| CN110248047B (en) | Synchronous switching method and system of video matrix | |
| CN115442615A (en) | Video coding method and device, electronic equipment and storage medium | |
| CN110266911B (en) | Decoding method and device capable of realizing synchronous switching of video matrix and readable storage medium | |
| KR100899666B1 (en) | Dispersed multistreaming transmission apparatus | |
| US20160057431A1 (en) | Method and device for transmitting a sequence of pictures, and corresponding method and device for reception, computer program and recording medium | |
| KR100827152B1 (en) | .264 apparatus and method of an efficient h.264 decoding for nal unit error detection | |
| CN103313017B (en) | Multichannel kinescope method and system | |
| JP2014135728A (en) | Video transmission system and video transmission method | |
| US20140369424A1 (en) | Video codec system and video stream transmission method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200908 |
|
| RJ01 | Rejection of invention patent application after publication |