CN115396696A - Video data transmission method, system, processing device and storage medium - Google Patents

Video data transmission method, system, processing device and storage medium Download PDF

Info

Publication number
CN115396696A
CN115396696A CN202211005970.1A CN202211005970A CN115396696A CN 115396696 A CN115396696 A CN 115396696A CN 202211005970 A CN202211005970 A CN 202211005970A CN 115396696 A CN115396696 A CN 115396696A
Authority
CN
China
Prior art keywords
independent
video
independent element
image frame
moving speed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211005970.1A
Other languages
Chinese (zh)
Other versions
CN115396696B (en
Inventor
郑夏桐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Netease Hangzhou Network Co Ltd
Original Assignee
Netease Hangzhou Network Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Netease Hangzhou Network Co Ltd filed Critical Netease Hangzhou Network Co Ltd
Priority to CN202211005970.1A priority Critical patent/CN115396696B/en
Publication of CN115396696A publication Critical patent/CN115396696A/en
Application granted granted Critical
Publication of CN115396696B publication Critical patent/CN115396696B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/27—Server based end-user applications
    • H04N21/274—Storing end-user multimedia data in response to end-user request, e.g. network recorder
    • H04N21/2743—Video hosting of uploaded data from client

Abstract

The embodiment of the application provides a video data transmission method, a video data transmission system, processing equipment and a storage medium, and relates to the technical field of computers. The method comprises the steps that a video sending end carries out element analysis on a current image frame of a collected video, and a plurality of independent elements in the current image frame and the moving speed of each independent element are determined; respectively determining the transmission frame rate of each independent element according to the moving speed of each independent element; and according to the transmission frame rate of each independent element, respectively sending each independent element to at least one video receiving end, so that each video receiving end combines each independent element into an image frame to be displayed. The method can split the picture in the video into a plurality of independent elements and transmit the picture according to the transmission frame rate of each independent element, thereby ensuring the video quality of a video receiving end and reducing the consumption of flow and the requirement on network bandwidth in the transmission process.

Description

Video data transmission method, system, processing device and storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a video data transmission method, a video data transmission system, a video data processing device, and a storage medium.
Background
The network live broadcast is an information network publishing mode which synchronously makes and publishes information along with the occurrence and development processes of events on site and has a bidirectional circulation process. The form of the live broadcast can comprise live broadcast of sports events and literary and artistic activities, live broadcast of network real persons and the like.
In the live broadcasting process, in order to obtain clear pictures, the anchor and the user need to keep higher network speed. In the prior art, when the network speed of a user or a main broadcast is reduced, a server adaptively reduces the frame rate and the definition of a transmitted video so as to maintain the watching of the user as much as possible.
However, when the frame rate down-regulation amplitude is too large, video pause may be caused, which affects the smoothness of viewing, and the down-regulation of the definition may also cause the user to see a live interface that is not clear.
Disclosure of Invention
The application provides a video data transmission method, a video data transmission system, a video data transmission processing device and a storage medium, which can divide a picture in a video into a plurality of independent elements and transmit the picture according to the transmission frame rate of each independent element, so that the video quality of a video receiving end is ensured, and the consumption of flow and the requirement on network bandwidth in the transmission process are reduced.
The embodiment of the application can be realized as follows:
in a first aspect, an embodiment of the present application provides a video data transmission method, which is applied to a video sending end, and the method includes:
performing element analysis on a current image frame of a collected video, and determining a plurality of independent elements in the current image frame and the moving speed of each independent element;
respectively determining the transmission frame rate of each independent element according to the moving speed of each independent element;
and according to the transmission frame rate of each independent element, respectively sending each independent element to at least one video receiving end, so that each video receiving end combines each independent element into an image frame to be displayed.
In an alternative embodiment, the element parsing a current image frame of the captured video, and determining a plurality of independent elements in the current image frame and a moving speed of each of the independent elements, includes:
determining a plurality of independent elements in the current image frame according to the image characteristics of the current image frame;
determining a type of each of the independent elements, the type comprising: new element types and existing element types;
and determining the moving speed of each independent element according to the type of each independent element.
In an alternative embodiment, the determining the moving speed of each of the independent elements according to the type of each of the independent elements includes:
if the type of the independent element is the existing element type, determining the moving speed of the independent element according to the contour information of the independent element in the current image frame and the contour information of the independent element in a historical image frame, wherein the historical image frame comprises: at least one image frame preceding the current image frame, the contour information being used to characterize the edge shape of the individual element and the position of the individual element in the corresponding image frame.
In an alternative embodiment, the determining the moving speed of the independent element according to the contour information of the independent element in the current image frame and the contour information of the independent element in the historical image frame includes:
determining difference information between contour information of the independent element in the current image frame and contour information of the independent element in a history image frame;
and determining the moving speed of the independent element according to the difference information.
In an alternative embodiment, the difference information is used to represent a position difference between a pixel point of the contour information of the independent element in the history image frame and a corresponding pixel point of the contour information of the independent element in the current image frame;
the determining the moving speed of the independent element according to the difference information comprises:
determining a time difference value according to the difference value between the display time of the historical image frame and the display time of the current image frame;
and taking the ratio of the difference information to the time difference value as the moving speed of the independent element.
In an alternative embodiment, the determining the moving speed of each independent element according to the type of each independent element includes:
and if the type of the independent element is the new element type, determining the moving speed of the independent element to be a preset moving speed.
In an alternative embodiment, the determining the type of each of the independent elements includes:
and searching the independent element from an independent element library corresponding to the video, if the independent element is searched, determining the type of the independent element as the existing element type, otherwise, determining the type of the independent element as the new element type.
In an optional embodiment, the method further comprises:
if a new independent element is identified from the image frames of the video that is not present in the independent element library, adding the new independent element to the independent element library.
In an optional implementation manner, the determining, according to the moving speed of each of the independent elements, the transmission frame rate of each of the independent elements respectively includes:
comparing the moving speed of each independent element with a preset speed upper limit and a preset speed lower limit corresponding to each preset grade to determine the transmission grade corresponding to the independent element, wherein each preset grade is used for indicating a transmission frame rate;
and taking the transmission frame rate corresponding to the transmission grade as the transmission frame rate of the independent element.
In an alternative embodiment, before the comparing the moving speed of each of the independent elements with the preset upper speed limit and the preset lower speed limit corresponding to each preset level, the method further includes:
dividing the transmission frame rate into a plurality of preset levels, and determining a preset upper speed limit and a preset lower speed limit corresponding to each preset level.
In a second aspect, an embodiment of the present application provides a video data transmission method, which is applied to a video receiving end, and the method includes:
receiving a plurality of independent elements sent by the video sending end;
and sequentially combining the independent elements into image frames, and playing a plurality of image frames.
In an alternative embodiment, the combining the independent elements into an image frame in sequence includes:
and sequentially combining the independent elements into an image frame according to the contour information of the independent elements.
In a third aspect, an embodiment of the present application provides a first video data transmission apparatus, including:
the element analysis module is used for carrying out element analysis on a current image frame of the acquired video and determining a plurality of independent elements in the current image frame and the moving speed of each independent element;
a transmission frame rate determining module, configured to determine a transmission frame rate of each independent element according to a moving speed of each independent element;
and the independent element sending module is used for respectively sending each independent element to at least one video receiving end according to the transmission frame rate of each independent element, so that each video receiving end combines each independent element into an image frame to display the image frame.
The element analysis module is further specifically configured to determine a plurality of independent elements in the current image frame according to image features of the current image frame; determining a type of each of the independent elements, the type comprising: new element types and existing element types; and determining the moving speed of each independent element according to the type of each independent element.
The element analysis module is further specifically configured to, if the type of the independent element is the existing element type, determine a moving speed of the independent element according to the profile information of the independent element in the current image frame and the profile information of the independent element in a history image frame, where the history image frame includes: at least one image frame preceding the current image frame, the contour information being used to characterize the edge shape of the individual element and the position of the individual element in the corresponding image frame.
The element parsing module is further specifically configured to determine difference information between contour information of the independent element in the current image frame and contour information of the independent element in a historical image frame; and determining the moving speed of the independent element according to the difference information.
The element analysis module is further specifically configured to determine a time difference value according to a difference value between the display time of the historical image frame and the display time of the current image frame; and taking the ratio of the difference information to the time difference value as the moving speed of the independent element.
The element analysis module is further specifically configured to determine that the moving speed of the independent element is a preset moving speed if the type of the independent element is the new element type.
The element parsing module is specifically further configured to search the independent element from an independent element library corresponding to the video, determine that the type of the independent element is the existing element type if the independent element is found, and determine that the type of the independent element is the new element type if the independent element is not found.
An element library update module to add a new independent element to the independent element library if the new independent element is identified from an image frame of the video that is not present in the independent element library.
The element library updating module is specifically further configured to compare the moving speed of each independent element with a preset speed upper limit and a preset speed lower limit corresponding to each preset level, and determine a transmission level corresponding to the independent element, where each preset level is used to indicate a transmission frame rate; and taking the transmission frame rate corresponding to the transmission grade as the transmission frame rate of the independent element.
The element library updating module is further specifically configured to divide a transmission frame rate into a plurality of preset levels, and determine a preset upper speed limit and a preset lower speed limit corresponding to each preset level.
In a fourth aspect, an embodiment of the present application provides a second video data transmission apparatus, including:
and the element receiving module is used for receiving a plurality of independent elements sent by the video sending end.
And the element combination module is used for sequentially combining the independent elements into image frames and playing a plurality of image frames.
The element combination module is further specifically configured to combine each of the independent elements into an image frame in sequence according to the contour information of each of the independent elements.
In a fifth aspect, an embodiment of the present application provides a video data transmission system, including: the system comprises a plurality of video uploading ends, a video sending end and a plurality of video receiving ends, wherein the video sending end is respectively in communication connection with the plurality of video uploading ends and the plurality of video receiving ends;
the video uploading end is used for acquiring a video and sending the video to the video sending end;
the video sending end is configured to execute the video data transmission method according to any one of the first aspect, determine a plurality of independent elements in the video and a transmission frame rate of each of the independent elements, and send each of the independent elements to a corresponding plurality of video receiving ends respectively;
the video receiving end is configured to execute the video data transmission method according to any one of the second aspects, and combine the received multiple independent elements into an image frame for display.
In a sixth aspect, an embodiment of the present application provides an electronic device, where the electronic device includes: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, when the electronic device is operated, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to execute the steps of the video data transmission method according to any one of the first aspect or the second aspect.
In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps of the video data transmission method according to any one of the first aspect or the video data transmission method according to any one of the second aspect.
The beneficial effects of the embodiment of the application include:
by adopting the video data transmission method, the video data transmission system, the video data transmission processing device and the video data transmission storage medium, the image frames in the videos can be analyzed into a plurality of independent elements, the transmission frame rate is given to the independent elements according to the moving speed of the independent elements, and finally the independent elements are respectively transmitted to the video receiving end according to the transmission frame rate of the independent elements. Under the condition of ensuring the definition of the video displayed by the video receiving end, the consumption of the flow and the requirement on the network bandwidth in the video transmission process are reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a schematic structural diagram of a video data transmission system according to an embodiment of the present application;
fig. 2 is a schematic flowchart illustrating steps of a video data transmission method according to an embodiment of the present application;
fig. 3 is a flowchart illustrating a step of determining a moving speed of an independent element in a video data transmission method according to an embodiment of the present application;
fig. 4 is a schematic flowchart illustrating a further step of determining a moving speed of an independent element in a video data transmission method according to an embodiment of the present application;
fig. 5 is a flowchart illustrating a step of determining a moving speed of an independent element according to difference information in a video data transmission method according to an embodiment of the present application;
fig. 6 is a schematic flowchart illustrating a step of determining a frame rate for video data transmission according to an embodiment of the present application;
fig. 7 is a schematic flowchart illustrating a procedure of generating video frames by combining video receiving ends of a video data transmission method according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of a first video data transmission apparatus according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of a second video data transmission apparatus according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Icon: 1011-a first video upload terminal; 1012-second video uploading end; 1013-nth video upload end; 102-a video transmitting end; 1031-a first video receiving end; 1032-a second video receiving end; 1033-nth video receiving end; 100-a first video data transmission device; 1001-element parsing module; 1002-a transmission frame rate determining module; 1003-independent element sending module; 1004-element library update module; 110-a second video data transmission device; 1101-an element receiving module; 1102-element combination module; 2001-a processor; 2002-a storage medium; 2003-bus.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
It should be noted that the features of the embodiments of the present application may be combined with each other without conflict.
In the transmission process of live broadcast data, the most difficult realization is the balance between the network bandwidth and the image definition. The definition of the live-broadcast picture is positively correlated with the network bandwidth, and when the network bandwidth is low, the video may be jammed or even cannot be loaded. To maintain the continuity of the video viewed by the user, the server adaptively adjusts the transmission speed and the sharpness of the video transmitted to the user device downward, which affects the sharpness and smoothness of the video viewed by the user.
Based on this, the application provides a video data transmission method, a video data transmission system, a video data transmission processing device and a storage medium, each image frame in a video is analyzed into a plurality of independent elements, then differential transmission is performed according to the moving speed of each independent element, occupation of network bandwidth by repeated transmission of independent elements with small or non-moving speed is avoided, and the transmission efficiency of video data is improved on the premise of ensuring the definition of the transmitted video.
A video data transmission method, a system, a processing device, and a storage medium provided in the embodiments of the present application are explained below with reference to a plurality of specific application examples.
Fig. 1 is a schematic structural diagram of a video data transmission system according to an embodiment of the present application, and as shown in fig. 1, the video data transmission system includes: the video transmitting terminal 102 is in communication connection with the plurality of video uploading terminals and the plurality of video receiving terminals respectively.
The video uploading end is used for collecting videos and sending the videos to the video sending end 102.
The video uploading end can be arranged at the video publisher end, can be a smart phone, or a combination of a camera and a mobile computer, and can shoot real-time live video and transmit the live video to the video sending end 102.
As shown in fig. 1, there may be a plurality of video uploading terminals, such as a first video uploading terminal 1011, a second video uploading terminal 1012 \8230andan nth video uploading terminal 1013, where the specific number is determined according to the carrying capacity of the video sending terminal 102.
The video uploading terminal may also be another intelligent terminal storing the prerecorded video, and can transmit the prerecorded video to the video sending terminal 102 in real time.
The video sending end 102 is configured to execute a video data transmission method executed by the video sending end 102 in the following embodiments, determine a plurality of independent elements in a video and a transmission frame rate of each independent element, and send each independent element to each video receiving end respectively.
The video sending end 102 may be a server provided by a live broadcast platform or other video relay operators, the network bandwidth and the data processing efficiency of the server are high, after receiving a real-time video transmitted by a video uploading end, each image frame in the video can be analyzed into a plurality of independent elements, after determining the moving speed of each independent element, each independent element in the video of the video uploading end is transmitted to a plurality of video receiving ends corresponding to the video uploading end with low delay according to a transmission frame rate matched with the moving speed.
In addition, the video sending end 102 may simultaneously receive videos collected and transmitted by a plurality of video uploading ends in real time, and execute the video data transmission method of the video sending end 102 to perform synchronous processing on the videos transmitted by the plurality of video uploading ends in the following embodiments.
The video receiving end is configured to execute a video data transmission method executed by the video receiving end in the following embodiments, and merge the received multiple independent elements into an image frame for display.
The video receiving end can be a terminal device for watching videos by a user, and includes but is not limited to various devices with networking functions and display functions, such as intelligent devices, vehicle-mounted devices, computing devices, handheld devices, wearable devices and the like.
The video transmitting terminal 102 may transmit the video of the video uploading terminal to at least one corresponding video receiving terminal according to the corresponding relationship between the video receiving terminals and the video uploading terminal. Illustratively, the first video uploading end 1011 may correspond to the first video receiving end 1031, the second video receiving end 1032, and the nth video uploading end 1013 corresponds to the nth video receiving end 1033.
Optionally, the connection mode between the video sending end 102 and the video uploading end and the connection mode between the video receiving ends may be through wireless network, or through optical fiber or other communication lines, which is not specifically limited herein.
Based on the system architecture, the method and the system can be applied to scenes such as live broadcast and the like which relate to video transmission. For example, in a live scene, a terminal device held by a main broadcast is a video uploading end. The anchor uses a video uploading end to shoot the video in real time and upload the video to a video sending end in real time, and the video sending end can be a server of a live broadcast platform, for example. The video sending end sends the processed video to one or more video receiving ends. Each video receiving end respectively refers to a terminal device held by a user watching live broadcasting.
Next, a video data transmission method applied to a video transmitting end is explained.
Fig. 2 is a schematic flowchart illustrating steps of a video data transmission method according to an embodiment of the present application, where an execution main body of the method may be a video sending end in the video data transmission system, and as shown in fig. 2, the method may include the following steps:
s201, element analysis is carried out on a current image frame of the collected video, and a plurality of independent elements in the current image frame and the moving speed of each independent element are determined.
The content in each image frame may be divided into different image layers according to respective boundaries, each image layer serves as an independent element and occupies a portion of the image frame, and the independent element may be, for example, a water cup, a director, a background wall, a bookshelf, or the like, which is not limited herein.
The moving speed of an individual element refers to the speed of the individual element moving in a position or a boundary on a fixed-size image frame within a certain period of time in a video. It is understood that if the position and boundary of the independent element are changed greatly in a short time, the moving speed of the independent element is faster, and conversely, the moving speed is slower.
After a video sending end receives a video stream transmitted by a certain video uploading end, namely a plurality of continuous image frames, the video sending end can take each image frame as a current image frame in sequence from an image frame at the beginning of the video stream, and then identify the content in the current image frame and divide the content to obtain a plurality of independent elements in the current image frame.
Next, the video sending end may compare the position of the independent element in the current image frame, which is analyzed from the current video frame, with the position of the independent element in a previous period of time, to determine the moving speed of the independent element.
And S202, respectively determining the transmission frame rate of each independent element according to the moving speed of each independent element.
The frame rate of transmission may be an index for measuring the number of frames transmitted per second by the video sender, for example, 1 second/frame.
The video sending end can match the adaptive transmission frame rate for the independent elements with different moving speeds. For example, individual elements moving slower may be matched to a smaller transfer rate, while individual elements moving faster may be matched to a larger transfer rate.
And S203, respectively sending each independent element to at least one video receiving end according to the transmission frame rate of each independent element, so that the video receiving end combines each independent element into an image frame to display the image frame.
After the transmission rate of each independent element is determined, at least one video receiving end is determined according to the corresponding relation between the video uploading end and the video receiving end, and each independent element is respectively sent to the corresponding video receiving end according to the transmission frame rate of each independent element in the current frame.
It should be noted that the video sending end may use image frames every other preset number of frames as the current image frame, and may also use each image frame as the current image frame in sequence. When the current image frame is acquired at intervals, the video transmitting end transmits independent elements of a plurality of image frames between the current image frame and the next current image frame to at least one video receiving end at the current transmission frame rate after determining the transmission frame rate of each independent element in the current image frame and before analyzing the next current image frame. When the video sending end carries out element analysis on each image frame in the video stream in sequence, after each image frame and the image frame are subjected to element analysis, the transmission rate of each independent element is obtained to have a one-to-many corresponding relation, and therefore each independent element in the current image frame can be transmitted according to the transmission rate of the independent element of the current image frame.
It can be understood that when the independent elements are transmitted, the corresponding position information and the corresponding outline information are carried. After each video receiving end receives the multiple independent elements sent by the video sending end, each independent element which is transmitted last time can be spliced to a corresponding position in the image frame according to the position information and the outline information of each independent element and the transmission frame rate of each independent element to form the image frame. For example, a transmission frame rate of a certain independent element is 5 seconds/frame, that is, the independent element participates in the composition of the image frames in all the image frames displayed in the next 5 seconds of the image frame. And if the independent element is not received again after 5 seconds, the independent element does not continuously participate in the synthesis of the image frame.
In this embodiment, the video receiving end parses each image frame in the video into a plurality of independent elements, and then performs differentiated transmission according to the moving speed of each independent element, so that occupation of network bandwidth by repeated transmission of independent elements with low or no moving speed is avoided, and the transmission efficiency of video data is improved on the premise of ensuring the definition of the transmitted video.
Alternatively, as shown in fig. 3, in the step S201, performing element parsing on the current image frame of the captured video, and determining a plurality of independent elements and a moving speed of each independent element in the current image frame may be implemented by the following steps S301 to S303:
s301, according to the image characteristics of the current image frame, a plurality of independent elements in the current image frame are determined.
The video sending end can identify the boundary of each article in the current image frame by adopting a method based on color sampling or a method based on propagation, and obtain a plurality of image layers by segmentation, wherein the image layers are used as a plurality of independent elements in the current image frame.
Where the color sampling based approach may assume that the local regions of the independent elements are contiguous, then neighboring pixels with similar colors have a correlation in statistical characteristics. Therefore, different color distribution clusters can be respectively established for each independent element, and then pixel points occupied by a plurality of independent elements and boundaries, namely contour information, can be determined in a mode of sampling and classifying pixels in an unknown region to the different color distribution clusters, so that the independent elements can be segmented from an image frame.
The propagation-based method can assume that the independent elements meet the smoothness property, and constructs a Laplace matrix by taking the transparency of the independent elements as a characteristic gradient field of the independent elements, so that pixel points occupied by the independent elements, namely position information and contour information, are determined, and the independent elements are segmented from the image frame.
S302, determining the type of each independent element.
The types include: new element types as well as existing element types.
The new element type is a new independent element that does not appear before the current image frame, the existing element type may be an independent element that has appeared for a while before the current image frame, and the existing element is synthesized in the last video frame before the independent element related to the current video frame is received at the video receiving end.
And S303, determining the moving speed of each independent element according to the type of each independent element.
The video receiving end can determine whether the moving speed of each independent element needs to be judged through the image frame received within the past preset time according to whether the independent element obtained by analysis is a new element type or an existing element type. It will be appreciated that since the new element type has not occurred within the last preset time, the current moving speed may not be determined by the comparison between image frames.
In this embodiment, the video receiving end determines a plurality of independent elements according to the image characteristics of each image frame, and determines the moving speed according to the types of the independent elements. The speed of analyzing and transmitting the image frames is improved, and the live broadcast delay is reduced.
Optionally, in step S303, determining a moving speed of each independent element according to the type of each independent element may include: and if the type of the independent element is the existing element type, determining the moving speed of the independent element according to the contour information of the independent element in the current image frame and the contour information of the independent element in the historical image frame.
Wherein the history image frame comprises: at least one image frame preceding the current image frame, the contour information being used to characterize the edge shape of the individual element and the position of the individual element in the corresponding image frame.
With the above embodiment, after the independent element is subjected to element parsing by the video sending end, the contour information of the independent element, that is, the boundary of the independent element in the image frame, and the pixel point position occupied by the boundary in the corresponding image frame are determined. Optionally, when the independent elements are transmitted, in addition to the outline information, the characteristic information of the independent elements in the outline information may also be included, and the characteristic information is transmitted to the plurality of video receiving ends together.
The number of the historical image frames may be determined according to the preset observation time and the frame rate of the video stream transmitted by the video uploading end, and for example, if the preset observation time is 2 seconds and the frame rate of the video stream transmitted is 10 frames/second, the historical image frames may be the first 20 frames of the current image frame. Optionally, the preset observing time may be equal to a preset frame number for performing element parsing by the video sending end in the foregoing embodiment.
In this way, the moving speed of each independent element can be determined according to the change situation of the outline information of the image frame sequence formed by the historical image frame of the current image frame or the change situation of the outline information of the current image frame relative to the historical image frame.
In this embodiment, by comparing the profile information of the independent element in the current image frame with the profile information of the independent element in the historical image frame, the moving speed of each independent element is quantized, so that the subsequent matching of the transmission frame rate is facilitated, and the occupation of the transmission bandwidth is reduced.
Alternatively, as shown in fig. 4, in the above embodiment, if the type of the independent element is the existing element type, determining the moving speed of the independent element according to the contour information of the independent element in the current image frame and the contour information of the independent element in the historical image frame may be implemented by the following steps S401 to S402.
S401, determining difference information between the contour information of the independent element in the current image frame and the contour information of the independent element in the historical image frame.
If the independent element is a anchor, the image characteristics of the independent element in each image frame may not change greatly, but the contour information in the current image frame and the historical image frame may change, and the contour information is reflected as difference information of the independent element.
S402, determining the moving speed of the independent element according to the difference information.
Further, the video sending end can determine whether the independent element moves and the moving speed in the time interval between the current image frame and one or more historical image frames through the determined difference information of the independent element after the current image frame and the historical image frames.
In this embodiment, the video sending end quantizes the moving speed of each independent element according to the profile information of the independent element in the current image frame and the difference information obtained by comparison in the historical image frame, so that the subsequent matching of the transmission frame rate is facilitated, and the occupation of the transmission bandwidth is reduced.
Optionally, the difference information is used to represent a position difference between a pixel point of the contour information of the independent element in the history image frame and a corresponding pixel point of the contour information of the independent element in the current image frame.
It can be understood that the pixel points included in the edge contour of the independent element generally have a small change, and therefore, a plurality of pixel points may be selected from the contour of the independent element in the current image frame, and then the positions of the plurality of pixel points in the historical image frame are respectively determined, and further, the moving distances of the plurality of pixel points are respectively calculated according to a preset algorithm, and optionally, the average value of the moving distances may be used as the position difference value of the independent element to represent the difference information of the independent element.
As shown in fig. 5, in the above step S402, the moving speed of the independent element is determined according to the difference information, and the steps S501 to S502 described below can be implemented.
And S501, determining a time difference value according to the difference value between the display time of the historical image frame and the display time of the current image frame.
And if the video receiving end determines the difference information according to the comparison between one frame in the historical image frames and the current image frame, taking the difference between the display time of the historical image frame at the video receiving end and the display time of the current image frame at the video receiving end as the time difference.
If the video receiving end takes the multiframes with the same time interval in the historical image frames as the current image frames, the average value of pairwise comparison of the adjacent image frames in the time interval is taken as difference information, and the time interval is taken as a time difference value.
And S502, taking the ratio of the difference information to the time difference value as the moving speed of the independent element.
Next, the video sending end may determine the moving speed of the independent element in the unit time according to the ratio of the difference information to the time difference value.
In this embodiment, the video sending end accurately and quickly quantizes the moving speed of the independent element according to the ratio of the difference information to the time difference.
Optionally, in the step S303, determining a moving speed of each independent element according to the type of each independent element may include: and if the type of the independent element is the new element type, determining the moving speed of the independent element to be the preset moving speed.
The method can generate an independent element library by analyzing elements of each image frame of a video stream transmitted by a video uploading end within a preset time interval to obtain a plurality of independent elements. The independent element library is updated in real time, and the independent elements included in the independent element library are the independent elements currently displayed on the video receiving end or to be displayed on the video receiving end.
Therefore, after the video sending end analyzes the current image frame, a plurality of independent elements are obtained through analysis, and feature matching is carried out on each independent element and each independent element in the independent image library. If a certain independent element obtained by analysis in the current image frame cannot be matched with each independent element in the independent image library, determining that the independent element is an independent element which is newly appeared in the current image frame relative to the previous current image frame, wherein the type of the independent element can be a new element type.
For an individual element whose type is a new element type, since the individual element does not appear in a previous frame, the moving speed thereof cannot be determined by comparison with a history image frame, and the moving speed of the individual element may be set to a preset moving speed.
In this embodiment, the moving speed of the independent element of which the type of the independent element is the new element type is set to be the preset moving speed, so that a fault caused by the fact that the new independent element cannot determine the moving speed is avoided, and the reliability of the method is improved.
Optionally, the video data transmission method provided in the embodiment of the present application may further include: if a new independent element is identified from the image frames of the video that is not present in the independent element library, the new independent element is added to the independent element library.
When the video sending end analyzes the current image frame, the obtained at least one independent element and any independent element in the independent element library cannot establish a corresponding relation through image feature matching, and the independent element is used as a new independent element to be added into the independent element library.
In this embodiment, the video sending end adds the new independent element to the independent element library, so that the independent element library is maintained in real time, and an error caused by that the new independent element makes an image frame displayed by the video receiving end not be updated in time is avoided.
Optionally, in the step S303, determining a moving speed of each independent element according to the type of each independent element may include: and searching the independent element from the independent element library corresponding to the video, if the independent element is searched, determining the type of the independent element as the existing element type, otherwise, determining the type of the independent element as the new element type.
By the embodiment, the video sending end maintains the independent element library in real time. Therefore, the video sending end can match the independent element library according to a plurality of independent elements obtained after element analysis is carried out on the current image frame, if a certain independent element can be matched with a certain independent element in the independent element library, the fact that the independent element appears in the image frame displayed by the last video receiving end is confirmed, and then each image frame before the next current image frame is continuously sent according to the moving speed generated by new calculation of the independent element and the matching frame rate.
In the embodiment, the type of the independent element matched with the independent element in the independent element library is marked as the existing element type, so that the occupation of repeated transmission on network bandwidth is avoided, and the consumption of flow in the video transmission process is reduced.
Alternatively, as shown in fig. 6, in the step S202, the transmission frame rate of each independent element is determined according to the moving speed of each independent element, which may be implemented by the following steps S601 to S602:
s601, comparing the moving speed of each independent element with a preset upper speed limit and a preset lower speed limit corresponding to each preset level, and determining a transmission level corresponding to the independent element.
Wherein, each preset grade is respectively used for indicating a transmission frame rate.
Optionally, a plurality of preset levels may be set, each preset level corresponding to a moving speed range, an upper limit of the moving speed range being a preset upper limit, and a lower limit of the moving speed range being a preset lower limit.
Further, after the current image frame is subjected to element analysis, the moving speed of the independent element of which the determined type is the existing element type is sequentially compared with each preset level, the moving speed of the independent element is determined to fall into the moving speed range corresponding to which preset level, and then the transmission level corresponding to the preset level is used as the transmission level of the independent element according to the corresponding relation between the preset level and the transmission level.
Alternatively, the rank value of the transmission rank may be equal to the rank value of the preset rank.
S602, the transmission frame rate corresponding to the transmission rank is taken as the transmission frame rate of the independent element.
It should be noted that each transmission level corresponds to a different transmission frame rate, and then, the video sending end may assign the transmission frame rate indicated by the transmission level of the independent element to the independent element, so that the independent element transmits the independent element at the transmission frame rate before the next current video frame.
In this embodiment, by adaptively assigning the transmission frame rate set in a gradient manner to the independent element, such differential transmission reduces the traffic consumption and the requirement on the network bandwidth during the transmission process while ensuring the video quality at the video receiving end.
Optionally, before comparing the moving speed of each independent element with the preset upper speed limit and the preset lower speed limit corresponding to each preset level in step S601, the method may further include: dividing the transmission frame rate into a plurality of preset levels, and determining a preset upper speed limit and a preset lower speed limit corresponding to each preset level.
The possible transmission frame rates of the independent elements may be divided into a plurality of preset levels according to a certain gradient interval, a preset lower speed limit of a first preset level in each preset level may be 0, and a preset upper speed limit may be a sum of a preset upper speed limit and a preset lower speed limit and the gradient interval.
The first preset level upper speed limit may be taken as a second preset level lower limit. Alternatively, the moving speed of a certain individual element equal to the lower limit of the second preset level may be classified into the second preset level.
In this embodiment, the transmission frame rate is divided into a plurality of preset levels, so that each independent element can be matched with an adaptive transmission frame rate, thereby reducing the consumption of traffic in the video transmission process.
Optionally, as shown in fig. 7, an embodiment of the present application further provides a video data transmission method, where an execution subject of the method may be a video receiving end in the video data transmission system, and the method includes the following steps:
and S701, receiving a plurality of independent elements sent by a video sending end.
According to the embodiment, the video sending end divides the content in each image frame into different image layers according to respective boundaries, each image layer is used as an independent element, a plurality of independent elements are obtained, and the independent elements are sent to at least one video receiving end according to the transmission frame rate corresponding to each independent element. And the video receiving end receives the plurality of independent elements sent by the video sending end.
S702, combining the independent elements into image frames in sequence, and playing a plurality of image frames.
It can be understood that when the independent elements are transmitted, the corresponding position information and the corresponding outline information are carried. After each video receiving end receives the plurality of independent elements sent by the video sending end, each independent element which is transmitted last time can be spliced to a corresponding position in the image frame according to the position information and the contour information of each independent element and the transmission frame rate of each independent element to form the image frame.
In this embodiment, the video receiving end combines a plurality of independent elements into an image frame, and on the premise of ensuring the definition of the transmitted video, the occupation of network bandwidth by repeated transmission of independent elements with low or no movement speed is avoided.
Optionally, in step S702, sequentially combining the independent elements into an image frame may include: and sequentially combining the independent elements into an image frame according to the outline information of the independent elements.
The contour information of each independent element is used for describing the boundary of the independent element and the position of a pixel point in the boundary in an image frame, so that the image frames can be spliced at corresponding positions in the image frame to be combined into the image frame when being combined according to the contour information of the independent elements.
In this embodiment, the video receiving end can splice the independent elements, thereby reducing the display delay in the live broadcasting process.
Referring to fig. 8, an embodiment of the present application provides a first video data transmission apparatus 100, including:
an element analysis module 1001, configured to perform element analysis on a current image frame of an acquired video, and determine a plurality of independent elements and a moving speed of each independent element in the current image frame;
a transmission frame rate determining module 1002, configured to determine the transmission frame rate of each independent element according to the moving speed of each independent element;
the independent element sending module 1003 is configured to send each independent element to at least one video receiving end according to the transmission frame rate of each independent element, so that each video receiving end combines each independent element into an image frame and displays the image frame.
The element parsing module 1001 is further specifically configured to determine a plurality of independent elements in the current image frame according to image features of the current image frame; determining the type of each independent element, wherein the type comprises the following steps: new element types and existing element types; and determining the moving speed of each independent element according to the type of each independent element.
The element parsing module 1001 is further specifically configured to, if the type of the independent element is an existing element type, determine a moving speed of the independent element according to the profile information of the independent element in the current image frame and the profile information of the independent element in the historical image frame, where the historical image frame includes: at least one image frame preceding the current image frame, the contour information being used to characterize the edge shape of the individual element and the position of the individual element in the corresponding image frame.
The element parsing module 1001 is further specifically configured to determine difference information between contour information of the independent element in the current image frame and contour information of the independent element in the historical image frame; based on the difference information, the moving speed of the individual element is determined.
The element parsing module 1001 is further specifically configured to determine a time difference value according to a difference value between the display time of the historical image frame and the display time of the current image frame; and taking the ratio of the difference information to the time difference value as the moving speed of the independent element.
The element parsing module 1001 is further specifically configured to determine that the moving speed of the independent element is the preset moving speed if the type of the independent element is the new element type.
The element parsing module 1001 is further specifically configured to search an independent element from an independent element library corresponding to the video, determine that the type of the independent element is an existing element type if the independent element is found, and determine that the type of the independent element is a new element type if the independent element is not found.
An element library update module 1004 for adding a new independent element to the independent element library if a new independent element is identified from the image frames of the video that is not present in the independent element library.
The element library updating module 1004 is further specifically configured to compare the moving speed of each independent element with a preset speed upper limit and a preset speed lower limit corresponding to each preset level, and determine a transmission level corresponding to the independent element, where each preset level is used to indicate a transmission frame rate; and taking the transmission frame rate corresponding to the transmission level as the transmission frame rate of the independent element.
The element library updating module 1004 is further specifically configured to divide the transmission frame rate into a plurality of preset levels, and determine a preset upper speed limit and a preset lower speed limit corresponding to each preset level.
Referring to fig. 9, an embodiment of the present application further provides a second video data transmission apparatus 110, including:
an element receiving module 1101, configured to receive multiple independent elements sent by a video sending end.
And an element combining module 1102, configured to sequentially combine the individual elements into an image frame, and play back a plurality of image frames.
The element combination module 1102 is further configured to sequentially combine the independent elements into an image frame according to the contour information of the independent elements.
Fig. 10 shows a schematic structural diagram of an electronic device provided in an embodiment of the present application, including: a processor 2001, a storage medium 2002 and a bus 2003, the storage medium 2002 storing machine readable instructions executable by the processor 2001, the processor 2001 communicating with the storage medium 2002 via the bus 2003 when the electronic device is operating a video data transmission method as in the embodiments, the processor 2001 executing the machine readable instructions, the processor 2001 a preamble of the video data transmission method item to perform the following steps:
performing element analysis on a current image frame of the acquired video, and determining a plurality of independent elements in the current image frame and the moving speed of each independent element;
respectively determining the transmission frame rate of each independent element according to the moving speed of each independent element;
and according to the transmission frame rate of each independent element, respectively sending each independent element to at least one video receiving end so that each video receiving end combines each independent element into an image frame to display.
The processor 2001, when performing element parsing on a current image frame of the captured video and determining a plurality of independent elements in the current image frame and a moving speed of each independent element, is specifically configured to:
determining a plurality of independent elements in the current image frame according to the image characteristics of the current image frame;
determining the type of each independent element, wherein the type comprises the following steps: new element types and existing element types;
and determining the moving speed of each independent element according to the type of each independent element.
The processor 2001, when performing the determination of the moving speed of each individual element according to the type of each individual element, is specifically configured to:
if the type of the independent element is the existing element type, determining the moving speed of the independent element according to the contour information of the independent element in the current image frame and the contour information of the independent element in the historical image frame, wherein the historical image frame comprises: at least one image frame preceding the current image frame, the contour information being used to characterize the edge shape of the individual element and the position of the individual element in the corresponding image frame.
The processor 2001, when performing the determination of the moving speed of the individual element based on the contour information of the individual element in the current image frame and the contour information of the individual element in the history image frame, is specifically configured to:
determining difference information between contour information of the independent elements in the current image frame and contour information of the independent elements in the historical image frame;
and determining the moving speed of the independent element according to the difference information.
The processor 2001, when performing the determination of the moving speed of the independent element based on the difference information, is specifically configured to:
determining a time difference value according to the difference value between the display time of the historical image frame and the display time of the current image frame;
and taking the ratio of the difference information to the time difference value as the moving speed of the independent element.
The processor 2001, when performing the determining the type of each individual element, is specifically configured to:
and searching the independent element from the independent element library corresponding to the video, if the independent element is searched, determining the type of the independent element as the existing element type, otherwise, determining the type of the independent element as the new element type.
In one possible embodiment, the method further comprises:
if a new independent element is identified from the image frames of the video that is not present in the independent element library, the new independent element is added to the independent element library.
The processor 2001, when performing the determination of the frame rate of transmission of each independent element according to the moving speed of each independent element, is specifically configured to:
comparing the moving speed of each independent element with a preset speed upper limit and a preset speed lower limit corresponding to each preset grade to determine the transmission grade corresponding to the independent element, wherein each preset grade is respectively used for indicating one transmission frame rate;
and taking the transmission frame rate corresponding to the transmission grade as the transmission frame rate of the independent element.
The processor 2001, before performing the comparison of the moving speed of each independent element with the preset upper speed limit and the preset lower speed limit corresponding to each preset level, is specifically configured to:
dividing the transmission frame rate into a plurality of preset levels, and determining a preset upper speed limit and a preset lower speed limit corresponding to each preset level.
Processor 2001 of the preamble of the video data transmission method item to carry out the following steps:
receiving a plurality of independent elements sent by a video sending end;
and sequentially combining the independent elements into image frames, and playing the image frames.
The processor 2001, when performing the sequential combination of the individual elements into an image frame, is specifically configured to:
and sequentially combining the independent elements into an image frame according to the outline information of the independent elements.
By the mode, the picture in the video is split into the plurality of independent elements, and the transmission is carried out according to the transmission frame rate of each independent element, so that the video quality of the video receiving end is ensured, and the consumption of flow and the requirement on network bandwidth in the transmission process are reduced.
An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the processor executes the following steps:
performing element analysis on a current image frame of the acquired video, and determining a plurality of independent elements in the current image frame and the moving speed of each independent element;
respectively determining the transmission frame rate of each independent element according to the moving speed of each independent element;
and respectively sending each independent element to at least one video receiving end according to the transmission frame rate of each independent element, so that each video receiving end combines each independent element into an image frame to display.
The processor is specifically configured to, when performing element analysis on a current image frame of the captured video and determining a plurality of independent elements and a moving speed of each independent element in the current image frame:
determining a plurality of independent elements in the current image frame according to the image characteristics of the current image frame;
determining the type of each independent element, wherein the type comprises the following steps: new element types and existing element types;
and determining the moving speed of each independent element according to the type of each independent element.
When the processor determines the moving speed of each independent element according to the type of each independent element, the processor is specifically configured to:
if the type of the independent element is the existing element type, determining the moving speed of the independent element according to the contour information of the independent element in the current image frame and the contour information of the independent element in the historical image frame, wherein the historical image frame comprises: at least one image frame preceding the current image frame, the contour information being used to characterize the edge shape of the individual element and the position of the individual element in the corresponding image frame.
The processor, when executing the determining of the moving speed of the independent element according to the contour information of the independent element in the current image frame and the contour information of the independent element in the historical image frame, is specifically configured to:
determining difference information between contour information of the independent element in the current image frame and contour information of the independent element in the historical image frame;
based on the difference information, the moving speed of the individual element is determined.
When determining the moving speed of the independent element according to the difference information, the processor is specifically configured to:
determining a time difference value according to the difference value between the display time of the historical image frame and the display time of the current image frame;
the ratio of the difference information to the time difference value is taken as the moving speed of the independent element.
When determining the type of each independent element, the processor is specifically configured to:
and searching the independent element from the independent element library corresponding to the video, if the independent element is searched, determining the type of the independent element as the existing element type, otherwise, determining the type of the independent element as the new element type.
In one possible embodiment, the method further comprises:
if a new independent element is identified from the image frames of the video that is not present in the independent element library, the new independent element is added to the independent element library.
When the processor is configured to determine the frame rate of transmission of each independent element according to the moving speed of each independent element, the processor is specifically configured to:
comparing the moving speed of each independent element with a preset speed upper limit and a preset speed lower limit corresponding to each preset grade to determine the transmission grade corresponding to the independent element, wherein each preset grade is respectively used for indicating one transmission frame rate;
and taking the transmission frame rate corresponding to the transmission grade as the transmission frame rate of the independent element.
Before the processor compares the moving speed of each independent element with the preset speed upper limit and the preset speed lower limit corresponding to each preset grade, the processor is specifically configured to:
dividing the transmission frame rate into a plurality of preset levels, and determining a preset upper speed limit and a preset lower speed limit corresponding to each preset level.
The preamble of the video data transmission method of a processor to carry out the following steps:
receiving a plurality of independent elements sent by a video sending end;
and combining the independent elements into image frames in sequence, and playing a plurality of image frames.
The processor, when performing the sequential combination of the individual elements into the image frame, is specifically configured to:
and sequentially combining the independent elements into an image frame according to the outline information of the independent elements.
By the mode, the picture in the video is split into the independent elements and transmitted according to the transmission frame rate of each independent element, so that the video quality of the video receiving end is ensured, and meanwhile, the consumption of flow and the requirement on network bandwidth in the transmission process are reduced.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus once an item is defined in one figure, it need not be further defined and explained in subsequent figures, and moreover, the terms "first", "second", "third", etc. are used merely to distinguish one description from another and are not to be construed as indicating or implying relative importance.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (15)

1. A video data transmission method is applied to a video sending end, and comprises the following steps:
performing element analysis on a current image frame of a collected video, and determining a plurality of independent elements in the current image frame and the moving speed of each independent element;
respectively determining the transmission frame rate of each independent element according to the moving speed of each independent element;
and according to the transmission frame rate of each independent element, respectively sending each independent element to at least one video receiving end, so that each video receiving end combines each independent element into an image frame to be displayed.
2. The method according to claim 1, wherein the element parsing a current image frame of the captured video, determining a plurality of independent elements in the current image frame and a moving speed of each of the independent elements, comprises:
determining a plurality of independent elements in the current image frame according to the image characteristics of the current image frame;
determining a type of each of the independent elements, the type comprising: new element types and existing element types;
and determining the moving speed of each independent element according to the type of each independent element.
3. The method according to claim 2, wherein said determining a moving speed of each of the individual elements according to a type of each of the individual elements comprises:
if the type of the independent element is the existing element type, determining the moving speed of the independent element according to the contour information of the independent element in the current image frame and the contour information of the independent element in a historical image frame, wherein the historical image frame comprises: at least one image frame preceding the current image frame, the contour information being used to characterize the edge shape of the individual element and the position of the individual element in the corresponding image frame.
4. The method according to claim 3, wherein the determining the moving speed of the independent element according to the contour information of the independent element in the current image frame and the contour information of the independent element in a history image frame comprises:
determining difference information between contour information of the independent element in the current image frame and contour information of the independent element in a historical image frame;
and determining the moving speed of the independent element according to the difference information.
5. The video data transmission method according to claim 4, wherein the difference information is used to represent a position difference between a pixel point of the contour information of the independent element in the history image frame and a corresponding pixel point of the contour information of the independent element in the current image frame;
the determining the moving speed of the independent element according to the difference information comprises:
determining a time difference value according to the difference value between the display time of the historical image frame and the display time of the current image frame;
and taking the ratio of the difference information to the time difference value as the moving speed of the independent element.
6. The method according to claim 2, wherein said determining a moving speed of each of the individual elements according to a type of each of the individual elements comprises:
and if the type of the independent element is the new element type, determining that the moving speed of the independent element is a preset moving speed.
7. The method for transmitting video data according to claim 2, wherein the determining the type of each of the independent elements comprises:
and searching the independent element from an independent element library corresponding to the video, if the independent element is searched, determining the type of the independent element as the existing element type, otherwise, determining the type of the independent element as the new element type.
8. The method for transmitting video data according to claim 7, wherein the method further comprises:
if a new independent element is identified from the image frames of the video that is not present in the independent element library, adding the new independent element to the independent element library.
9. The method according to any one of claims 1 to 8, wherein said determining a frame rate for transmission of each of the independent elements according to the moving speed of each of the independent elements comprises:
comparing the moving speed of each independent element with a preset speed upper limit and a preset speed lower limit corresponding to each preset grade to determine the transmission grade corresponding to the independent element, wherein each preset grade is respectively used for indicating one transmission frame rate;
and taking the transmission frame rate corresponding to the transmission level as the transmission frame rate of the independent element.
10. The method according to claim 9, wherein before comparing the moving speed of each of the independent elements with the preset upper speed limit and the preset lower speed limit corresponding to each of the preset levels, the method further comprises:
dividing the transmission frame rate into a plurality of preset levels, and determining a preset upper speed limit and a preset lower speed limit corresponding to each preset level.
11. A video data transmission method, applied to a video receiving end, the method comprising:
receiving a plurality of independent elements sent by a video sending end;
and sequentially combining the independent elements into image frames, and playing a plurality of image frames.
12. The method of claim 11, wherein said combining each of said independent elements into an image frame in sequence comprises:
and sequentially combining the independent elements into an image frame according to the contour information of the independent elements.
13. A video data transmission system, comprising: the system comprises a plurality of video uploading ends, a video sending end and a plurality of video receiving ends, wherein the video sending end is respectively in communication connection with the plurality of video uploading ends and the plurality of video receiving ends;
the video uploading end is used for acquiring a video and sending the video to the video sending end;
the video sending end is configured to execute the video data transmission method according to any one of claims 1 to 10, determine a plurality of independent elements in the video and a transmission frame rate of each of the independent elements, and send each of the independent elements to a corresponding plurality of video receiving ends respectively;
the video receiving end is configured to execute the video data transmission method according to any one of claims 11 to 12, and combine the received multiple independent elements into an image frame for display.
14. An electronic device, characterized in that the electronic device comprises: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor communicating with the storage medium via the bus when the electronic device is operating, the processor executing the machine-readable instructions to perform the steps of the video data transmission method of any one of claims 1-10 or the video data transmission method of any one of claims 11-12.
15. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the video data transmission method according to any one of claims 1 to 10 or the video data transmission method according to any one of claims 11 to 12.
CN202211005970.1A 2022-08-22 2022-08-22 Video data transmission method, system, processing device and storage medium Active CN115396696B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211005970.1A CN115396696B (en) 2022-08-22 2022-08-22 Video data transmission method, system, processing device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211005970.1A CN115396696B (en) 2022-08-22 2022-08-22 Video data transmission method, system, processing device and storage medium

Publications (2)

Publication Number Publication Date
CN115396696A true CN115396696A (en) 2022-11-25
CN115396696B CN115396696B (en) 2024-04-12

Family

ID=84120131

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211005970.1A Active CN115396696B (en) 2022-08-22 2022-08-22 Video data transmission method, system, processing device and storage medium

Country Status (1)

Country Link
CN (1) CN115396696B (en)

Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105959633A (en) * 2016-05-26 2016-09-21 北京志光伯元科技有限公司 Video transmission method and device
CN106792263A (en) * 2016-12-09 2017-05-31 东方网力科技股份有限公司 A kind of method of transmitting video data, apparatus and system
CN107396019A (en) * 2017-08-11 2017-11-24 维沃移动通信有限公司 A kind of slow motion video method for recording and mobile terminal
WO2017215279A1 (en) * 2016-06-12 2017-12-21 杭州海康威视数字技术股份有限公司 Video playback method and apparatus
WO2018076614A1 (en) * 2016-10-31 2018-05-03 武汉斗鱼网络科技有限公司 Live video processing method, apparatus and device, and computer readable medium
CN108600783A (en) * 2018-04-23 2018-09-28 深圳银澎云计算有限公司 A kind of method of frame rate adjusting, device and terminal device
CN109246433A (en) * 2018-09-26 2019-01-18 北京红云融通技术有限公司 Method for video coding and device, coding/decoding method and device, Video transmission system
CN110784745A (en) * 2019-11-26 2020-02-11 科大讯飞股份有限公司 Video transmission method, device, system, equipment and storage medium
CN112312229A (en) * 2020-10-27 2021-02-02 唐桥科技(杭州)有限公司 Video transmission method and device, electronic equipment and storage medium
CN112672154A (en) * 2020-12-15 2021-04-16 上海信联信息发展股份有限公司 Live video playing method and device, server and computer readable storage medium
EP3813363A1 (en) * 2020-05-13 2021-04-28 Beijing Baidu Netcom Science Technology Co., Ltd. Video frame transmission method, apparatus, electronic device and readable storage medium
CN113473131A (en) * 2021-07-01 2021-10-01 成都国科微电子有限公司 Video coding code rate dynamic adjustment method and device, electronic equipment and storage medium
CN113747222A (en) * 2021-09-07 2021-12-03 湖南五凌电力科技有限公司 Picture synchronous transmission method and system based on edge calculation
CN114079824A (en) * 2021-11-02 2022-02-22 深圳市洲明科技股份有限公司 Transmitting card, display device, computer device, storage medium, and control method for transmitting card
CN114363696A (en) * 2021-12-16 2022-04-15 广州方硅信息技术有限公司 Display processing method for inserting video support element, electronic terminal and storage medium
CN114666621A (en) * 2022-03-22 2022-06-24 阿里云计算有限公司 Page processing method, device and equipment

Patent Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105959633A (en) * 2016-05-26 2016-09-21 北京志光伯元科技有限公司 Video transmission method and device
WO2017215279A1 (en) * 2016-06-12 2017-12-21 杭州海康威视数字技术股份有限公司 Video playback method and apparatus
WO2018076614A1 (en) * 2016-10-31 2018-05-03 武汉斗鱼网络科技有限公司 Live video processing method, apparatus and device, and computer readable medium
CN106792263A (en) * 2016-12-09 2017-05-31 东方网力科技股份有限公司 A kind of method of transmitting video data, apparatus and system
CN107396019A (en) * 2017-08-11 2017-11-24 维沃移动通信有限公司 A kind of slow motion video method for recording and mobile terminal
CN108600783A (en) * 2018-04-23 2018-09-28 深圳银澎云计算有限公司 A kind of method of frame rate adjusting, device and terminal device
CN109246433A (en) * 2018-09-26 2019-01-18 北京红云融通技术有限公司 Method for video coding and device, coding/decoding method and device, Video transmission system
CN110784745A (en) * 2019-11-26 2020-02-11 科大讯飞股份有限公司 Video transmission method, device, system, equipment and storage medium
EP3813363A1 (en) * 2020-05-13 2021-04-28 Beijing Baidu Netcom Science Technology Co., Ltd. Video frame transmission method, apparatus, electronic device and readable storage medium
CN112312229A (en) * 2020-10-27 2021-02-02 唐桥科技(杭州)有限公司 Video transmission method and device, electronic equipment and storage medium
CN112672154A (en) * 2020-12-15 2021-04-16 上海信联信息发展股份有限公司 Live video playing method and device, server and computer readable storage medium
CN113473131A (en) * 2021-07-01 2021-10-01 成都国科微电子有限公司 Video coding code rate dynamic adjustment method and device, electronic equipment and storage medium
CN113747222A (en) * 2021-09-07 2021-12-03 湖南五凌电力科技有限公司 Picture synchronous transmission method and system based on edge calculation
CN114079824A (en) * 2021-11-02 2022-02-22 深圳市洲明科技股份有限公司 Transmitting card, display device, computer device, storage medium, and control method for transmitting card
CN114363696A (en) * 2021-12-16 2022-04-15 广州方硅信息技术有限公司 Display processing method for inserting video support element, electronic terminal and storage medium
CN114666621A (en) * 2022-03-22 2022-06-24 阿里云计算有限公司 Page processing method, device and equipment

Also Published As

Publication number Publication date
CN115396696B (en) 2024-04-12

Similar Documents

Publication Publication Date Title
CN110519621B (en) Video recommendation method and device, electronic equipment and computer readable medium
CN111950653B (en) Video processing method and device, storage medium and electronic equipment
US11748870B2 (en) Video quality measurement for virtual cameras in volumetric immersive media
Guttmann et al. Semi-automatic stereo extraction from video footage
CN104754413B (en) Method and apparatus for identifying television signals and recommending information based on image search
CN112565825A (en) Video data processing method, device, equipment and medium
CN107633441A (en) Commodity in track identification video image and the method and apparatus for showing merchandise news
CN112954450B (en) Video processing method and device, electronic equipment and storage medium
CN112565653B (en) Video frame insertion method, system, electronic equipment and storage medium
CN112584234A (en) Video image frame complementing method and related device
CN109389550B (en) Data processing method, device and computing equipment
CN110719416A (en) Live broadcast method, communication equipment and computer readable storage medium
CN112767313A (en) Video quality evaluation method and device and electronic equipment
CN115022679A (en) Video processing method, video processing device, electronic equipment and medium
CN111263183A (en) Singing state identification method and singing state identification device
CN106658095A (en) Webcasting video transmission method, server and user equipment
CN116437028B (en) Video display method and system
CN113473172B (en) VR video caching method and device, caching service device and storage medium
Nur Yilmaz A no reference depth perception assessment metric for 3D video
CN115396696B (en) Video data transmission method, system, processing device and storage medium
CN116168045A (en) Method and system for dividing sweeping lens, storage medium and electronic equipment
CN114501051B (en) Method and device for displaying marks of live objects, storage medium and electronic equipment
Göring et al. Automated genre classification for gaming videos
CN117278776A (en) Multichannel video content real-time comparison method and device, equipment and storage medium
US8090872B2 (en) Visual media viewing system and method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant