CN113747222A - Picture synchronous transmission method and system based on edge calculation - Google Patents
Picture synchronous transmission method and system based on edge calculation Download PDFInfo
- Publication number
- CN113747222A CN113747222A CN202111044004.6A CN202111044004A CN113747222A CN 113747222 A CN113747222 A CN 113747222A CN 202111044004 A CN202111044004 A CN 202111044004A CN 113747222 A CN113747222 A CN 113747222A
- Authority
- CN
- China
- Prior art keywords
- picture
- transmission
- transmitted
- edge side
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 147
- 230000001360 synchronised effect Effects 0.000 title claims abstract description 80
- 238000000034 method Methods 0.000 title claims abstract description 37
- 239000011159 matrix material Substances 0.000 claims abstract description 97
- 230000011218 segmentation Effects 0.000 claims abstract description 15
- 230000008859 change Effects 0.000 claims description 13
- 238000001514 detection method Methods 0.000 claims description 8
- 230000004044 response Effects 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 6
- 239000000872 buffer Substances 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 5
- 108010001267 Protein Subunits Proteins 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000012790 confirmation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
- H04N21/43076—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0852—Delays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
- H04L43/0894—Packet rate
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/141—Setup of application sessions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Environmental & Geological Engineering (AREA)
- Multimedia (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a picture synchronous transmission method and a system based on edge calculation, wherein the method comprises the following steps: establishing connection between the edge side and the synchronous side; the edge side obtains a frame picture to be transmitted and carries out matrix block segmentation on the frame picture to be transmitted; the edge side carries out picture comparison by taking the matrix block as a unit and carries out transmission decision according to the picture comparison information; and the synchronous side performs transmission feedback and picture display according to the transmission information of the edge side. The invention completes the decision of synchronous transmission of the picture through edge calculation, reduces the performance requirement and calculation of the synchronous side, and has strong practicability; meanwhile, the transmission pictures are processed by adopting a matrix block segmentation method, and transmission decision is carried out according to picture comparison information, so that stable and real-time picture synchronization is realized, the picture transmission quality is effectively ensured, the bandwidth is saved, and the time delay is reduced.
Description
Technical Field
The invention relates to the technical field of data transmission, in particular to a picture synchronous transmission method and system based on edge calculation.
Background
The synchronous transmission of the picture is widely applied to application scenes of the internet of things such as teleconferencing and monitoring systems. Due to the large bandwidth demand of video streams, when the same scene is faced for a long time, a large amount of invalid data transmission is generated, and when network signals fluctuate, the imaging quality of pictures is directly affected. The current major transmission techniques are the following:
(1) multiplexing synchronization technique
The technique simplifies synchronization between video streams by multiplexing data of multiple video streams into one data stream or one message without requiring an additional control channel and a synchronization clock. However, when a certain video stream does not have enough data, the time slice of the video stream cannot be effectively utilized, which causes waste of bandwidth resources, and the technique is not suitable for the situation that the video stream comes from different nodes.
(2) Time stamp synchronization technique
The technology makes the video data time-stamped according to the time sequence, and the data with the same time stamp are simultaneously expressed without additional channels and synchronous information and without changing the data stream. However, this technique requires a network-wide synchronized clock and the read and write time stamp overhead is large.
(3) Feedback synchronization techniques
The technology carries out synchronous control on a sending end or a receiving end according to the receiving end disorder detection information, and solves the problem of poor adaptability caused by different nodes of data source sources through a multi-source heterogeneous data fusion technology. However, the technology has certain hysteresis during picture synchronization, and the real-time performance is poor; in addition, the technology is suitable for a relatively stable network environment, and synchronous picture distortion is easy to occur when the network is unstable.
Disclosure of Invention
Based on the above, the invention provides a method and a system for synchronous picture transmission based on edge calculation, which are used for solving the problems of large data bandwidth occupation, high picture synchronization delay, picture distortion and the like in the conventional synchronous picture transmission.
Based on the above purpose, an embodiment of the present invention provides a method for synchronously transmitting pictures based on edge calculation, including:
establishing connection between the edge side and the synchronous side;
the edge side obtains a frame picture to be transmitted and carries out matrix block segmentation on the frame picture to be transmitted;
the edge side carries out picture comparison by taking the matrix block as a unit and carries out transmission decision according to the picture comparison information;
and the synchronous side performs transmission feedback and picture display according to the transmission information of the edge side.
Preferably, the picture synchronization transmission method based on edge calculation further includes:
and the synchronous side calculates network time delay and transmission rate according to a preset detection period, judges whether the network transmission condition is failed according to the network time delay and the transmission rate, and sends a data packet containing failure information to the edge side if the network transmission condition is judged to be failed.
Preferably, the obtaining, by the edge side, a frame picture to be transmitted, and performing matrix block segmentation on the frame picture to be transmitted includes:
the edge side determines the division quantity according to the resolution of the frame picture to be transmitted;
and the edge side divides the frame picture to be transmitted according to the division number by taking the pixel and the aspect ratio of the frame picture to be transmitted as the reference to obtain a plurality of matrix blocks.
Preferably, the image comparison is performed by the edge side using a matrix block as a unit, and a transmission decision is performed according to the image comparison information, including:
the edge side judges whether the synchronous side caches the frame picture to be transmitted or not;
if so, the edge side compares the frame picture to be transmitted with the previous frame picture by taking the matrix block as a unit to acquire picture comparison information; the picture comparison information is used for judging whether the picture of the frame to be transmitted and the previous picture are changed or not;
the edge side makes a transmission decision according to the picture comparison information so as to send corresponding transmission information to the synchronous side;
and when the edge side detects that the frame to be transmitted is empty, the edge side sends a data packet containing a terminator to the synchronization side and releases the connection.
Preferably, the comparing, by the edge side, the frame picture to be transmitted with the previous frame picture by using the matrix block as a unit to obtain picture comparison information, including:
preprocessing each matrix block by the edge side, and calculating the gray average value of all pixels according to the preprocessed matrix blocks;
the edge side obtains the characteristic value of each pixel by comparing the gray level of each pixel with the gray level average value, and obtains the fingerprint information of the matrix block according to the characteristic value of each pixel;
the edge side obtains picture comparison information by comparing the fingerprint information of the matrix block corresponding to the frame picture to be transmitted and the previous frame picture; the picture comparison information comprises that the picture of the frame to be transmitted is not changed, partially changed and completely changed.
Preferably, the edge side performs a transmission decision according to the picture comparison information to send corresponding transmission information to the synchronization side, including:
when the picture comparison information indicates that the picture of the frame to be transmitted does not change, sending a timestamp to the synchronization side;
when the picture comparison information indicates that the picture of the frame to be transmitted partially changes, acquiring a plurality of matrix blocks covering a change area in equal quantity to construct a matrix group, and sending the matrix group to the synchronization side;
and when the picture comparison information indicates that the frame picture to be transmitted is completely changed, sending the frame picture to be transmitted to the synchronous side.
Preferably, the edge side establishes a connection with the synchronization side, including:
the synchronous side sends a connection request to the edge side;
the edge side receives the connection request and sends a connection response instruction to the synchronous side;
the synchronous side receives the connection response instruction, sends a data packet to the edge side and displays the successful connection; the data packet comprises network time delay, network bandwidth, a timestamp and a cache state of a frame picture to be transmitted.
In addition, the embodiment of the invention also provides a picture synchronous transmission system based on edge calculation, which comprises an edge side and a synchronous side connected with the edge side;
the edge side comprises a picture segmentation module and a transmission decision module; the picture division module is used for acquiring a frame picture to be transmitted and carrying out matrix block division on the frame picture to be transmitted; the transmission decision module is used for carrying out picture comparison by taking a matrix block as a unit and carrying out transmission decision according to the picture comparison information;
the synchronous side comprises a transmission feedback module and a picture display module; the transmission feedback module is used for performing transmission feedback according to the transmission information of the edge side; the picture display module is used for displaying pictures according to the transmission information of the edge side:
preferably, the transmission feedback module of the synchronization side is further configured to calculate a network delay and a transmission rate according to a preset detection period, determine whether the network transmission condition fails according to the network delay and the transmission rate, and send a data packet including failure information to the edge side if it is determined that the network transmission condition fails.
As can be seen from the above, the method for frame synchronous transmission based on edge calculation according to the embodiment of the present invention has the following effective effects: the decision of synchronous transmission of the picture is completed through edge calculation, so that the performance requirement and the calculation of a synchronous side are reduced, and the practicability is high; meanwhile, the transmission pictures are processed by adopting a matrix block segmentation method, and transmission decision is carried out according to picture comparison information, so that stable and real-time picture synchronization is realized, the picture transmission quality is effectively ensured, the bandwidth is saved, and the time delay is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart illustrating a method for frame synchronization transmission based on edge calculation according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating the step S30 of the method for synchronous transmission of frames based on edge calculation according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a method for synchronous transmission of frames based on edge calculation according to another embodiment of the present invention;
FIG. 4 is a block diagram of a frame synchronization transmission system based on edge calculation according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects to be solved by the present invention more clearly apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, an embodiment of the present invention provides a method for synchronously transmitting frames based on edge calculation, which specifically includes the following steps:
in step S10, the edge side establishes a connection with the sync side.
In the embodiment, the edge side is used for acquiring an original picture for coding transmission; the synchronous side is used for obtaining the compressed picture for decoding and displaying, and before picture synchronous transmission, the edge side and the synchronous side establish connection through strong confirmation.
Preferably, step S10 specifically includes the following steps:
step S101, the synchronous side sends a connection request to the edge side.
Step S102, the edge side receives the connection request and sends a connection response instruction to the synchronous side.
And step S103, the synchronous side receives the connection response instruction, sends a data packet to the edge side and displays the successful connection.
In this embodiment, the data packet includes information such as network delay, network bandwidth, timestamp, and buffer status of the frame to be transmitted.
That is, before the synchronous transmission of the picture, the synchronous side first initiates a connection request, and then sends a data packet to the synchronous side according to a connection confirmation instruction sent by the edge side in response to the connection request, thereby completing the communication connection between the edge side and the synchronous side.
And step S20, the edge side acquires the frame picture to be transmitted and performs matrix block segmentation on the frame picture to be transmitted.
In this embodiment, before performing the picture synchronous transmission, the edge performs matrix block segmentation on the frame picture to be transmitted to obtain a plurality of (k) matrix blocks, and further, the matrix blocks can be numbered to facilitate the subsequent comparison of different frame pictures.
Preferably, step S20 specifically includes the following steps:
in step S201, the edge determines the division number according to the resolution of the frame to be transmitted.
In step S202, the edge divides the frame picture to be transmitted according to the number of divisions by using the pixels and the aspect ratio of the frame picture to be transmitted as references, and obtains a plurality of (k) matrix blocks.
In this embodiment, the division number is in direct proportion to the resolution of the frame picture to be transmitted, and the higher the resolution of the frame picture to be transmitted is, the larger the value k of the division number is. Optionally, the value k of the number of divisions satisfies 4 ≦ k ≦ 40.
Specifically, before picture synchronous transmission, the edge side firstly confirms the division number k according to the resolution ratio of the frame picture to be transmitted, the proportional relation between the resolution ratio and the division number, and then divides the frame picture to be transmitted into k matrix blocks with the size of a multiplied by b by taking the pixel and the aspect ratio (the ratio of the longitudinal dimension to the transverse dimension) a: b of the frame picture to be transmitted as a reference.
And step S30, the edge side carries out picture comparison by taking the matrix block as a unit and carries out transmission decision according to the picture comparison information.
In this embodiment, the edge side performs picture synchronization transmission by lossless coding, and during transmission, it is first determined whether the synchronization side buffers a frame picture to be transmitted, and if the synchronization side does not buffer the frame picture to be transmitted, it is determined that the frame picture to be transmitted is transmitted for the first time, and the edge side transmits all contents of the frame picture to be transmitted to the synchronization side; if the synchronous side caches the frame picture to be transmitted, the edge side obtains picture comparison information used for judging whether the frame picture to be transmitted changes (namely judging whether the frame picture to be transmitted is consistent with the previous frame picture) by comparing the frame picture to be transmitted with the previous frame picture, and performs transmission decision according to the picture comparison information to obtain corresponding transmission information.
Preferably, as shown in fig. 2, step S30 includes the steps of:
in step S301, the edge side determines whether the synchronization side buffers a frame to be transmitted.
In this embodiment, when the edge side performs the picture synchronous transmission, the buffer status of the frame picture to be transmitted is determined according to the data packet sent by the synchronization side, and then it is determined whether the synchronization side buffers the frame picture to be transmitted. Understandably, when the caching state of the frame picture to be transmitted is cached, judging that the frame picture to be transmitted is cached at the synchronous side; and when the caching state of the frame picture to be transmitted is not cached, judging that the frame picture to be transmitted is not cached at the synchronous side.
Step S302, if yes, the edge side compares the frame picture to be transmitted with the previous frame picture by taking the matrix block as a unit to acquire picture comparison information; the frame comparison information is used for judging whether the frame to be transmitted changes; if not, the frame picture to be transmitted is sent to the synchronous side.
Preferably, if it is determined that the synchronization side buffers the frame picture to be transmitted, picture comparison is performed through steps S3021 to S3023; otherwise, the frame picture to be transmitted is sent to the synchronous side, and correspondingly, the synchronous side carries out coverage updating according to all the contents of the frame picture to be transmitted. Specifically, the step S302 of comparing the frame picture to be transmitted with the previous frame picture by using the matrix block as a unit at the edge side includes the following steps:
step S3021, the edge side performs preprocessing on each matrix block, and calculates the average grayscale value of all pixels according to the preprocessed matrix block. Wherein the preprocessing includes a reduction processing and a gradation processing.
Step S3022, the edge side obtains the feature value of each pixel by comparing the gray level of each pixel with the average gray level value, and obtains the fingerprint information of the matrix block according to the feature value of each pixel.
Step S3023, the edge side obtains picture comparison information by comparing fingerprint information of the matrix block corresponding to the frame picture to be transmitted and the previous frame picture, where the picture comparison information includes that the frame picture to be transmitted does not change, partially changes, and completely changes.
Understandably, when picture comparison is performed on the edge side, firstly, for each matrix block, the matrix block is subjected to reduction processing to remove detail information in a picture, a matrix block with a preset size of z × z is obtained, gray processing is performed on the reduced matrix block to obtain a gray picture, and then the gray average value of all pixels (namely z × z pixels) in the matrix block is calculated.
Then, comparing the gray level of each pixel in the matrix block with the average gray level value, and if the gray level of a certain pixel is greater than or equal to the average gray level value, determining that the characteristic value of the pixel is 1; if the gray scale of a certain pixel is smaller than the average gray scale value, the characteristic value of the pixel is determined to be 0, and then the fingerprint information of the matrix block is formed according to the characteristic value (namely 1 or 0) of each pixel in the matrix block. Wherein, the fingerprint information of the matrix block is an integer of z × z bits. Alternatively, the preset size zxz is 16 x 16, and accordingly, the fingerprint information of the matrix block is an integer of 256 bits.
And finally, comparing the frame picture to be transmitted with the matrix block corresponding to the previous frame picture, namely the two matrix blocks with the same number, judging whether the fingerprint information of the two matrix blocks with the same number is different, if the fingerprint information of the two matrix blocks with the same number is different, determining that the frame picture to be transmitted is the same as the matrix block corresponding to the previous frame picture, and if the fingerprint information of the two matrix blocks with the same number is not different, determining that the matrix block corresponding to the frame picture to be transmitted is different from the matrix block corresponding to the previous frame picture, and further obtaining picture comparison information whether the frame picture to be transmitted is changed. It can be understood that if k matrix blocks of a frame to be transmitted and a previous frame are the same, it is determined that the frame to be transmitted does not change, if k matrix blocks of the frame to be transmitted and the previous frame are different, it is determined that the frame to be transmitted completely changes, and if x (x < k) matrix blocks of the frame to be transmitted and the previous frame are different, it is determined that the frame to be transmitted partially changes.
Step S303, the edge side makes a transmission decision according to the picture comparison information so as to send corresponding transmission information to the synchronous side.
Preferably, the edge side performs a transmission decision through steps S3031 to S3032 according to the picture comparison information obtained in the above steps, so as to send corresponding transmission information to the synchronization side. Specifically, step S303 includes the following steps:
step S3031, when the picture comparison information indicates that the frame picture to be transmitted does not change, sending the timestamp to the synchronization side.
Step S3032, when the picture comparison information indicates that the picture of the frame to be transmitted partially changes, obtaining a plurality of matrix blocks covering the change area in equal amount to construct a matrix group, and sending the matrix group to the synchronization side.
Step S3033, when the picture comparison information indicates that the frame picture to be transmitted has a complete change, the frame picture to be transmitted is sent to the synchronization side.
It can be understood that when all matrix blocks (k matrix blocks) of the frame picture to be transmitted and the previous frame picture are the same, the frame picture to be transmitted is determined to be unchanged, the timestamp is used as transmission information and sent to the synchronization side, and accordingly, the synchronization side updates the timestamp and keeps the frame picture to be transmitted as the content of the next frame picture.
And when the frame picture to be transmitted is the same as a partial matrix block (x matrix blocks, and x is less than k) of the previous frame picture, determining that the frame picture to be transmitted is partially changed, acquiring x matrix blocks covering a change area with the same quantity to construct a matrix group, sending the matrix group to a synchronization side as transmission information, correspondingly, decoding the matrix group by the synchronization side, and synchronously updating k-x matrix blocks in the picture to be transmitted.
And when all the matrixes (k matrix blocks) of the frame picture to be transmitted and the previous frame picture are different, determining that the frame picture to be transmitted is completely changed, namely determining scene change, sending the frame picture to be transmitted to a synchronous side as transmission information, and correspondingly, performing coverage updating on the synchronous side according to all the contents of the frame picture to be transmitted.
Step S304, when the edge side detects that the frame to be transmitted is empty, the edge side sends a data packet containing a terminator to the synchronization side, and releases the connection.
Understandably, when the synchronous transmission of the picture is carried out at the edge side, whether a frame to be transmitted is empty is detected, if the frame to be transmitted is empty, the transmission task is judged to be terminated, a data packet containing a terminator is sent to the synchronous side, and the connection is released; and if the frame to be transmitted is not empty, the next frame is continuously sent to the synchronous side.
In the embodiment, the image comparison at the edge side is performed through the calculation of the similarity of the matrix block corresponding to the frame image to be transmitted and the previous frame image, so that the efficiency and the accuracy of the image comparison are improved, the image synchronous transmission decision at the edge side is further completed through the image comparison information, the real-time performance of the transmission decision is improved, the performance requirement and the calculation cost at the synchronous side are reduced, and the practicability is higher.
In step S40, the sync side performs transmission feedback and picture display according to the transmission information of the edge side.
In this embodiment, the synchronization side updates the next frame picture according to the transmission information sent by the edge side, and sends a data packet to the edge side when updating the next frame, where the data packet includes information such as network delay, network bandwidth, timestamp, and cache state of the frame picture to be transmitted.
Furthermore, the synchronous side displays the next frame of picture by adopting a matrix block segmentation method which is the same as that of the edge side, and caches the picture with a preset cache period to provide a basis for displaying the picture in the next second and provide a corresponding data packet for the edge side to continuously transmit after the network is reconnected. Wherein the preset buffer period is 25 frames, namely 1 second.
As can be seen from the above, the method for transmitting frames synchronously based on edge calculation according to the present embodiment has the following effective effects: the decision of synchronous transmission of the picture is completed through edge calculation, so that the performance requirement and the calculation of a synchronous side are reduced, and the practicability is high; meanwhile, the transmission pictures are processed by adopting a matrix block segmentation method, and transmission decision is carried out according to picture comparison information, so that stable and real-time picture synchronization is realized, the picture transmission quality is effectively ensured, the transmission bandwidth is saved, and the transmission delay is reduced.
As shown in fig. 3, in another embodiment, the method for transmitting picture synchronization based on edge calculation further includes:
and S50, the synchronization side calculates the network delay and the transmission rate every other preset detection period, judges whether the network transmission condition is failed according to the network delay and the transmission rate, and sends a data packet containing failure information to the edge side if the network transmission condition is judged to be failed.
In the present embodiment, the preset detection period is 50 frames, i.e., 2 seconds.
Specifically, the synchronization side calculates network delay and transmission rate once every 50 frames, respectively detects whether the network delay is greater than or equal to a delay preset value and detects whether the transmission rate is less than or equal to a rate preset value, when the network delay is greater than or equal to the delay preset value and/or the transmission rate is less than or equal to the rate preset value, the network transmission condition fault is judged, and a data packet containing fault information is immediately sent to the edge side, and correspondingly, when the edge side receives the data packet containing the fault information, the frame picture to be transmitted is cached according to a timestamp; and when the network delay is smaller than the delay preset value and the transmission rate is larger than the rate preset value, the network transmission condition is judged to be normal, and at the moment, a data packet containing fault information does not need to be sent to the edge side. The failure information may be a failure symbol.
It can be understood that the present embodiment maintains the connection between the edge side and the synchronization side through various feedbacks, and handles the connection interruption through the dual buffer mechanism, so that the image transmission reliability is stronger.
As shown in fig. 4, in addition, an embodiment of the present invention further provides a picture synchronization transmission system based on edge calculation, which includes an edge side 10, a synchronization side 20 connected to the edge side 10;
the edge side 10 includes a picture segmentation module 110 and a transmission decision module 120; the frame dividing module 110 is configured to obtain a frame to be transmitted, and perform matrix block division on the frame to be transmitted; the transmission decision module 120 is configured to perform picture comparison by using the matrix block as a unit, and perform transmission decision according to the picture comparison information;
the synchronization side 20 includes a transmission feedback module 210 and a screen display module 220; the transmission feedback module 210 is configured to perform transmission feedback according to the transmission information of the edge side; the image display module 220 is used for displaying images according to the transmission information of the edge side.
Further, as shown in fig. 4, the transmission feedback module 210 of the synchronization side 20 is further configured to calculate a network delay and a transmission rate according to a preset detection period, determine whether the network transmission condition fails according to the network delay and the transmission rate, and send a data packet including failure information to the edge side if it is determined that the network transmission condition fails.
Further, the frame segmentation module 110 of the edge side 10 includes the following sub-units, and the detailed description of each functional unit is as follows:
the quantity determining unit is used for determining the dividing quantity according to the resolution of the frame picture to be transmitted;
and the dividing unit is used for dividing the frame picture to be transmitted according to the dividing quantity by taking the pixel and the aspect ratio of the frame picture to be transmitted as the reference to obtain a plurality of matrix blocks.
Further, the transmission decision module 120 of the edge side 10 includes the following sub-units, and the detailed description of each functional unit is as follows:
the buffer judging unit is used for judging whether the synchronous side buffers the frame picture to be transmitted or not;
the picture comparison unit is used for comparing the frame picture to be transmitted with the previous frame picture by taking the matrix block as a unit if the frame picture to be transmitted is the previous frame picture, and obtaining picture comparison information; the frame comparison information is used for judging whether a frame to be transmitted and a previous frame are changed or not;
the transmission decision unit is used for carrying out transmission decision according to the picture comparison information so as to send corresponding transmission information to the synchronous side;
and the connection release unit is used for sending the data packet containing the terminator to the synchronous side and releasing the connection when detecting that the frame to be transmitted is empty.
Further, the picture comparison unit includes the following sub-units, and the detailed description of each functional unit is as follows:
the picture preprocessing unit is used for preprocessing each matrix block and calculating the gray average value of all pixels according to the preprocessed matrix blocks;
the fingerprint information acquisition unit is used for acquiring the characteristic value of each pixel by comparing the gray level of each pixel with the average value of the gray levels and acquiring the fingerprint information of the matrix block according to the characteristic value of each pixel;
the fingerprint information comparison unit is used for comparing fingerprint information of a matrix block corresponding to a frame picture to be transmitted and a previous frame picture to obtain picture comparison information; the picture contrast information contains that the picture of the frame to be transmitted is not changed, is partially changed and is completely changed.
Further, the transmission decision unit includes the following sub-units, and the detailed description of each functional unit is as follows:
the first transmission unit is used for sending the time stamp to the synchronization side when the picture comparison information indicates that the frame picture to be transmitted does not change;
the second transmission unit is used for acquiring a plurality of matrix blocks covering a change area in equal quantity to construct a matrix group and sending the matrix group to the synchronization side when the picture comparison information is that the picture of the frame to be transmitted partially changes;
and the third transmission unit is used for sending the frame picture to be transmitted to the synchronous side when the picture comparison information indicates that the frame picture to be transmitted is completely changed.
The system of the foregoing embodiment is used to implement the corresponding method in the foregoing embodiment, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to those examples; within the idea of the invention, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity.
The present embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omissions, modifications, substitutions, improvements and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the invention.
Claims (9)
1. A picture synchronous transmission method based on edge calculation is characterized by comprising the following steps:
establishing connection between the edge side and the synchronous side;
the edge side obtains a frame picture to be transmitted and carries out matrix block segmentation on the frame picture to be transmitted;
the edge side carries out picture comparison by taking the matrix block as a unit and carries out transmission decision according to the picture comparison information;
and the synchronous side performs transmission feedback and picture display according to the transmission information of the edge side.
2. The picture synchronization transmission method based on edge calculation as claimed in claim 1, further comprising:
and the synchronous side calculates network time delay and transmission rate according to a preset detection period, judges whether the network transmission condition is failed according to the network time delay and the transmission rate, and sends a data packet containing failure information to the edge side if the network transmission condition is judged to be failed.
3. The image synchronous transmission method based on edge calculation according to claim 1, wherein the edge side obtains a frame image to be transmitted and performs matrix block segmentation on the frame image to be transmitted, including:
the edge side determines the division quantity according to the resolution of the frame picture to be transmitted;
and the edge side divides the frame picture to be transmitted according to the division number by taking the pixel and the aspect ratio of the frame picture to be transmitted as the reference to obtain a plurality of matrix blocks.
4. The method of claim 1, wherein the edge side performs picture comparison in units of matrix blocks and performs transmission decision according to the picture comparison information, and the method comprises:
the edge side judges whether the synchronous side caches the frame picture to be transmitted or not;
if so, the edge side compares the frame picture to be transmitted with the previous frame picture by taking the matrix block as a unit to acquire picture comparison information; the picture comparison information is used for judging whether the picture of the frame to be transmitted and the previous picture are changed or not;
the edge side makes a transmission decision according to the picture comparison information so as to send corresponding transmission information to the synchronous side;
and when the edge side detects that the frame to be transmitted is empty, the edge side sends a data packet containing a terminator to the synchronization side and releases the connection.
5. The method according to claim 4, wherein the edge side compares the frame to be transmitted with the previous frame by taking a matrix block as a unit to obtain the frame comparison information, and comprises:
preprocessing each matrix block by the edge side, and calculating the gray average value of all pixels according to the preprocessed matrix blocks;
the edge side obtains the characteristic value of each pixel by comparing the gray level of each pixel with the gray level average value, and obtains the fingerprint information of the matrix block according to the characteristic value of each pixel;
the edge side obtains picture comparison information by comparing the fingerprint information of the matrix block corresponding to the frame picture to be transmitted and the previous frame picture; the picture comparison information comprises that the picture of the frame to be transmitted is not changed, partially changed and completely changed.
6. The method as claimed in claim 5, wherein the edge side makes a transmission decision according to the picture comparison information to send corresponding transmission information to the synchronization side, and the method comprises:
when the picture comparison information indicates that the picture of the frame to be transmitted does not change, sending a timestamp to the synchronization side;
when the picture comparison information indicates that the picture of the frame to be transmitted partially changes, acquiring a plurality of matrix blocks covering a change area in equal quantity to construct a matrix group, and sending the matrix group to the synchronization side;
and when the picture comparison information indicates that the frame picture to be transmitted is completely changed, sending the frame picture to be transmitted to the synchronous side.
7. The picture synchronization transmission method based on edge calculation as claimed in claim 1, wherein the edge side establishes connection with the synchronization side, comprising:
the synchronous side sends a connection request to the edge side;
the edge side receives the connection request and sends a connection response instruction to the synchronous side;
the synchronous side receives the connection response instruction, sends a data packet to the edge side and displays the successful connection; the data packet comprises network time delay, network bandwidth, a timestamp and a cache state of a frame picture to be transmitted.
8. A picture synchronous transmission system based on edge calculation is characterized by comprising an edge side and a synchronous side connected with the edge side;
the edge side comprises a picture segmentation module and a transmission decision module; the picture division module is used for acquiring a frame picture to be transmitted and carrying out matrix block division on the frame picture to be transmitted; the transmission decision module is used for carrying out picture comparison by taking a matrix block as a unit and carrying out transmission decision according to the picture comparison information;
the synchronous side comprises a transmission feedback module and a picture display module; the transmission feedback module is used for performing transmission feedback according to the transmission information of the edge side; and the picture display module is used for displaying pictures according to the transmission information of the edge side.
9. The system according to claim 8, wherein the transmission feedback module of the synchronization side is further configured to calculate a network delay and a transmission rate according to a preset detection period, determine whether the network transmission condition is faulty according to the network delay and the transmission rate, and send a data packet containing fault information to the edge side if it is determined that the network transmission condition is faulty.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111044004.6A CN113747222A (en) | 2021-09-07 | 2021-09-07 | Picture synchronous transmission method and system based on edge calculation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111044004.6A CN113747222A (en) | 2021-09-07 | 2021-09-07 | Picture synchronous transmission method and system based on edge calculation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113747222A true CN113747222A (en) | 2021-12-03 |
Family
ID=78736573
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111044004.6A Pending CN113747222A (en) | 2021-09-07 | 2021-09-07 | Picture synchronous transmission method and system based on edge calculation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113747222A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115396696A (en) * | 2022-08-22 | 2022-11-25 | 网易（杭州）网络有限公司 | Video data transmission method, system, processing device and storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102208026A (en) * | 2011-05-27 | 2011-10-05 | 电子科技大学 | Method for extracting digital video fingerprints |
| CN103235956A (en) * | 2013-03-28 | 2013-08-07 | 天脉聚源(北京)传媒科技有限公司 | Method and device for detecting advertisements |
| CN104462873A (en) * | 2013-09-13 | 2015-03-25 | 北大方正集团有限公司 | Picture processing method and picture processing device |
| KR20160027651A (en) * | 2014-09-02 | 2016-03-10 | 에스케이텔레콤 주식회사 | Method and apparatus for indexing moving picture |
| CN106681679A (en) * | 2017-01-19 | 2017-05-17 | 深圳市云点科技有限公司 | Remote desktop picture transmission method and device |
| CN108965806A (en) * | 2018-07-12 | 2018-12-07 | 江门市金佣网有限公司 | A kind of data transmission method and device based on long-range sales exhibition system |
| CN110366000A (en) * | 2019-08-30 | 2019-10-22 | 北京青岳科技有限公司 | A kind of video pictures transmission method and system |
| CN110913181A (en) * | 2019-12-06 | 2020-03-24 | 太原师范学院 | Warehouse video monitoring system based on edge calculation |
| WO2021012863A1 (en) * | 2019-07-25 | 2021-01-28 | 深圳创维－Rgb电子有限公司 | Detection method for still picture of video, terminal, and computer-readable storage medium |
| CN112291587A (en) * | 2019-07-25 | 2021-01-29 | 上海达龙信息科技有限公司 | Dynamic video frame processing method, system, medium and server of remote desktop |
| WO2021052323A1 (en) * | 2019-09-20 | 2021-03-25 | 京东方科技集团股份有限公司 | Video signal sending method, video signal receiving method, video signal sending apparatus, video signal receiving apparatus, and display device |
| US20210105367A1 (en) * | 2018-06-21 | 2021-04-08 | Autel Robotics Co., Ltd. | Data transmission control method, information sending end and receiving end and aerial vehicle image transmission system |
-
2021
- 2021-09-07 CN CN202111044004.6A patent/CN113747222A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102208026A (en) * | 2011-05-27 | 2011-10-05 | 电子科技大学 | Method for extracting digital video fingerprints |
| CN103235956A (en) * | 2013-03-28 | 2013-08-07 | 天脉聚源(北京)传媒科技有限公司 | Method and device for detecting advertisements |
| CN104462873A (en) * | 2013-09-13 | 2015-03-25 | 北大方正集团有限公司 | Picture processing method and picture processing device |
| KR20160027651A (en) * | 2014-09-02 | 2016-03-10 | 에스케이텔레콤 주식회사 | Method and apparatus for indexing moving picture |
| CN106681679A (en) * | 2017-01-19 | 2017-05-17 | 深圳市云点科技有限公司 | Remote desktop picture transmission method and device |
| US20210105367A1 (en) * | 2018-06-21 | 2021-04-08 | Autel Robotics Co., Ltd. | Data transmission control method, information sending end and receiving end and aerial vehicle image transmission system |
| CN108965806A (en) * | 2018-07-12 | 2018-12-07 | 江门市金佣网有限公司 | A kind of data transmission method and device based on long-range sales exhibition system |
| WO2021012863A1 (en) * | 2019-07-25 | 2021-01-28 | 深圳创维－Rgb电子有限公司 | Detection method for still picture of video, terminal, and computer-readable storage medium |
| CN112291587A (en) * | 2019-07-25 | 2021-01-29 | 上海达龙信息科技有限公司 | Dynamic video frame processing method, system, medium and server of remote desktop |
| CN110366000A (en) * | 2019-08-30 | 2019-10-22 | 北京青岳科技有限公司 | A kind of video pictures transmission method and system |
| WO2021052323A1 (en) * | 2019-09-20 | 2021-03-25 | 京东方科技集团股份有限公司 | Video signal sending method, video signal receiving method, video signal sending apparatus, video signal receiving apparatus, and display device |
| CN110913181A (en) * | 2019-12-06 | 2020-03-24 | 太原师范学院 | Warehouse video monitoring system based on edge calculation |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115396696A (en) * | 2022-08-22 | 2022-11-25 | 网易（杭州）网络有限公司 | Video data transmission method, system, processing device and storage medium |
| CN115396696B (en) * | 2022-08-22 | 2024-04-12 | 网易（杭州）网络有限公司 | Video data transmission method, system, processing device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7483489B2 (en) | Streaming multimedia data over a network having a variable bandwith | |
| KR101511674B1 (en) | Apparatus and method for controlling of intra-refreshing in video telephony communication system | |
| EP0753968B1 (en) | Moving picture transmission system and moving picture transmission apparatus | |
| US6684354B2 (en) | Data transmission method, data transmission apparatus, data receiving apparatus, and packet data structure | |
| US20070280129A1 (en) | System and method for calculating packet loss metric for no-reference video quality assessment | |
| JP2009545204A (en) | System and method for jitter buffer reduction in scalable coding | |
| CN111147860B (en) | Video data decoding method and device | |
| US20070116129A1 (en) | System and method for transmitting digital video stream using SVC scheme | |
| CN113747222A (en) | Picture synchronous transmission method and system based on edge calculation | |
| JPH10145794A (en) | Image encoding method and device | |
| EP1298926A1 (en) | Information presentation device and method | |
| CN116634187A (en) | Video stream processing method, device, equipment and medium based on confluence sending mechanism | |
| US11575608B2 (en) | Managing data flow between source node and recipient node | |
| JP2000050235A (en) | Supervisory device | |
| CN112565693B (en) | Method, system and equipment for monitoring video on demand | |
| JP2000278720A (en) | Device and method for evaluating dynamic picture communication | |
| CN112272305B (en) | Multi-channel real-time interactive video cache storage method | |
| CN114866763A (en) | Video quality evaluation method and device, terminal equipment and storage medium | |
| JP3259691B2 (en) | Multicast system and receiving device | |
| KR100363550B1 (en) | Encoder and decoder in a wireless terminal for retransmitting a moving picture | |
| CN113489745B (en) | Video data transmission method, device, equipment and storage medium | |
| CN115426503B (en) | Transcoding service switching method, device, equipment and storage medium | |
| KR100527846B1 (en) | Method of controlling bit rate in digital audio broadcasting system and apparatus and method for converting media stream using it | |
| JP2004328353A (en) | Correction method and apparatus for inverted stream packet | |
| KR930003202B1 (en) | Transmitting apparatus of digital picture image signal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |