CN108495128A - Video transmission system based on super-resolution rebuilding technology and transmission method - Google Patents
Video transmission system based on super-resolution rebuilding technology and transmission method Download PDFInfo
- Publication number
- CN108495128A CN108495128A CN201810195620.3A CN201810195620A CN108495128A CN 108495128 A CN108495128 A CN 108495128A CN 201810195620 A CN201810195620 A CN 201810195620A CN 108495128 A CN108495128 A CN 108495128A
- Authority
- CN
- China
- Prior art keywords
- video
- super
- macro block
- type
- resolution rebuilding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 53
- 238000005516 engineering process Methods 0.000 title claims abstract description 30
- 238000000034 method Methods 0.000 title claims abstract description 21
- 238000013459 approach Methods 0.000 claims abstract description 3
- 238000013139 quantization Methods 0.000 claims description 20
- 239000013598 vector Substances 0.000 claims description 18
- 238000013527 convolutional neural network Methods 0.000 claims description 16
- 238000004891 communication Methods 0.000 claims description 11
- 238000005070 sampling Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 4
- 235000013399 edible fruits Nutrition 0.000 claims 2
- 241000208340 Araliaceae Species 0.000 claims 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 claims 1
- 235000003140 Panax quinquefolius Nutrition 0.000 claims 1
- 235000008434 ginseng Nutrition 0.000 claims 1
- 230000000694 effects Effects 0.000 description 4
- 230000004913 activation Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000004888 barrier function Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
- H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Discrete Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The present invention relates to technical field of video transmission, specially a kind of Video transmission system based on super-resolution rebuilding technology and corresponding video transmission method, the system includes gateway device, for connecting client and video server, gateway device can receive video and the video received is carried out super-resolution rebuilding and by the transmission of video after super-resolution rebuilding to client.This approach includes the following steps：Video reception step, gateway device receive the video data that video server is sent；Super-resolution rebuilding step, gateway device carry out super-resolution rebuilding to video data；Video forwarding step, gateway device is by the transmission of video after super-resolution rebuilding to client.Video transmission system and transmission method provided by the invention based on super-resolution rebuilding technology, it is of high cost can to solve the High-Quality Video Transmission that existing video transmission manner and Transmission system are brought due to the limitation of public-network bandwidth, flow and expense, the problems such as poor user experience.
Description
Technical field
The present invention relates to technical field of video transmission, in particular to a kind of regarding based on super-resolution rebuilding technology
Frequency Transmission system and transmission method.
Background technology
With science technology and society expanding economy, display technology and display equipment have obtained significant progress, with hand
For machine, in the time of a few years, the resolution ratio of smart mobile phone screen is realized from 720P to 1080P, then from 1080P to 2K
Leap, and the smart mobile phone of latest version even has begun the screen using 4K resolution ratio, and high-resolution screen is theoretical
On can bring the finer and smoother image quality of user, preferably experienced to user, however true in use, its effect is but often by it
The constraint of the resolution ratio of broadcasting content.Many people had with experience：Splendid, the details exhibition in the 4K television images that market is seen
It is now very perfect, but when TV is seen after buying back home have been found that result of broadcast is general, this is because market play be all and electricity
Depending on the high-resolution video of resolution match, and the screen resolution in radio station is usually relatively low, the advantage of high-resolution screen and
Meaning is unable to get sufficient embodiment when playing the video of low resolution.
Above-mentioned problem can be solved by two ways, and a kind of is exactly that the provider of video content externally exports height
The video content of resolution ratio, solves the problems, such as from the root cause；Another kind be exactly client by some algorithms to the image quality of video into
Row is adjusted, to improve display effect.
Be limited to existing technical conditions, above two solution can there is a problem of in specific implementation it is many, first
Higher-quality video content, which is meant that, needs to transmit more data contents, from the perspective of content supplier, they
Video server not only to consume more bandwidth and flow, but also to have stronger processing capacity, this is unquestionably
The cost of content supplier can be increased；From the perspective of user, even if provider provides the video content of high quality, still
By existing public network transmission speed and flow and expense the problems such as limited, user can only also receive regarding for low-definition version
Frequency content；Secondly, the computing capability of existing hardware device especially mobile device and intelligent television equipment is limited, client
The adjusting of video image quality is also only limitted to that simple interpolation algorithm is used to sample low resolution video content to match display
The resolution ratio of equipment, effect is poor, unsatisfactory.
Invention content
An object of the present invention is to provide a kind of Video transmission system based on super-resolution rebuilding technology, can solve
Existing video transmission manner and Transmission system are limited to public-network bandwidth and expense and existing High-Quality Video Transmission is of high cost,
And then the problem of influencing user experience.
In order to solve the above-mentioned technical problem, this patent provides the following technical solutions：
Based on the Video transmission system of super-resolution rebuilding technology, including：
Gateway device, the gateway device is for connecting client and video server, and the gateway device is for receiving
The video received is simultaneously carried out super-resolution rebuilding by the video of video server transmission, and the gateway device is additionally operable to oversubscription
Transmission of video after resolution reconstruction is to client.
In technical solution of the present invention, by the way that gateway device is arranged between video server and client, and gateway is used
Equipment carries out video data as the intermediate member for transmitting data, and between the video data of forwarding video server
Super-resolution rebuilding is then forwarded to client, client either user video playing terminal, such as TV box, hand
Mechanical, electrical brain etc. can also be other network equipments of next stage, such as interchanger, router.Video server is to gateway device
Between usually using public network transmission data, in technical solution of the present invention, can still be adopted between video server and gateway device
With existing transmission mode, bandwidth, flow or the load capacity of server need not be increased, by existing public-network bandwidth and will not be taken
Limitation, it is as wireless in present family, being commonly installed one due to being usually LAN between gateway device and client
Router is come the LAN that founds a family, and for mobile phone in family and online computing, bandwidth speed is usually enough to user in LAN and goes
It uses, free and flow is unrestricted, therefore gateway device can export high-resolution video data to client, by client
End plays or transmission, to provide better video playing experience to the user.
Compared with prior art, in technical solution of the present invention, by the way that gateway device is arranged, same video matter can kept
Under the premise of amount, reduce video server exit bandwidth and video code flow public network, that is, the Internet transmission bandwidth and stream
Amount is effectively reduced bandwidth and the flow consumption of video code flow, reduces server stress, reduce the operation of video content provider
Cost, while the network flow cost of ordinary user is reduced, the speed for improving user video buffering and playing is reduced due to bandwidth
Interim card problem caused by problem improves video image quality, brings the better usage experience of user.
Further, the gateway device includes data communication module, Video decoding module, super-resolution rebuilding module, institute
Data communication module is stated for into row data communication, the Video decoding module to be for will be from video server and client
The video that video server receives is decoded, and the super-resolution rebuilding module is used to carry out decoded video data
Super-resolution rebuilding.
The video data that video server is received by data communication module, by Video decoding module to video data
It is decoded, rebuilds the image data of video, by super-resolution rebuilding module, super-resolution rebuilding is carried out to image, finally
Generate the video data of super-resolution.
Further, the gateway device further includes video encoding module, and the video encoding module is used for super-resolution
Video data after reconstruction is encoded.
The video data after reconstruction is encoded by video encoding module, convenient for being transferred to client.
Further, the super-resolution rebuilding module includes convolutional neural networks module, the convolutional neural networks module
For carrying out super-resolution rebuilding to video data according to training result.
Using the powerful learning ability of convolutional neural networks module, pass through the training certain to convolutional neural networks module
Afterwards, it is high that traditional super-resolution algorithms computation complexity can be solved, cannot effectively extract feature, big wait of practical application cost lacks
Point.
Further, the gateway device is the router for being equipped with the corresponding function module.
Router is daily most common gateway device, and super-resolution rebuilding is carried out to video data by router,
The service of video super-resolution can be provided for the terminal device in entire LAN.
The present invention also provides a kind of video transmission method based on super-resolution rebuilding technology, this method includes following step
Suddenly：
Video reception step, gateway device receive the video data that video server is sent；
Super-resolution rebuilding step, gateway device carry out super-resolution rebuilding to video data；
Video forwarding step, gateway device is by the transmission of video after super-resolution rebuilding to client.
Further, the super-resolution rebuilding step includes the following steps：
S1：Obtain the data of a frame video and decoding in video data；
S2：Simultaneously reconstruction image is decoded to the macro block of the frame video image；
S3：Super-resolution rebuilding is carried out to the image of the macro block after the reconstruction.
Explanation of nouns：Macro block is the basic unit of Video coding processing, and usual macroblock size is 16x16 pixel.
Macro block is decoded in the step, reconstruction image and super-resolution rebuilding step, using macro block as minimum list
Member, it is respectively independent, the diffusion and transmission of error code can be limited.
Further, the S2 specifically includes following steps：
S2-1：Judge the type of previous frame, if the type of present frame is I frames, macro block all in present frame is carried out
Decoding and reconstruction image completely；If the type of present frame is B frames or P frames, S2-2 is executed to each macro block in present frame；
S2-2 includes the following steps：
S2-2-1：Judge the type of current macro, if current macro type is intra prediction macro block, executes S2-2-
2；If current macro type is SKIP or Direct macro blocks, S2-2-3 is executed；If current macro type is inter-prediction
Macro block then executes S2-2-4；
S2-2-2：Simultaneously reconstruction image directly is decoded to current macro according to the data content of present frame；
S2-2-3：Directly using the reconstruction image of corresponding predicted position in the corresponding prediction frame of the macro block as the macro block
Reconstruction image；
S2-2-4：Decoding obtains motion vector and residual error coefficient, judges whether residual error coefficient is less than given threshold value, if so,
Then sample motion vector, directly using the reconstruction image of respective motion vectors position in the corresponding prediction frame of the macro block as the macro block
Reconstruction image；If it is not, then decoding to obtain reconstruction image according to standard decoding process.
Explanation of nouns：I frames are also known as intracoded frame, are a kind of independent frames of included all information, without reference to other figures
As can independently be decoded；
P frames are also known as inter prediction encoding frame, need to refer to the frame of front and could decode；
B frames are also known as bi-directional predictive coding frame, and needing to refer to the front and subsequent frame could decode.
Macro block is distinguished according to the type of frame first, for I frames, since it can independently be decoded, thus in it
All macro blocks directly can completely be decoded and be realized the reconstruction of image；For P frames and B frames, then need for macro block
Specific type of prediction is encoded,, can be directly according to present frame as the macro block in I frames for intra prediction macro block
Data content directly current macro is decoded and reconstruction image；For the macro block of SKIP types or Direct patterns, directly
The image for connecing corresponding position in its associated prediction frame of copy obtains it first as reconstruction image for inter-prediction macro block
Residual error coefficient, if residual error coefficient is too small, the image conduct for copying respective motion vectors position in prediction frame associated with it should
The reconstruction image of macro block uses the algorithm of standard, that is, obtains prediction frame respective motion vectors if residual error coefficient is more than preset value
The data of position are simultaneously worth final reconstructed image data plus residual error coefficient.
Further, the super-resolution rebuilding step further includes：
S4：Macro block data after super-resolution rebuilding is encoded.
Further, the type of prediction and quantization parameter of macro block can be preserved when S2 is decoded macro block, the S4 is specifically wrapped
Include following steps：
S4-1：The type of previous frame is judged, if the type of present frame is I frames, using the S2 macro blocks decoded
Type of prediction of the type of prediction as coding, the quantization parameter to decode the obtained macro block subtract preset value as the amount of coding
Change parameter to encode the macro block；If the type of present frame is B frames or P frames, each macro block in present frame is executed
S4-2；
S4-2 includes the following steps：
S4-2-1：Judge the type of current macro, if current macro type is intra prediction macro block, executes S4-2-
2；If current macro type is SKIP or Direct macro blocks, S4-2-3 is executed；If current macro type is inter-prediction
Macro block then executes S4-2-4；
S4-2-2：Type of prediction using the type of prediction of the S2 macro blocks decoded as coding, is obtained with decoding
The macro block quantization parameter subtract preset value as coding quantization parameter the macro block is encoded；
S4-2-3：The macro block is encoded using SKIP or Direct patterns；
S4-2-4：If judging whether residual error coefficient is less than given threshold value, if so, using motion vector after sampling and pre-
The type of prediction that type is surveyed as coding encodes the macro block；If it is not, using the prediction class of the S2 macro blocks decoded
Type of prediction of the motion vector as coding after type and sampling, the quantization parameter to decode the obtained macro block subtract preset value
Quantization parameter as coding encodes the macro block.
Further, in the S3 steps super-resolution rebuilding using convolutional neural networks super-resolution rebuilding technology.
Super-resolution problem is handled using convolutional neural networks, strong with learning ability, computation complexity is low, can
The advantages that effectively extraction feature, practical application cost is small, it is suitble to be applied in gateway device.
Description of the drawings
Fig. 1 is that the present invention is based on the logic diagrams in the Video transmission system embodiment of super-resolution rebuilding technology；
Fig. 2 is that the present invention is based on convolutional neural networks modules in the Video transmission system embodiment of super-resolution rebuilding technology
Logic diagram；
Fig. 3 is that the present invention is based on super-resolution rebuilding steps in the Video transmission system embodiment of super-resolution rebuilding technology
Flow chart.
Specific implementation mode
Below by specific implementation mode, the present invention is described in further detail：
Reference sign：Video server U1, gateway device U2, LAN U3, client U4.
As shown in Figure 1, Video transmission system of the present embodiment based on super-resolution rebuilding technology includes gateway device U2, net
It closes the entire transmission process needs of equipment U2 to work together with video server U1, client U4, the client U4 and webmaster of user
Equipment forms LAN U3, and client U4 accesses internet by gateway device U2, with video server U1 network connections, client
End U4 can be carried out data transmission by gateway device U2 with to video server U1.
Gateway device U2 includes data communication module, Video decoding module, super-resolution rebuilding module and Video coding mould
Block, data communication module are used for video server U1 and client U4 into row data communication, and Video decoding module is used for will
The video received from video server U1 is decoded to obtain the data of each frame, and super-resolution rebuilding module includes frame type
Judgment module, macro block (mb) type judgment module, macroblock image rebuild module and convolutional neural networks module, frame type judging module
For judging the type of each frame for P frames, I frames or B frames, macro block (mb) type judgment module is for judging each macro block in each frame
Type, macroblock image rebuild module according to the type of frame and the type of macro block to the corresponding image of each macro block in each frame
Reconstruction recovery is carried out, convolutional neural networks module is used to carry out super-resolution rebuilding to the image of macro block according to training result, depending on
For encoding the video data after super-resolution rebuilding, the video after coding is passed by communication module frequency coding module again
It is defeated by client U4.
In the present embodiment, gateway device U2 is the router for being equipped with respective service software systems, can receive client
The video request of U4 is held, and video request is forwarded to video server U1, video server U1 is by video data transmission to routing
Device, router are decoded it after receiving video data, obtain the data of the macro block of each frame, and establish corresponding
Macroblock image, then super-resolution rebuilding is carried out to the image of macro block by convolutional neural networks module, generate the macro of super-resolution
Then block number evidence encodes macro block by video encoding module, generate corresponding video stream data, is forwarded to client U4,
It plays or propagates for client U4.Gateway device U2 can also be other equipment, such as interchanger, relay.
In the present embodiment, such as Fig. 2, convolutional neural networks module includes 7 layers of convolutional layer, first layer include convolutional layer CONV and
Activation primitive layer, wherein convolution step-length are decoded macroblock size, ensure not overlapping block operation in this way, it is multiple to greatly reduce calculating
Miscellaneous degree, convolution kernel size are 5, characteristic 56；The second layer includes convolutional layer CONV and activation primitive layer, parameter to layer 6
It is as shown in Figure 2 respectively；Layer 7 i.e. last layer is convolutional layer, convolution kernel 9, characteristic 1.Convolutional neural networks algorithm
Existing technology, that as disclosed in the Chinese patent application No. is 201710489400.7 of detail are used with process
Sample, details are not described herein.
In the present embodiment, client U4 can be the playback terminal of user, computer as shown in Figure 1, mobile phone, tablet,
Laptop etc. can also be the gateway device U2 or the network equipment of other next stage, in router, interchanger, network
After device etc..
For the clearer operation principle for illustrating the present invention, the present embodiment also discloses a kind of based on super-resolution rebuilding
The video transmission method of technology, this approach includes the following steps：
Video reception step, gateway device U2 receive the video data that video server U1 is sent；
Super-resolution rebuilding step, gateway device U2 carry out super-resolution rebuilding to video data；
Video forwarding step, gateway device U2 give the transmission of video after super-resolution rebuilding to client U4.
Wherein, super-resolution rebuilding step is as shown in figure 3, include the following steps：
S1：Obtain the data of a frame video and decoding in video data；
S2：Simultaneously reconstruction image is decoded to the macro block of the frame video image；
S3：The image of the macro block after the reconstruction is surpassed using the super-resolution rebuilding technology based on convolutional neural networks
Resolution reconstruction；
S4：Macro block data after super-resolution rebuilding is encoded.
S2 specifically includes following steps：
S2-1：The type of prediction and quantization parameter for obtaining macro block, judge the type of previous frame, if the type of present frame is I
Frame then carries out decoding and reconstruction image completely to macro block all in present frame；If the type of present frame is B frames or P frames,
S2-2 is executed to each macro block in present frame；
S2-2 includes the following steps：
S2-2-1：Judge the type of current macro, if current macro type is intra prediction macro block, executes S2-2-
2；If current macro type is SKIP or Direct macro blocks, S2-2-3 is executed；If current macro type is inter-prediction
Macro block then executes S2-2-4；
S2-2-2：Simultaneously reconstruction image directly is decoded to current macro according to the data content of present frame；
S2-2-3：Directly using the reconstruction image of corresponding predicted position in the corresponding prediction frame of the macro block as the macro block
Reconstruction image；
S2-2-4：Decoding obtains motion vector and residual error coefficient, if judging whether residual error coefficient is less than given threshold value, if
Then to up-sample motion vector, directly using the macro block it is corresponding prediction frame in respective motion vectors position reconstruction image as
The reconstruction image of the macro block；If it is not, then up-sampling motion vector, and decode to obtain reconstruction image according to standard decoding process.
S4 specifically includes following steps：
S4-1：The type of previous frame is judged, if the type of present frame is I frames, using the S2 macro blocks decoded
Type of prediction of the type of prediction as coding, the quantization parameter to decode the obtained macro block subtract preset value (in the present embodiment
Preset value is that the quantization parameter 1) as coding encodes the macro block；If the type of present frame is B frames or P frames, right
Each macro block executes S4-2 in present frame；
S4-2 includes the following steps：
S4-2-1：Judge the type of current macro, if current macro type is intra prediction macro block, executes S4-2-
2；If current macro type is SKIP or Direct macro blocks, S4-2-3 is executed；If current macro type is inter-prediction
Macro block then executes S4-2-4；
S4-2-2：Type of prediction using the type of prediction of the S2 macro blocks decoded as coding, is obtained with decoding
The macro block quantization parameter subtract preset value as coding quantization parameter the macro block is encoded；
S4-2-3：The macro block is encoded using SKIP or Direct patterns；
S4-2-4：If judge residual error coefficient whether be less than given threshold value, if so, using up-sampling after motion vector and
Type of prediction encodes the macro block as the type of prediction of coding；If it is not, using the prediction of the S2 macro blocks decoded
Type of prediction of the motion vector as coding after type and up-sampling, the quantization parameter to decode the obtained macro block subtracts pre-
If value encodes the macro block as the quantization parameter of coding.
The above are merely the embodiment of the present invention, the common sense such as well known concrete structure and characteristic are not made excessively herein in scheme
Description, all common of technical field that the present invention belongs to before one skilled in the art know the applying date or priority date
Technological know-how can know the prior art all in the field, and with using routine experiment means before the date
Ability, one skilled in the art can improve in conjunction with self-ability under the enlightenment that the application provides and implement we
Case, some typical known features or known method should not become the barrier that one skilled in the art implement the application
Hinder.It should be pointed out that for those skilled in the art, without departing from the structure of the invention, if can also make
Dry modification and improvement, these should also be considered as protection scope of the present invention, these all do not interfere with the effect that the present invention is implemented and
Patent practicability.The scope of protection required by this application should be based on the content of the claims, the specific reality in specification
Apply the content that the records such as mode can be used for explaining claim.
Claims (10)
1. the Video transmission system based on super-resolution rebuilding technology, it is characterised in that：Including：
Gateway device, the gateway device is for connecting client and video server, and the gateway device is for receiving video
The video received is simultaneously carried out super-resolution rebuilding by the video of server transport, and the gateway device is additionally operable to super-resolution
Transmission of video after reconstruction is to client.
2. the Video transmission system according to claim 1 based on super-resolution rebuilding technology, it is characterised in that：The net
It includes data communication module, Video decoding module, super-resolution rebuilding module to close equipment, the data communication module be used for regarding
The video that frequency server and client are used to receive from video server into row data communication, the Video decoding module
It is decoded, the super-resolution rebuilding module is used to carry out super-resolution rebuilding to decoded video data.
3. the Video transmission system according to claim 2 based on super-resolution rebuilding technology, it is characterised in that：The net
It further includes video encoding module to close equipment, and the video encoding module is for compiling the video data after super-resolution rebuilding
Code.
4. the Video transmission system according to claim 3 based on super-resolution rebuilding technology, it is characterised in that：It is described super
Resolution reconstruction module includes convolutional neural networks module, and the convolutional neural networks module is used for according to training result to video
Data carry out super-resolution rebuilding.
5. a kind of video transmission method based on super-resolution rebuilding technology, this approach includes the following steps：
Video reception step, gateway device receive the video data that video server is sent；
Super-resolution rebuilding step, gateway device carry out super-resolution rebuilding to video data；
Video forwarding step, gateway device is by the transmission of video after super-resolution rebuilding to client.
6. the video transmission method according to claim 5 based on super-resolution rebuilding technology, it is characterised in that：It is described super
Resolution reconstruction process includes the following steps：
S1：Obtain the data of a frame video and decoding in video data；
S2：Simultaneously reconstruction image is decoded to the macro block of the frame video image；
S3：Super-resolution rebuilding is carried out to the image of the macro block after the reconstruction.
7. the video transmission method according to claim 6 based on super-resolution rebuilding technology, it is characterised in that：The S2
Specifically include following steps：
S2-1：Judge the type of previous frame, if the type of present frame is I frames, macro block all in present frame is carried out complete
Decode simultaneously reconstruction image；If the type of present frame is B frames or P frames, S2-2 is executed to each macro block in present frame；
S2-2 includes the following steps：
S2-2-1：Judge the type of current macro, if current macro type is intra prediction macro block, executes S2-2-2；Such as
Fruit current macro type is SKIP or Direct macro blocks, then executes S2-2-3；If current macro type is inter-prediction macro block,
Then execute S2-2-4；
S2-2-2：Simultaneously reconstruction image directly is decoded to current macro according to the data content of present frame；
S2-2-3：Directly using the reconstruction image of corresponding predicted position in the corresponding prediction frame of the macro block as the reconstruction of the macro block
Image；
S2-2-4：Decoding obtains motion vector and residual error coefficient, judges whether residual error coefficient is less than given threshold value, if so, adopting
Sample motion vector, directly using the reconstruction image of respective motion vectors position in the corresponding prediction frame of the macro block as the weight of the macro block
Build image；If it is not, then decoding to obtain reconstruction image according to standard decoding process.
8. the video transmission method according to claim 6 based on super-resolution rebuilding technology, it is characterised in that：It is described super
Resolution reconstruction process further includes：
S4：Macro block data after super-resolution rebuilding is encoded.
9. the video transmission method according to claim 8 based on super-resolution rebuilding technology, it is characterised in that：S2 is to macro
Block can preserve the type of prediction and quantization parameter of macro block when being decoded, the S4 specifically includes following steps：
S4-1：The type of previous frame is judged, if the type of present frame is I frames, using the prediction of the S2 macro blocks decoded
Type of prediction of the type as coding, the quantization parameter to decode the obtained macro block subtract preset value as the quantization ginseng of coding
It is several that the macro block is encoded；If the type of present frame is B frames or P frames, S4-2 is executed to each macro block in present frame；
S4-2 includes the following steps：
S4-2-1：Judge the type of current macro, if current macro type is intra prediction macro block, executes S4-2-2；Such as
Fruit current macro type is SKIP or Direct macro blocks, then executes S4-2-3；If current macro type is inter-prediction macro block,
Then execute S4-2-4；
S4-2-2：Type of prediction using the type of prediction of the S2 macro blocks decoded as coding is somebody's turn to do with what decoding obtained
The quantization parameter of macro block subtracts preset value and is encoded to the macro block as the quantization parameter of coding；
S4-2-3：The macro block is encoded using SKIP or Direct patterns；
S4-2-4：If judging whether residual error coefficient is less than given threshold value, if so, using motion vector and prediction class after sampling
Type encodes the macro block as the type of prediction of coding；If it is not, using the S2 macro blocks decoded type of prediction and
The type of prediction of motion vector after sampling as coding, the quantization parameter to decode the obtained macro block subtract preset value as
The quantization parameter of coding encodes the macro block.
10. the video transmission method according to claim 6 based on super-resolution rebuilding technology, it is characterised in that：It is described
Super-resolution rebuilding is using convolutional neural networks super-resolution rebuilding technology in S3 steps.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810195620.3A CN108495128A (en) | 2018-03-09 | 2018-03-09 | Video transmission system based on super-resolution rebuilding technology and transmission method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810195620.3A CN108495128A (en) | 2018-03-09 | 2018-03-09 | Video transmission system based on super-resolution rebuilding technology and transmission method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108495128A true CN108495128A (en) | 2018-09-04 |
Family
ID=63338640
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810195620.3A Withdrawn CN108495128A (en) | 2018-03-09 | 2018-03-09 | Video transmission system based on super-resolution rebuilding technology and transmission method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108495128A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109361919A (en) * | 2018-10-09 | 2019-02-19 | 四川大学 | A kind of image coding efficiency method for improving combined super-resolution and remove pinch effect |
| CN111147893A (en) * | 2018-11-02 | 2020-05-12 | 华为技术有限公司 | Video self-adaption method, related equipment and storage medium |
| CN111405296A (en) * | 2020-03-11 | 2020-07-10 | 北京邮电大学 | Video data transmission method, video data processing device and electronic equipment |
| CN111580765A (en) * | 2020-04-27 | 2020-08-25 | Oppo广东移动通信有限公司 | Screen projection method, screen projection device, storage medium, screen projection equipment and screen projection equipment |
| CN111681167A (en) * | 2020-06-03 | 2020-09-18 | 腾讯科技（深圳）有限公司 | Image quality adjusting method and device, storage medium and electronic equipment |
| CN112330542A (en) * | 2020-11-18 | 2021-02-05 | 重庆邮电大学 | Image reconstruction system and method based on CRCSAN network |
| CN112449140A (en) * | 2019-08-29 | 2021-03-05 | 华为技术有限公司 | Video super-resolution processing method and device |
| CN113259766A (en) * | 2021-05-24 | 2021-08-13 | 广东东软学院 | Video file repair method |
| CN114244819A (en) * | 2022-01-28 | 2022-03-25 | 西安万像电子科技有限公司 | Method for decoding video stream through router, storage medium and electronic equipment |
| US11410275B2 (en) * | 2019-09-23 | 2022-08-09 | Tencent America LLC | Video coding for machine (VCM) based system and method for video super resolution (SR) |
| CN117061824A (en) * | 2023-10-12 | 2023-11-14 | 深圳云天畅想信息科技有限公司 | Streaming media video frame supplementing method and device and computer equipment |
-
2018
- 2018-03-09 CN CN201810195620.3A patent/CN108495128A/en not_active Withdrawn
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109361919A (en) * | 2018-10-09 | 2019-02-19 | 四川大学 | A kind of image coding efficiency method for improving combined super-resolution and remove pinch effect |
| CN111147893A (en) * | 2018-11-02 | 2020-05-12 | 华为技术有限公司 | Video self-adaption method, related equipment and storage medium |
| US11509860B2 (en) | 2018-11-02 | 2022-11-22 | Huawei Technologies Co., Ltd. | Video adaptation method, related device, and storage medium |
| CN111147893B (en) * | 2018-11-02 | 2021-10-22 | 华为技术有限公司 | Video self-adaption method, related equipment and storage medium |
| CN112449140B (en) * | 2019-08-29 | 2021-09-14 | 华为技术有限公司 | Video super-resolution processing method and device |
| US11922599B2 (en) | 2019-08-29 | 2024-03-05 | Huawei Technologies Co., Ltd. | Video super-resolution processing method and apparatus |
| CN112449140A (en) * | 2019-08-29 | 2021-03-05 | 华为技术有限公司 | Video super-resolution processing method and device |
| US11410275B2 (en) * | 2019-09-23 | 2022-08-09 | Tencent America LLC | Video coding for machine (VCM) based system and method for video super resolution (SR) |
| CN111405296A (en) * | 2020-03-11 | 2020-07-10 | 北京邮电大学 | Video data transmission method, video data processing device and electronic equipment |
| CN111580765A (en) * | 2020-04-27 | 2020-08-25 | Oppo广东移动通信有限公司 | Screen projection method, screen projection device, storage medium, screen projection equipment and screen projection equipment |
| CN111580765B (en) * | 2020-04-27 | 2024-01-12 | Oppo广东移动通信有限公司 | Screen projection method, screen projection device, storage medium, screen projection equipment and screen projection equipment |
| CN111681167A (en) * | 2020-06-03 | 2020-09-18 | 腾讯科技（深圳）有限公司 | Image quality adjusting method and device, storage medium and electronic equipment |
| CN112330542A (en) * | 2020-11-18 | 2021-02-05 | 重庆邮电大学 | Image reconstruction system and method based on CRCSAN network |
| CN112330542B (en) * | 2020-11-18 | 2022-05-03 | 重庆邮电大学 | Image reconstruction system and method based on CRCSAN network |
| CN113259766A (en) * | 2021-05-24 | 2021-08-13 | 广东东软学院 | Video file repair method |
| CN114244819A (en) * | 2022-01-28 | 2022-03-25 | 西安万像电子科技有限公司 | Method for decoding video stream through router, storage medium and electronic equipment |
| CN117061824A (en) * | 2023-10-12 | 2023-11-14 | 深圳云天畅想信息科技有限公司 | Streaming media video frame supplementing method and device and computer equipment |
| CN117061824B (en) * | 2023-10-12 | 2024-01-26 | 深圳云天畅想信息科技有限公司 | Streaming media video frame supplementing method and device and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108495128A (en) | Video transmission system based on super-resolution rebuilding technology and transmission method | |
| US12047606B2 (en) | Transform kernel selection and entropy coding | |
| US20190020888A1 (en) | Compound intra prediction for video coding | |
| US11102477B2 (en) | DC coefficient sign coding scheme | |
| US11558631B2 (en) | Super-resolution loop restoration | |
| CN107534669A (en) | Single stream transmission method for multi-user's video conference | |
| US11647223B2 (en) | Dynamic motion vector referencing for video coding | |
| US20210021859A1 (en) | Same frame motion estimation and compensation | |
| AU2007231799A1 (en) | High-performance video transcoding method | |
| US9369732B2 (en) | Lossless intra-prediction video coding | |
| US10225573B1 (en) | Video coding using parameterized motion models | |
| US10382767B2 (en) | Video coding using frame rotation | |
| Zhu et al. | Compression-dependent transform-domain downward conversion for block-based image coding | |
| US20220094950A1 (en) | Inter-Prediction Mode-Dependent Transforms For Video Coding | |
| US10820014B2 (en) | Compound motion-compensated prediction | |
| US10491923B2 (en) | Directional deblocking filter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20180904 |