CN105657541A - Frame processing method and device - Google Patents

Frame processing method and device Download PDF

Info

Publication number
CN105657541A
CN105657541A CN201511024885.XA CN201511024885A CN105657541A CN 105657541 A CN105657541 A CN 105657541A CN 201511024885 A CN201511024885 A CN 201511024885A CN 105657541 A CN105657541 A CN 105657541A
Authority
CN
China
Prior art keywords
frame sequence
reconstruct
sequence
residual
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201511024885.XA
Other languages
Chinese (zh)
Inventor
陈琼
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN201511024885.XA priority Critical patent/CN105657541A/en
Publication of CN105657541A publication Critical patent/CN105657541A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440254—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering signal-to-noise parameters, e.g. requantization
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The embodiment of the invention discloses a frame processing method and device, relating to the technical field of images. By means of the frame processing method and device, the quality of an image output by a terminal can be improved. The specific scheme comprises the following steps of: separating a frame sequence into a main frame sequence and an affiliated frame sequence; obtaining an insertion frame sequence of the main frame sequence; subtracting the affiliated frame sequence and the insertion frame sequence so as to obtain a first residual frame sequence; combining a main code stream, which is formed by coding the main frame sequence, and an affiliated code stream, which is formed by coding the first residual frame sequence, so as to form a transmission code stream; and sending the transmission code stream to the terminal, such that the terminal recovers the frame sequence according to the transmission code stream. The frame processing method and device disclosed by the invention are used for frame processing.

Description

A kind of frame processing method and device
Technical field
The present invention relates to image technique field, particularly relate to a kind of frame processing method and device.
Background technology
At present, the photodiode that frame per second is 50Hz (hertz) or 60Hz is (English: lightemittingdiode, abbreviation: LED) display screen, liquid-crystal display are (English: liquidcrystaldisplay, abbreviation: LCD) more and more popular, the video (being made up of frame sequence) that frame per second is 50Hz or 60Hz is also more and more universal. But, due to the restriction of the network bandwidth, there is provided the Video service device of film source when transmission of video, usually only it is be transferred to terminal after code stream (frame per second of this yard of stream is 25Hz or 30Hz) by the odd-numbered frame sequence in the frame sequence of this video or even frame sequence encoding, so can reduce the network bandwidth taken in video transmitting procedure.
Due to terminal reception is the code stream based on odd-numbered frame sequence or even frame sequence encoding, cause the frame per second of frame per second lower than the screen of terminal of this yard of stream, therefore, in order to the frame per second that improves code stream with better and the frame per second of the screen of terminal mate, after terminal receives this yard of stream and this yard of stream decodes the even frame sequence (main frame sequence hereinafter referred to as reconstruct) of the odd-numbered frame sequence obtaining reconstruct or reconstruct, motion estimation motion is adopted to compensate (English: motionestimationmotioncompensation, abbreviation: MEMC) technology, frame sequence carries out changing in frame per second (English: framerateupconversion, abbreviation: FRUC), concrete, first (English: motionestimation by estimation, abbreviation: ME) obtain in the frame sequence of reconstruct that often motion vector between adjacent two frames is (English: motionvector, abbreviation: MV), then adopt motion compensated interpolation (English: motioncompensatedinterpolation, abbreviation: MCI) algorithm, according to MV, these adjacent two frames are carried out interpolation arithmetic, with the slotting frame obtained between these adjacent two frames, namely this slotting frame can be regarded as the even frame between adjacent two odd-numbered frame, or the odd-numbered frame between adjacent two even frame.
But, in the above-mentioned methods, slotting frame between adjacent two reconstructed frame that terminal obtains and even frame in former frame sequence or odd-numbered frame, the situation that error is excessive may occur, thus when the slotting frame sequence obtained is merged output by terminal with reconstructed frame sequence, may there is halo effect or occur damaged in the image exported, causes the quality of the image that terminal presents lower.
Summary of the invention
In view of this, the present invention provides a kind of frame processing method and device, it is possible to improve the quality of the image that terminal exports.
For achieving the above object, the present invention adopts following technical scheme:
First aspect, the embodiment of the present invention provides a kind of frame processing method, comprising:
Frame sequence is separated into main frame sequence and subsidiary frame sequence;
Obtain the slotting frame sequence of described main frame sequence;
Subtract each other to obtain the first residual frame sequence by described subsidiary frame sequence and described slotting frame sequence;
Merge to form transmission code stream by the main bit stream formed after described main frame sequence being encoded and the attached code stream that formed after described first residual frame sequence being encoded;
Described transmission code stream is sent to terminal, so that described terminal recovers described frame sequence according to described transmission code stream.
In the frame processing method that the embodiment of the present invention provides, Video service device can subsidiary frame sequence in getting frame sequence and the first residual frame sequence between slotting frame sequence, the error of this first residual frame sequence for reducing between subsidiary frame sequence and slotting frame sequence, to improve the quality of the image that terminal exports. namely when Video service device by by this first residual frame sequence through coding after be sent to terminal after, so that terminal can according to the subsidiary frame sequence of this first residual frame sequence of reconstruct and the reconstruct of slotting frame retrieval, due to reconstruct subsidiary frame sequence and frame sequence in subsidiary frame sequence between error, much smaller than the error between slotting frame sequence and subsidiary frame sequence, therefore, after the main frame sequence of the subsidiary frame sequence of described reconstruct and described reconstruct is merged the frame sequence output of the reconstruct obtained by terminal, the quality of the image presented, the quality of the image presented after slotting frame sequence being merged the frame sequence output of the reconstruct obtained with the main frame sequence of reconstruct higher than terminal in prior art far away, namely by frame processing method provided by the invention, the picture quality that terminal exports can be improved.
Optionally, in conjunction with above-mentioned first aspect, in the implementation that the first is possible, described subtract each other to obtain the first residual frame sequence by described subsidiary frame sequence and described slotting frame sequence, comprising:
The residual frame sequence that described subsidiary frame sequence and described slotting frame sequence are subtracted each other rear gained is directly as the first residual frame sequence.
Optionally, in conjunction with above-mentioned first aspect, in the 2nd kind of possible implementation of first aspect, described subtract each other to obtain the first residual frame sequence by described subsidiary frame sequence and described slotting frame sequence, comprising:
The residual frame sequence that described subsidiary frame sequence and described slotting frame sequence subtract each other rear gained is carried out pre-treatment, using pretreated described residual frame sequence as the first residual frame sequence.
Optionally, in conjunction with the 2nd kind of possible implementation of above-mentioned first aspect, in the implementation that the third is possible, described residual frame sequence is carried out that pre-treatment comprises in following four kinds of modes one or more:
By the chromatic(ity)component zero setting of each residual frame in described residual frame sequence;
According to default scaling, the resolving power of each residual frame in described residual frame sequence is reduced;
According to default noise figure, the pixel of each residual frame in described residual frame sequence is carried out denoising; And
According to default quantized value, the pixel of each residual frame in described residual frame sequence is quantized.
Optionally, in conjunction with the third possible implementation of above-mentioned first aspect, in the 4th kind of possible implementation, described method also comprises:
Being merged in described transmission code stream by the parameter adopted when described residual frame sequence is carried out pre-treatment and be sent to described terminal, described parameter comprises at least one in described scaling, described noise figure and described quantized value.
Optionally, in conjunction with above-mentioned first aspect, in the 5th kind of possible implementation of first aspect, described subtract each other to obtain the first residual frame sequence by described subsidiary frame sequence and described slotting frame sequence, comprising:
After the sequence belonging to the designated area of described frame sequence in described subsidiary frame sequence and the sequence that belongs to described designated area in described slotting frame sequence being subtracted each other, the residual frame sequence of gained is as the first residual frame sequence.
Optionally, in conjunction with above-mentioned first aspect, in the 6th kind of possible implementation of first aspect, described subtract each other to obtain the first residual frame sequence by described subsidiary frame sequence and described slotting frame sequence, comprising:
After the sequence belonging to the designated area of described frame sequence in described subsidiary frame sequence and the sequence that belongs to described designated area in described slotting frame sequence being subtracted each other, the residual frame sequence of gained carries out pre-treatment, will carry out pretreated described residual frame sequence as the first residual frame sequence.
Optionally, in conjunction with the 5th kind of possible implementation of above-mentioned first aspect or the 6th kind of possible implementation of first aspect, in the 7th kind of possible implementation, described method also comprises:
The information of described designated area is merged in described transmission code stream and is sent to described terminal.
2nd kind of possibility implementation of above-mentioned first aspect is to any one implementation in the 6th kind of possible implementation of above-mentioned first aspect, the bit number of the first residual frame sequence can both be reduced, it is thus possible to reduce Video service device to terminal transmission first residual frame sequence encoded after generate attached code stream time, the network bandwidth that this attached code stream takies.
Optionally, in conjunction with the first possible implementation of above-mentioned first aspect or first aspect to any one implementation in the 7th kind of possible implementation of first aspect, in the 8th kind of possible implementation, the slotting frame sequence of the described main frame sequence of described acquisition comprises:
Carrying out changing in frame per second to obtain described slotting frame sequence and motion vector MV to described main frame sequence by transfer algorithm in frame per second, the described main frame sequence of reconstruct is carried out changing in frame per second by described MV for described terminal;
Described attached code stream also comprises described MV.
Second aspect, the embodiment of the present invention provides a kind of frame processing method, comprising:
Receive transmission code stream, and described transmission code stream is separated into main bit stream and attached code stream;
Described main bit stream and described attached code stream decode to obtain the main frame sequence of reconstruct and the first residual frame sequence of reconstruct respectively;
Obtain the slotting frame sequence of the main frame sequence of described reconstruct;
The subsidiary frame sequence of the first residual frame sequence according to described reconstruct and the reconstruct of described slotting frame retrieval;
The subsidiary frame sequence of the main frame sequence of described reconstruct and described reconstruct is carried out the frame sequence merging to form reconstruct.
In frame processing method provided by the invention, terminal can obtain the first residual frame sequence of reconstruct according to the attached code stream decoding that Video service device sends, and after terminal obtains slotting frame sequence, terminal can further according to the first residual frame sequence of reconstruct and the subsidiary frame sequence of this slotting frame retrieval reconstruct, owing to the first residual frame sequence is that Video service device subtracts each other acquisition according to the subsidiary frame sequence in frame sequence and slotting frame sequence, therefore, reconstruct subsidiary frame sequence and frame sequence in subsidiary frame sequence between error, much smaller than the error between slotting frame sequence and subsidiary frame sequence, therefore, when the quality of the image that terminal presents after the main frame sequence of the subsidiary frame sequence of described reconstruct and described reconstruct is merged the frame sequence output of the reconstruct formed, the quality of the image presented after slotting frame sequence being merged the frame sequence output of the reconstruct formed with the main frame sequence of reconstruct higher than terminal in prior art, namely by frame processing method provided by the invention, the picture quality that terminal exports can be improved.
Optionally, in conjunction with above-mentioned second aspect, in the first possible implementation of second aspect, the subsidiary frame sequence of described the first residual frame sequence according to described reconstruct and the reconstruct of described slotting frame retrieval, comprising:
Described slotting frame sequence is added, with the first residual frame sequence of described reconstruct, the subsidiary frame sequence obtaining described reconstruct.
Optionally, in conjunction with above-mentioned second aspect, in the 2nd kind of possible implementation of second aspect, the subsidiary frame sequence of described the first residual frame sequence according to described reconstruct and the reconstruct of described slotting frame retrieval, comprising:
First residual frame sequence of described reconstruct is carried out pre-treatment;
Described slotting frame sequence is added, with the first residual frame sequence carrying out pretreated described reconstruct, the subsidiary frame sequence obtaining described reconstruct.
Optionally, in conjunction with the 2nd kind of possible implementation of above-mentioned second aspect, in the implementation that the third is possible, described transmission code stream also comprises scaling, noise figure, and at least one in quantized value,
Described the first residual frame sequence to described reconstruct carries out one or more in the following four kinds of modes of pre-treatment:
By the chromatic(ity)component zero setting of each residual frame in the first residual frame sequence of described reconstruct;
According to default scaling, the resolving power of the first residual frame of each reconstruct in the first residual frame sequence of described reconstruct is amplified;
According to default noise figure, the pixel of the first residual frame of each reconstruct in the first residual frame sequence of described reconstruct is carried out anti-denoising; And
According to default quantized value, the pixel of the first residual frame of each reconstruct in the first residual frame sequence of described reconstruct is carried out inverse quantization.
Optionally, in conjunction with above-mentioned second aspect, in the 4th kind of possible implementation of second aspect, described transmission code stream also comprises the information of designated area,
Described described slotting frame sequence is added, with the first residual frame sequence of described reconstruct, the subsidiary frame sequence obtaining described reconstruct, comprising:
Information according to described designated area determines to belong in described slotting frame sequence the sequence of described designated area;
First residual frame sequence of the sequence and described reconstruct that belong to described designated area in described slotting frame sequence is added the subsidiary frame sequence obtaining described reconstruct.
Optionally, in conjunction with above-mentioned second aspect, in the 5th kind of possible implementation of second aspect, described transmission code stream also comprises the information of designated area,
The subsidiary frame sequence of described the first residual frame sequence according to described reconstruct and the reconstruct of described slotting frame retrieval, comprising:
First residual frame sequence of described reconstruct is carried out pre-treatment;
Information according to described designated area determines to belong in described slotting frame sequence the sequence of described designated area;
The sequence belonging to described designated area in described slotting frame sequence is added, with the first residual frame sequence carrying out pretreated described reconstruct, the subsidiary frame sequence obtaining described reconstruct.
Optionally, in conjunction with the first possible implementation of above-mentioned second aspect or second aspect to any one implementation in the 5th kind of possible implementation of second aspect, in the 6th kind of possible implementation, described attached code stream also comprises motion vector MV
The slotting frame sequence of the described main frame sequence to obtaining described reconstruct, comprising:
According to described MV, carry out frame per second being changed obtain described slotting frame sequence to the main frame sequence of described reconstruct by transfer algorithm in frame per second.
The third aspect, the embodiment of the present invention provides a kind of Video service device, comprising:
Separating unit, for being separated into main frame sequence and subsidiary frame sequence by frame sequence;
Acquiring unit, for obtaining the slotting frame sequence of the described main frame sequence of described separating unit separation;
Described acquiring unit, also for subtracting each other to obtain the first residual frame sequence by described subsidiary frame sequence and described slotting frame sequence;
Merge cells, for merging to form transmission code stream by the main bit stream formed after described main frame sequence being encoded and the attached code stream that formed after described first residual frame sequence being encoded;
Send unit, for described transmission code stream is sent to terminal, so that described terminal recovers described frame sequence according to described transmission code stream.
The technique effect of the Video service device that the embodiment of the present invention provides see the technique effect of the Video service device described in the frame processing method that above-mentioned first aspect Video service device performs, can repeat no more herein.
In conjunction with the above-mentioned third aspect, in the first possible implementation of the third aspect,
Described acquiring unit, specifically for described subsidiary frame sequence and described slotting frame sequence are subtracted each other rear gained residual frame sequence directly as the first residual frame sequence.
In conjunction with the above-mentioned third aspect, in the 2nd kind of possible implementation of the third aspect,
Described acquiring unit, specifically for carrying out pre-treatment by the residual frame sequence that described subsidiary frame sequence and described slotting frame sequence subtract each other rear gained, using pretreated described residual frame sequence as the first residual frame sequence.
In conjunction with the 2nd kind of possible implementation of the above-mentioned third aspect, in the implementation that the third is possible,
Described merge cells, also for the parameter adopted when described residual frame sequence is carried out pre-treatment being merged in described transmission code stream.
In conjunction with the above-mentioned third aspect, in the 4th kind of possible implementation of the third aspect,
Described acquiring unit, after subtracting each other specifically for the sequence of the designated area using belonging to described frame sequence in described subsidiary frame sequence and the sequence that belongs to described designated area in described slotting frame sequence, the residual frame sequence of gained is as the first residual frame sequence.
In conjunction with the above-mentioned third aspect, in the 5th kind of possible implementation of the third aspect,
Described acquiring unit, after subtracting each other specifically for the sequence of the designated area by belonging to described frame sequence in described subsidiary frame sequence and the sequence that belongs to described designated area in described slotting frame sequence, the residual frame sequence of gained carries out pre-treatment, will carry out pretreated described residual frame sequence as the first residual frame sequence.
In conjunction with the 4th kind of possible implementation of the above-mentioned third aspect or the 5th kind of possible implementation of the third aspect, in the 6th kind of possible implementation,
Described merge cells, also for the information of described designated area being merged in described transmission code stream.
In conjunction with the first possible implementation of the above-mentioned third aspect or the third aspect to any one implementation in the 6th kind of possible implementation of the third aspect, in the 7th kind of possible implementation,
Described acquiring unit, specifically for carrying out changing in frame per second to obtain described slotting frame sequence and motion vector MV to described main frame sequence by transfer algorithm in frame per second, the described main frame sequence of reconstruct is carried out changing in frame per second by described MV for described terminal;
Described attached code stream also comprises described MV.
Fourth aspect, the embodiment of the present invention provides a kind of terminal, comprising:
Receive unit, for receiving transmission code stream, and described transmission code stream is separated into main bit stream and attached code stream;
Decoding unit, for decoding to obtain the main frame sequence of reconstruct and the first residual frame sequence of reconstruct respectively to the described main bit stream and described attached code stream that receive unit separation;
Acquiring unit, for obtaining the slotting frame sequence of the main frame sequence of the described reconstruct that decoding unit obtains;
Described acquiring unit, also for the subsidiary frame sequence of the first residual frame sequence according to described reconstruct and the reconstruct of described slotting frame retrieval;
Merge cells, the subsidiary frame sequence for the described reconstruct main frame sequence of the described reconstruct of decoding unit acquisition and described acquiring unit obtained carries out merging the frame sequence forming reconstruct.
The technique effect of the terminal that the embodiment of the present invention provides see the technique effect of the terminal described in the frame processing method that above-mentioned second aspect terminal performs, can repeat no more herein.
Optionally, in conjunction with above-mentioned fourth aspect, in the first possible implementation of fourth aspect,
Described acquiring unit, specifically for being added, with the first residual frame sequence of described reconstruct, the subsidiary frame sequence obtaining described reconstruct by described slotting frame sequence.
Optionally, in conjunction with above-mentioned fourth aspect, in the 2nd kind of possible implementation of fourth aspect,
Described acquiring unit, specifically for carrying out pre-treatment to the first residual frame sequence of described reconstruct;
Described acquiring unit, also for described slotting frame sequence is added the subsidiary frame sequence obtaining described reconstruct with the first residual frame sequence carrying out pretreated described reconstruct.
Optionally, in conjunction with above-mentioned fourth aspect, in the third possible implementation of fourth aspect, the described transmission code stream that described reception unit receives also comprises the information of designated area,
Described acquiring unit, specifically for determining to belong in described slotting frame sequence the sequence of described designated area according to the information of described designated area;
Described acquiring unit, also for the first residual frame sequence of the sequence and described reconstruct that belong to described designated area in described slotting frame sequence is added the subsidiary frame sequence obtaining described reconstruct.
Optionally, in conjunction with above-mentioned fourth aspect, in the 4th kind of possible implementation of fourth aspect, the described transmission code stream that described reception unit receives also comprises the information of designated area,
Described acquiring unit, specifically for carrying out pre-treatment to the first residual frame sequence of described reconstruct;
Described acquiring unit, also for determining to belong in described slotting frame sequence the sequence of described designated area according to the information of described designated area;
Described acquiring unit, also for the sequence belonging to described designated area in described slotting frame sequence is added the subsidiary frame sequence obtaining described reconstruct with the first residual frame sequence carrying out pretreated described reconstruct.
Optionally, in conjunction with the first possible implementation of above-mentioned fourth aspect or fourth aspect to the 4th kind of possible implementation of fourth aspect, in the 5th kind of possible implementation of fourth aspect, the described attached code stream of described reception unit separation also comprises motion vector MV
Described acquiring unit, specifically for according to described MV, carrying out changing in frame per second to obtain described slotting frame sequence to the main frame sequence of described reconstruct by transfer algorithm in frame per second.
5th aspect, the embodiment of the present invention provides a kind of Video service device, comprising: treater, storer, system bus and communication interface;
Described storer is for storing computer executed instructions, described treater is connected by described system bus with described storer, when described Video service device runs, described treater performs the described computer executed instructions that described storer stores, to make described Video service device perform the method for converting video frame rate as described in any one optional mode of above-mentioned first aspect or first aspect.
6th aspect, the embodiment of the present invention provides a kind of terminal, comprising: treater, storer, system bus and communication interface;
Described storer is for storing computer executed instructions, described treater is connected by described system bus with described storer, when described terminal operating, described treater performs the described computer executed instructions that described storer stores, to make described terminal perform the method for converting video frame rate as described in any one optional mode of above-mentioned second aspect or second aspect.
The Video service device that the embodiment of the present invention provides and terminal, by adopting above-mentioned frame processing method, Video service device can obtain slotting frame sequence, and subtract each other the first residual frame sequence obtaining between subsidiary frame sequence and slotting frame sequence by subsidiary frame sequence and slotting frame sequence, and by this first residual frame sequence after coding, be merged in video transmission code stream and be sent to terminal. and then, after terminal gets the first residual frame sequence of reconstruct of this first residual frame sequence, the slotting frame sequence that can obtain according to the first residual frame sequence of reconstruct and terminal, obtain the subsidiary frame sequence of reconstruct, due to reconstruct subsidiary frame sequence and frame sequence in subsidiary frame sequence between error, the error being less than between slotting frame sequence and subsidiary frame sequence, therefore, the quality of the image that the frame sequence of the reconstruct that terminal obtains presents after exporting, higher than the quality of the image presented after the frame sequence output of the reconstruct in prior art, slotting frame sequence merged and formed with the main frame sequence of reconstruct, therefore the frame processing method that the embodiment of the present invention provides is adopted, the quality of the image that terminal exports can be improved.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, it is briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, it is also possible to obtain other accompanying drawing according to these accompanying drawings.
The framework figure of a kind of Video transmission system end to end that Fig. 1 provides for prior art;
The algorithm schematic diagram of a kind of FRUC that Fig. 2 (a) provides for prior art;
The schematic diagram of the slotting frame between a kind of N frame that Fig. 2 (b) provides for prior art and N+1 frame;
The interaction figure of a kind of frame processing method that Fig. 3 provides for the embodiment of the present invention;
The structural representation of a kind of Video service device that Fig. 4 provides for the embodiment of the present invention;
The structural representation of a kind of terminal that Fig. 5 provides for the embodiment of the present invention;
The hardware architecture diagram of a kind of Video service device that Fig. 6 provides for the embodiment of the present invention;
The hardware architecture diagram of a kind of terminal that Fig. 7 provides for the embodiment of the present invention;
The schematic diagram of a kind of Vx1 interface sequence that Fig. 8 (a) provides for the embodiment of the present invention;
The schematic diagram of another kind of Vx1 interface sequence that Fig. 8 (b) provides for the embodiment of the present invention;
The schematic diagram of another Vx1 interface sequence that Fig. 8 (c) provides for the embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is only the present invention's part embodiment, instead of whole embodiments.
As shown in Figure 1, it is the framework figure of a kind of Video transmission system end to end, comprises Video service device and terminal that film source is provided.In the prior art; video processing device by transferring to terminal by the odd-numbered frame sequence in the frame sequence of video or even frame sequence after coding; such as odd-numbered frame sequence encoding is generated bit stream to terminal; after terminal receives code stream; the reconstructed frame sequence of odd-numbered frame sequence is obtained by decoding; then usually based on MEMC technology, reconstructed frame sequence is carried out FRUC.
Concrete, first Video service device tries to achieve in reconstructed frame sequence often MV between adjacent two frames by ME algorithm, then adopts MCI algorithm, obtains often slotting frame between adjacent two frames according to the MV tried to achieve. Exemplary, odd-numbered frame sequence is obtained for terminal, as shown in Fig. 2 (a), for the algorithm schematic diagram of FRUC, first Video service device tries to achieve in reconstructed frame sequence the MV between N frame and N+1 frame by ME algorithm, then adopting MCI algorithm, the MV according to trying to achieve obtains the slotting frame between N frame and N+1 frame. As shown in Fig. 2 (b), it it is the schematic diagram of the slotting frame between N frame and N+1 frame. It should be appreciated that when slotting frame between N frame and N+1 frame, time more close to even frame in former sequence of frames of video between N frame and N+1 frame, the quality of the image that terminal presents is more high.
It should be noted that, term " system " and " network " are often used interchangeably herein herein. Term "and/or" herein is only a kind of association relation describing affiliated partner, and expression can exist three kinds of relations, such as, A and/or B, it is possible to represent: Individual existence A, exists A and B simultaneously, these three kinds of situations of Individual existence B. In addition, character "/" herein, general expression forward-backward correlation is to the relation liking a kind of "or".
The embodiment of the present invention provides a kind of frame processing method, and as shown in Figure 3, the method can comprise:
S101, Video service device just frame sequence be separated into main frame sequence and subsidiary frame sequence.
In embodiments of the present invention, frame sequence can be the frame sequence of video waiting for transmission. Concrete, frame sequence can be divided into two portions sequence by Video service device, and wherein part sequence needs the sequence transferring to terminal, in embodiments of the present invention, this partial sequence is called main frame sequence, and the sequence that another part does not need to transfer to terminal is called subsidiary frame sequence.
Exemplary, in embodiments of the present invention, the even frame in frame sequence can be separated by Video service device with odd-numbered frame, and using odd-numbered frame sequence as main frame sequence, using even frame as subsidiary frame sequence, or using even frame sequence as main frame sequence, using odd-numbered frame as subsidiary frame sequence.
Video service device can also by frame n*3 in frame sequence (n=0,1,2 ...) separate as main sequence from frame sequence, the sequence that remaining sequence is in frame sequence frame n*3+1 and frame n*3+2 and forms.
It should be noted that, above-mentioned frame sequence is carried out divide according in actual video transmitting procedure, the requirement of the occupancy of the network bandwidth can be divided, to this, the embodiment of the present invention does not limit.
S102, Video service device obtain the slotting frame sequence of main frame sequence.
Subsidiary frame sequence and slotting frame sequence are subtracted each other to obtain the first residual frame sequence by S103, Video service device.
Concrete, in embodiments of the present invention, subsidiary frame sequence and slotting frame sequence can be subtracted each other the residual frame sequence of gained directly as the first residual frame sequence by Video service device. The residual frame sequence that subsidiary frame sequence and slotting frame sequence can also be subtracted each other rear gained by Video service device carries out series of preprocessing, then pretreated residual frame sequence will be carried out as the first residual frame sequence, residual frame sequence after can also the sequence of the designated area in subsidiary frame sequence and slotting frame sequence being subtracted each other is as the first residual frame sequence, and concrete enforcement mode will be described in detail in the following embodiments.
The main bit stream formed after main frame sequence being encoded and the attached code stream that formed after the first residual frame sequence being encoded are merged to form transmission code stream by S104, Video service device.
Concrete, in embodiments of the present invention, Video service device generates main bit stream after this main frame sequence being encoded, and Video service device generates attached code stream after this first residual frame sequence being encoded, and then main bit stream and attached code stream can be merged by Video service device, generate transmission code stream.
Exemplary, in this video transmission code stream, it is possible to deposit the data of main bit stream in front 100 bit positions, attached code stream is deposited in lower 100 bit positions, then main bit stream is deposited in lower 100 bit positions, analogizes successively. Attached code stream directly can also be left in supplemental enhancement information SEI (English full name: the supplementalenhancementinformation of main bit stream by Video service device, abbreviation: in SEI), namely this transmission code stream is the main bit stream housing attached code stream, thus the SEI of main bit stream can directly be decoded by terminal, so that attached code stream is separated from main bit stream.
This transmission code stream is sent to described terminal by S105, Video service device.
S106, terminal receive transmission code stream, and this transmission code stream is separated into main bit stream and attached code stream.
The attached code stream of this main bit stream and this is decoded to obtain the main frame sequence of reconstruct and the first residual frame sequence of reconstruct by S107, terminal respectively.
It should be noted that, attached code flows through after demoder decodes can only obtain the first residual frame sequence of reconstruct, but error between the first residual frame sequence of reconstruct and the first residual frame sequence is very little, can't cause serious impact. Equally, main bit stream can only obtain the main frame sequence of reconstruct after decoding, and the error between the main frame sequence of reconstruct and main poor frame sequence is also very little.
S108, terminal obtain the slotting frame sequence of the main frame sequence of this reconstruct.
S109, terminal are according to the subsidiary frame sequence of the first residual frame sequence of this reconstruct and the reconstruct of this slotting frame retrieval.
Concrete, in embodiments of the present invention, terminal the first residual frame sequence of directly this slotting frame sequence and this reconstruct can be added the subsidiary frame sequence obtaining and reconstructing. After first residual frame sequence of reconstruct first can also be carried out series of preprocessing by terminal, then frame sequence slotting with this be added, and to obtain the subsidiary frame sequence of reconstruct, concrete enforcement mode will be described in detail in the following embodiments.
The subsidiary frame sequence of the main frame sequence of this reconstruct and this reconstruct is carried out the frame sequence merging to form reconstruct by S110, terminal.
In the embodiment of the present invention, the error between subsidiary frame sequence in the subsidiary frame sequence of reconstruct obtained due to terminal and former frame sequence is much smaller than the error between this slotting frame sequence and subsidiary frame sequence, therefore, by the frame processing method adopting the embodiment of the present invention to provide, the quality of the image that the frame sequence of the reconstruct that terminal obtains presents after exporting, higher than the quality of the image presented after the frame sequence output of the reconstruct slotting frame sequence merged and formed with the main frame sequence of reconstruct in prior art.
Further, in embodiments of the present invention, in above-mentioned S102, Video service device obtains according to this main frame sequence and inserts frame sequence, and the following two kinds mode specifically can be adopted to realize:
Mode one, above-mentioned S102 specifically can comprise:
This main frame sequence is encoded by S102a, Video service device, the main frame sequence after coding is decoded again, carries out changing in frame per second to obtain this slotting frame sequence to this decoded main frame sequence by transfer algorithm in frame per second.
It should be noted that, owing to terminal can only by carrying out the main bit stream received decoding the main frame sequence obtaining reconstruct, instead of main frame sequence, therefore, in embodiments of the present invention, this main frame sequence first can be carried out coding and generate main bit stream by Video service device, again this main bit stream (main frame sequence after namely encoding) is decoded, to obtain the main frame sequence of the reconstruct identical with terminal, and then can carry out frame per second is changed to the main frame sequence (main frame sequence after namely encoding) of this reconstruct.
It should be noted that, in embodiments of the present invention, the main frame sequence of reconstruct is carried out transfer algorithm in the frame per second that adopts when changing in frame per second by Video service device, with terminal after the main frame sequence obtaining reconstruct, the main frame sequence of reconstruct is carried out transfer algorithm in the frame per second that adopts when changing in frame per second completely identical, specifically comprise and adopt with terminal identical ME algorithm in the hope of MV, the interpolation of recycling motion compensation is (English: motioncompensatedinterpolation, abbreviation: MCI) algorithm, the slotting frame sequence of the main frame sequence of this reconstruct is tried to achieve according to the MV tried to achieve.
Wherein, ME algorithm can comprise 3D recursive search (English: 3dimensionrecursivesearch) algorithm, PPC algorithm etc. Even Video service device adopts 3DRS algorithm to seek MV, then terminal adopts 3DRS algorithm to seek MV too; If video terminal adopts PPC algorithm to seek MV, then terminal adopts PPC algorithm to seek MV too.
Correspondingly, above-mentioned S108 specifically can comprise:
The main frame sequence of this reconstruct is undertaken changing in frame per second to obtain slotting frame sequence by S108a, terminal by transfer algorithm in frame per second.
It is understandable that, in embodiments of the present invention, owing to the main frame sequence of reconstruct is undertaken changing in frame per second by transfer algorithm in identical frame per second by Video service device with terminal, therefore, after the main frame sequence of reconstruct is carried out in frame per second conversion by Video service device, it is possible to obtain the slotting frame sequence identical with terminal.
Mode two, above-mentioned S102 specifically can comprise:
Main frame sequence is undertaken changing in frame per second to obtain this slotting frame sequence and motion vector MV by S102b, Video service device by transfer algorithm in frame per second, and this main frame sequence of reconstruct is carried out changing in frame per second by this MV for terminal.
In embodiments of the present invention, after former frame sequence waiting for transmission is separated into main frame sequence and subsidiary frame sequence by Video service device, can directly carry out main frame sequence frame per second is changed.
Concrete, Video service device need to adopt transfer algorithm in the frame per second adopted when carrying out changing in frame per second to the main frame sequence of reconstruct with terminal that main frame sequence is carried out frame per second. Comprise and first try to achieve MV by the ME algorithm in transfer algorithm in frame per second, recycle the MCI algorithm in transfer algorithm in this frame per second, try to achieve the slotting frame sequence of this main frame sequence according to the MV tried to achieve.
In embodiments of the present invention, main frame sequence is undertaken changing in frame per second by Video service device by transfer algorithm in frame per second, while obtaining this slotting frame sequence, the MV tried to achieve by above-mentioned ME algorithm can be obtained simultaneously, and then the first residual frame sequence and this MV can be all encoded in attached code stream by Video service device, so that terminal when to this attached code stream decoding, decode this MV simultaneously, and then terminal can directly utilize this MV when carrying out changing in frame per second to the main frame sequence reconstructed, obtain the slotting frame sequence of the main frame sequence of reconstruct.
Correspondingly, terminal is by decoding attached code stream, and after getting this MV, above-mentioned S108 specifically can comprise:
The main frame sequence of this reconstruct, according to this MV, is undertaken changing in frame per second to obtain slotting frame sequence by S108b, terminal by transfer algorithm in frame per second.
It should be noted that, in embodiments of the present invention, owing to terminal is in the process that the main frame sequence reconstructed carries out conversion in frame per second, it is the MV that domestic video server offer is provided, try to achieve the slotting frame sequence of the main frame sequence of reconstruct, namely ensureing that terminal carries out interpolation arithmetic with Video service device according to identical MV, so in order to simplify the treatment scheme of Video service device, main frame sequence can directly be carried out changing in frame per second by Video service device.Although what Video service device inputted when carrying out MC and operate is main frame sequence, what terminal inputted when carrying out MC and operate is this main frame sequence of reconstruct, but owing to terminal and Video service device adopt identical MV, and reconstruct main frame sequence and main frame sequence between error very little, therefore terminal obtain slotting frame sequence and Video service device obtain slotting frame sequence between error be negligible.
Further, in embodiments of the present invention, in above-mentioned S103, subsidiary frame sequence and slotting frame sequence are subtracted each other to obtain the first residual frame sequence by Video service device, can be achieved in several ways, enumerate three kinds of preferred implementations (mode 1, mode 2, mode 3 and mode 4) below and above-mentioned S103 is carried out exemplary explanation.
Mode 1, above-mentioned S103 specifically can comprise:
Slotting to this subsidiary frame sequence and this frame sequence is subtracted each other the residual frame sequence of rear gained directly as the first residual frame sequence by S103a, Video service device.
Correspondingly, above-mentioned S109 specifically can comprise:
This slotting frame sequence is added, with the first residual frame sequence of this reconstruct, the subsidiary frame sequence obtaining reconstruct by S109a, terminal.
What deserves to be explained is, in embodiments of the present invention, terminal is that the residual frame sequence that subsidiary frame sequence and slotting frame sequence subtract each other rear gained is directly done the first residual frame sequence, thus terminal after carrying out decoding the first residual frame sequence obtaining reconstruct to attached code stream, it is possible to by this slotting frame sequence to be added the subsidiary frame sequence directly obtaining reconstruct with the first residual frame sequence of this reconstruct. owing to the error between the subsidiary frame sequence in the subsidiary frame sequence of this reconstruct and former frame sequence is much smaller than the error between this slotting frame sequence and this subsidiary frame sequence, therefore, error between the frame sequence of the reconstruct that terminal obtains in the embodiment of the present invention and the frame sequence of former video, much smaller than terminal in prior art, slotting frame sequence is merged with the main frame sequence of reconstruct the error between the frame sequence of the reconstruct of formation and the frame sequence of former video, thus compared to prior art, the picture quality of terminal output can be improved by exporting the frame sequence of the reconstruct that terminal obtains in the embodiment of the present invention.
Optionally, in order to reduce Video service device to the network bandwidth taken during video terminal transmission of video transmission code stream, the embodiment of the present invention proposes such as under type 2, mode 3 and mode 4, by in the process obtaining this first residual frame sequence, reduce the bit number of this first residual frame sequence, thus reduce the encoded rear attached code stream generated of the first residual frame sequence in transmitting procedure, the network bandwidth taken.
Mode 2, above-mentioned S103 specifically can comprise:
The residual frame sequence that slotting to this subsidiary frame sequence and this frame sequence subtracts each other rear gained is carried out pre-treatment by S103b, Video service device, using pretreated residual frame sequence as the first residual frame sequence.
Wherein, the residual frame sequence that the slotting frame sequence of this subsidiary frame sequence and this is subtracted each other rear gained by Video service device carries out one or more that pre-treatment includes but not limited in following four kinds of modes. These four kinds of modes comprise:
A, Video service device are by the chromatic(ity)component zero setting of each residual frame in residual frame sequence.
The resolving power of each residual frame in residual frame sequence is reduced by B, Video service device according to the scaling preset.
The pixel of each residual frame in residual frame sequence is carried out denoising according to the noise figure preset by C, Video service device.
The pixel of each residual frame in residual frame sequence is quantized by D, Video service device according to the quantized value preset.
It should be noted that, in embodiments of the present invention, Video service device is when carrying out pre-treatment to residual frame sequence, can perform arbitrarily above-mentioned four kinds operation in one or more, if and Video service device selects multiple operation that residual frame sequence is carried out pre-treatment, so for performing order between this multiple operation, the embodiment of the present invention is not restricted.
Exemplary, if residual frame sequence is carried out pre-treatment by Video service device selection mode A, so, it is understandable that, in mode one, the the first residual frame sequence obtained due to Video service device is the brightness component of this residual frame sequence, therefore the bit number of the attached code stream that Video service device generates after the first residual frame sequence being encoded, it is less than the bit number that directly this residual frame sequence is carried out the attached code stream encoded and generate, thus reduces the network bandwidth that this attached code stream takies in transmitting procedure.
Further, after terminal obtains the first residual frame sequence of reconstruct by this attached code stream of decoding, slotting frame sequence is added with the first residual frame sequence of reconstruct, the subsidiary frame sequence of the reconstruct obtained, the slotting frame sequence being terminal acquisition adds this slotting frame sequence and the subsidiary frame sequence difference on brightness component. So, due to the error error that is less than between this slotting frame sequence and subsidiary frame sequence between the subsidiary frame sequence of reconstruct and subsidiary frame sequence, therefore, by the frame processing method adopting the embodiment of the present invention to provide, the quality of the image that the frame sequence of the reconstruct that terminal obtains presents after exporting, higher than the quality of the image presented after the frame sequence output of the reconstruct slotting frame sequence merged and formed with the main frame sequence of reconstruct in prior art.
Exemplary, if the residual frame sequence that the slotting frame sequence of this subsidiary frame sequence and this subtracts each other rear gained is carried out pre-treatment by Video service device selection mode B, C and D, so, the resolving power assuming each residual frame in this residual frame sequence is N, pixel is P, Video service device is according to the scaling M (M > 1), the noise figure D (D > 0) that preset, and quantized value Q (Q > 1), obtain the first residual frame after each the 2nd residual frame is reduced resolution processes, denoising and quantification treatment successively.
Concrete, the resolution, N that slotting to this subsidiary frame sequence and this frame sequence is first subtracted each other each residual frame in the residual frame sequence of rear gained by Video service device according to M is reduced into N/M. Then according to D, the pixel P of each residual frame is carried out denoising, exemplary, it is possible to according to formula PCThe pixel transitions of each residual frame is P by=sign (P) �� (| P |-D)C, complete denoising.
Further, the pixel in each residual frame is PCBasis on, then according to Q, the pixel of each residual frame is carried out quantification treatment. Exemplary, it is possible to according to formula Pq=PCThe pixel transitions of each residual frame is P by �� Qq, complete quantification treatment.
It should be appreciated that by above-mentioned pre-treatment, the resolving power of the first residual frame sequence that Video service device obtains is N/M, and pixel is Pq. Owing to N/M is less than N, PqBeing less than P, therefore the bit number of the first residual frame sequence subtracts each other the bit number of the residual frame sequence of rear gained much smaller than the slotting frame sequence of this subsidiary frame sequence and this. Thus reduce the network bandwidth that this first residual frame sequence generates attached code stream after coding and takies in transmitting procedure.
It should be noted that, residual frame sequence selection mode A, B, C and D can also be carried out pre-treatment by Video service device, it is also possible to select other combinations in A, B, C and D that residual frame sequence is carried out pre-treatment, will not enumerate herein.
Further, in mode 2, Video service device also needs to be merged in transmission code stream by the parameter adopted when residual frame sequence is carried out pre-treatment and is sent to terminal, so that terminal replys this residual frame sequence according to this parameter. Wherein, the parameter adopted when residual frame sequence is carried out pre-treatment by Video service device can comprise at least one in scaling, noise figure and quantized value.
Further, after at least one parameter being separated into out in scaling, noise figure and quantized value when terminal from transmission code stream, above-mentioned S109 specifically can comprise;
First residual frame sequence of reconstruct is carried out pre-treatment by S109b, terminal.
Wherein, the first residual frame sequence of reconstruct is carried out one or more that pre-treatment can include but not limited in following four kinds of modes by terminal, specifically comprises:
(1), the resolving power of the first residual frame of each reconstruct in the first residual frame sequence of this reconstruct is amplified by terminal according to this scaling.
(2), the pixel of the first residual frame of each reconstruct in the first residual frame sequence of this reconstruct is carried out anti-denoising according to described noise figure by terminal.
(3), the pixel of the first residual frame of each reconstruct in the first residual frame sequence of this reconstruct is carried out inverse quantization according to described quantized value by terminal.
Exemplary, corresponding to the citing in above-mentioned S103b, it is assumed that the resolving power of the first residual frame of each reconstruct in the first residual frame sequence that terminal obtains is N/M, and pixel is Pq, putting contracting ratio is M, and noise figure is D, and quantized value is Q.
The pixel of the first residual frame that each is first reconstructed by terminal according to Q carries out inverse quantization process, and such as terminal can according to formula PC=PqThe pixel transitions of each the first residual frame reconstructed is P by �� QC, complete inverse quantization process; Then, terminal according to noise figure D, in conjunction with according to formula P = P C ! = 0 : s i g n ( P C ) × ( | P C | + D ) P C = = 0 : 0 , The pixel transitions of the first residual frame each reconstructed is P, completes anti-denoising.
Further, terminal is enlarged into N according to the resolution, N/M of the first residual frame that each is reconstructed by M. Namely, after the first residual frame sequence of reconstruct is carried out pre-treatment according to Q, M, D by terminal, the resolving power of the first residual frame sequence of reconstruct is N, and pixel is P.
Slotting frame sequence is added, with the first residual frame sequence carrying out pretreated reconstruct, the subsidiary frame sequence obtaining reconstruct by S109c, terminal.
It is understandable that, error between the subsidiary frame sequence of this reconstruct obtained due to terminal in present example and subsidiary frame sequence is much smaller than the error between this slotting frame sequence and subsidiary frame sequence, therefore, the quality of the image that the frame sequence of the reconstruct that terminal obtains presents after exporting, higher than the quality of the image presented after the frame sequence output of the reconstruct in prior art, slotting frame sequence merged and formed with the main frame sequence of reconstruct, namely the frame processing method that the embodiment of the present invention provides is adopted, it is possible to improve the quality of the image that terminal exports.
Mode 3, Video service device can first be determined to belong to the sequence of the designated area of frame sequence in subsidiary frame sequence and belong to the sequence of this designated area in slotting frame sequence, and after then the sequence belonging to this designated area in subsidiary frame sequence and the sequence that belongs to this designated area in slotting frame sequence being subtracted each other, the residual frame sequence of gained is as the first residual frame sequence.
It should be appreciated that compared to mode 1, the first residual frame sequence in mode 3 is that the sequence of the designated area belonging to frame sequence in this subsidiary frame sequence and the sequence that belongs to this designated area in this slotting frame sequence subtract each other gained.
Wherein, this designated area can comprise and blocking (English: cover) region and area-of-interest (English: regionofinterest, abbreviation: at least one in ROI).
Exemplary, this frame sequence is the frame sequence of video, and occlusion area can comprise the channel logo position of television station, or screen menu type regulates (English: onscreendisplay, abbreviation: OSD) region etc., in these occlusion areas, the picture of video may be invisible. ROI is that people pay close attention to region when the image of former video is carried out image analysis, and the image that these are only paid close attention to region by people has high quality requirement.
Namely in mode 3, the sequence of the designated area belonging to frame sequence can be processed by Video service device, namely only obtain the first residual frame sequence of this designated area, thus reduce the network bandwidth that this first residual frame sequence generates attached code stream after coding and takies in transmitting procedure.
It should be noted that, when only obtaining the first residual frame sequence belonging to this designated area when Video service device, the information being used to indicate this designated area of this designated area need to be merged in transmission code stream and be sent to terminal by Video service device, so that terminal from transmission code stream, isolate the information of this designated area after, terminal can obtain the subsidiary frame sequence of reconstruct according to the first residual frame sequence of the information of this designated area, reconstruct and slotting frame sequence.
Concrete, in mode 3, above-mentioned S109 specifically can comprise:
S109c, terminal determine to belong in this slotting frame sequence the sequence of this designated area according to the information of this designated area.
First residual frame sequence of the sequence and this reconstruct that belong to this designated area in this slotting frame sequence is added the subsidiary frame sequence obtaining this reconstruct by S109d, terminal.
It is understandable that, after terminal decodes the information of this designated area from attached code stream, terminal can determine to belong in this slotting frame sequence according to the information of this designated area the sequence of this designated area, then add that terminal obtains the first residual frame sequence of reconstruct for belonging to the sequence of this designated area in this slotting frame sequence, generate the subsidiary frame sequence of reconstruct. owing to belonging to the error between the sequence of this designated area and the sequence belonging to this designated area in subsidiary frame sequence in the subsidiary frame sequence of reconstruct, it is less than and belongs to the error between the sequence of this designated area and the sequence belonging to designated area in subsidiary frame sequence in this slotting frame sequence, therefore, the quality of the image that the frame sequence of the reconstruct that terminal obtains presents after exporting, higher than the quality of the image presented after the frame sequence output of the reconstruct in prior art, slotting frame sequence merged and formed with the main frame sequence of reconstruct, namely the frame processing method that the embodiment of the present invention provides is adopted, the quality of the image that terminal exports can be improved.
Mode 4, combine aforesaid way 2 and the scheme of mode 3, it is specially Video service device can first determine to belong to the sequence of the designated area of frame sequence in subsidiary frame sequence and in slotting frame sequence, belong to the sequence of this designated area, then after the sequence belonging to this designated area in subsidiary frame sequence and the sequence that belongs to this designated area in slotting frame sequence being subtracted each other, the residual frame sequence of gained carries out pre-treatment, will carry out this residual frame sequence pretreated as the first residual frame sequence.
Wherein, after the sequence belonging to this designated area in subsidiary frame sequence and the sequence that belongs to this designated area in slotting frame sequence being subtracted each other, the residual frame sequence of gained carries out pretreated mode, with in mode 2, the residual frame sequence that subsidiary frame sequence and slotting frame sequence are subtracted each other rear gained to be carried out pretreated mode identical, repeat no more herein.
It is understandable that, in mode 4, Video service device only obtains the first residual frame sequence belonging to this designated area, and the first residual frame sequence obtained is through pretreated, thus reduce the network bandwidth that this first residual frame sequence generates attached code stream after coding and takies in transmitting procedure.
Further, in mode 4, when after this first residual frame sequence of Video service device, Video service device need to by the information of this designated area, and the parameter of employing when the residual frame sequence of gained carries out pre-treatment after the sequence belonging to this designated area in subsidiary frame sequence and the sequence that belongs to this designated area in slotting frame sequence being subtracted each other is incorporated in transmission code stream to be sent to terminal, so that the terminal information of isolating this designated area from transmission code stream and correlation parameter after, can according to the information of this designated area, correlation parameter, first residual frame sequence of reconstruct and slotting frame sequence obtain the subsidiary frame sequence of reconstruct.
Concrete, above-mentioned S109 specifically can comprise:
First residual frame sequence of reconstruct is carried out pre-treatment by S109e, terminal.
Wherein, in S109e, the first residual frame sequence of reconstruct is carried out pretreated mode by terminal, with terminal in above-mentioned S109b, the first residual frame sequence of reconstruct is carried out pretreated mode identical, repeats no more herein.
S109f, terminal determine to belong in this slotting frame sequence the sequence of this designated area according to the information of this designated area.
The sequence belonging to this designated area in this slotting frame sequence is added, with the 2nd residual frame sequence carrying out this reconstruct pretreated, the subsidiary frame sequence obtaining this reconstruct by S109g, terminal.
It is understandable that, with reason aforesaid way 3, in the embodiment of the present invention, due to the error between the frame sequence of designated area of subsidiary frame sequence of reconstruct and the frame sequence of the designated area of subsidiary frame sequence, the error being less than between the frame sequence of the frame sequence of the designated area of this slotting frame sequence and the designated area of subsidiary frame sequence, therefore, the quality of the image that the frame sequence of the reconstruct that terminal obtains presents after exporting, higher than the quality of the image presented after the frame sequence output of the reconstruct in prior art, slotting frame sequence merged and formed with the main frame sequence of reconstruct, namely the frame processing method that the embodiment of the present invention provides is adopted, the quality of the image that terminal exports can be improved.
The frame processing method that the embodiment of the present invention provides, terminal can obtain the first residual frame sequence of the reconstruct of the first residual frame sequence that Video service device provides, and slotting frame sequence is added the subsidiary frame sequence obtaining reconstruct with the first residual frame sequence of this reconstruct. Due to reconstruct subsidiary frame sequence and subsidiary frame sequence between error much smaller than the error between this slotting frame sequence and subsidiary frame sequence, therefore, the quality of the image that the frame sequence of the reconstruct that terminal obtains presents after exporting, higher than the quality of the image presented after the frame sequence output of the reconstruct in prior art, slotting frame sequence merged and formed with the main frame sequence of reconstruct, therefore the frame processing method that the embodiment of the present invention provides is adopted, it is possible to improve the quality of the image that terminal exports.
As shown in Figure 4, the embodiment of the present invention provides a kind of Video service device, and described Video service device is for the step performed by the Video service device that performs in frame processing method as shown in Figure 3. Described Video service device can comprise the module corresponding to corresponding steps. Example, comprising:
Separating unit 10, for being separated into main frame sequence and subsidiary frame sequence by frame sequence.
Acquiring unit 11, for obtaining the slotting frame sequence of the described main frame sequence of described separating unit 10 separation.
Described acquiring unit 11, also for subtracting each other to obtain the first residual frame sequence by described subsidiary frame sequence and described slotting frame sequence.
Merge cells 12, for merging to form transmission code stream by the main bit stream formed after described main frame sequence being encoded and the attached code stream that formed after described first residual frame sequence being encoded.
Send unit 13, for described transmission code stream is sent to terminal, so that described terminal recovers described frame sequence according to described transmission code stream.
Optionally, described acquiring unit 11, specifically for described subsidiary frame sequence and described slotting frame sequence are subtracted each other rear gained residual frame sequence directly as the first residual frame sequence.
Optionally, described acquiring unit 11, specifically for carrying out pre-treatment by the residual frame sequence that described subsidiary frame sequence and described slotting frame sequence subtract each other rear gained, using pretreated described residual frame sequence as the first residual frame sequence.
Optionally, described merge cells 12, also for the parameter adopted when described residual frame sequence is carried out pre-treatment being merged in described transmission code stream.
Optionally, described acquiring unit 11, after subtracting each other specifically for the sequence of the designated area using belonging to described frame sequence in described subsidiary frame sequence and the sequence that belongs to described designated area in described slotting frame sequence, the residual frame sequence of gained is as the first residual frame sequence.
Optionally, described acquiring unit 11, after subtracting each other specifically for the sequence of the designated area by belonging to described frame sequence in described subsidiary frame sequence and the sequence that belongs to described designated area in described slotting frame sequence, the residual frame sequence of gained carries out pre-treatment, will carry out pretreated described residual frame sequence as the first residual frame sequence.
Optionally, described merge cells 12, also for the information of described designated area being merged in described transmission code stream.
Optionally, described acquiring unit 11, specifically for carrying out changing in frame per second to obtain described slotting frame sequence and motion vector MV to described main frame sequence by transfer algorithm in frame per second, the described main frame sequence of reconstruct is carried out changing in frame per second by described MV for described terminal.
Described attached code stream also comprises described MV.
It is appreciated that, the Video service device of the present embodiment may correspond to the Video service device in the frame processing method of above-mentioned embodiment as described in Figure 3, and the division of each module in the Video service device of the present embodiment and/or function etc. are all to realize method flow as shown in Figure 3, for simplicity, do not repeat them here.
The embodiment of the present invention provides a kind of Video service device, it is possible to subsidiary frame sequence in getting frame sequence and the first residual frame sequence between slotting frame sequence, by the error of this first residual frame sequence to reduce between subsidiary frame sequence and slotting frame sequence. it is specifically as follows Video service device to obtain after the first residual frame sequence by subsidiary frame sequence and slotting frame sequence being subtracted each other, this the first residual frame sequence is sent to terminal after coding, so that after terminal gets this first residual frame sequence of reconstruct, the slotting frame sequence that can obtain according to the first residual frame sequence of reconstruct and terminal, obtain the subsidiary frame sequence of reconstruct, due to reconstruct subsidiary frame sequence and frame sequence in subsidiary frame sequence between error, the error being less than between slotting frame sequence and subsidiary frame sequence, therefore, after the subsidiary frame sequence of reconstruct is merged, with the main frame sequence of reconstruct, the frame sequence forming reconstruct by terminal, the quality of the image that terminal presents after the frame sequence of this reconstruct being exported, higher than the quality of the image presented after the frame sequence output of the reconstruct in prior art, slotting frame sequence merged and formed with the main frame sequence of reconstruct.
As shown in Figure 5, the embodiment of the present invention provides a kind of terminal, and described terminal is for the step performed by the terminal that performs in frame processing method as shown in Figure 3. Described terminal can comprise the module corresponding to corresponding steps. Example, comprising:
Receive unit 20, for receiving transmission code stream, and described transmission code stream is separated into main bit stream and attached code stream.
Decoding unit 21, for decoding to obtain the main frame sequence of reconstruct and the first residual frame sequence of reconstruct respectively to the described main bit stream and described attached code stream that receive unit 20 separation.
Acquiring unit 22, for obtaining the slotting frame sequence of the main frame sequence of the described reconstruct that decoding unit 21 obtains.
Described acquiring unit 22, also for the subsidiary frame sequence of the first residual frame sequence according to described reconstruct and the reconstruct of described slotting frame retrieval.
Merge cells 23, the subsidiary frame sequence for the described reconstruct main frame sequence and described acquiring unit 22 that decode the described reconstruct that unit 21 obtains obtained carries out merging the frame sequence forming reconstruct.
Optionally, described acquiring unit 22, specifically for being added, with the first residual frame sequence of described reconstruct, the subsidiary frame sequence obtaining described reconstruct by described slotting frame sequence.
Optionally, described acquiring unit 22, specifically for carrying out pre-treatment to the first residual frame sequence of described reconstruct.
Described acquiring unit 22, also for described slotting frame sequence is added the subsidiary frame sequence obtaining described reconstruct with the first residual frame sequence carrying out pretreated described reconstruct.
Optionally, the described transmission code stream that described reception unit 20 receives also comprises the information of designated area.
Described acquiring unit 22, specifically for determining to belong in described slotting frame sequence the sequence of described designated area according to the information of described designated area.
Described acquiring unit 22, also for the first residual frame sequence of the sequence and described reconstruct that belong to described designated area in described slotting frame sequence is added the subsidiary frame sequence obtaining described reconstruct.
Optionally, the described transmission code stream that described reception unit 20 receives also comprises the information of designated area.
Described acquiring unit 22, specifically for carrying out pre-treatment to the first residual frame sequence of described reconstruct.
Described acquiring unit 22, also for determining to belong in described slotting frame sequence the sequence of described designated area according to the information of described designated area.
Described acquiring unit 22, also for the sequence belonging to described designated area in described slotting frame sequence is added the subsidiary frame sequence obtaining described reconstruct with the first residual frame sequence carrying out pretreated described reconstruct.
Optionally, the described attached code stream of described reception unit 20 separation also comprises motion vector MV.
Described acquiring unit 22, specifically for according to described MV, carrying out changing in frame per second to obtain described slotting frame sequence to the main frame sequence of described reconstruct by transfer algorithm in frame per second.
It is appreciated that, the terminal of the present embodiment may correspond to the terminal in the frame processing method of above-mentioned embodiment as described in Figure 3, and the division of each module in the terminal of the present embodiment and/or function etc. are all to realize method flow as shown in Figure 3, for simplicity, do not repeat them here.
The present invention provides a kind of terminal, the first residual frame sequence of reconstruct can be obtained according to the attached code stream decoding that Video service device sends, and after terminal obtains slotting frame sequence, terminal can further according to the first residual frame sequence of reconstruct and the subsidiary frame sequence of this slotting frame retrieval reconstruct, owing to the first residual frame sequence Video service device obtains according to the subsidiary frame sequence in frame sequence and slotting frame sequence being subtracted each other, therefore, error between the subsidiary frame sequence of reconstruct and subsidiary frame sequence, the error being less than between slotting frame sequence and subsidiary frame sequence, thus, when the quality of the image that terminal presents after the main frame sequence of the subsidiary frame sequence of described reconstruct and described reconstruct is merged the frame sequence output of the reconstruct formed, the image presented after slotting frame sequence being merged output with the main frame sequence of reconstruct higher than terminal in prior art, namely by terminal provided by the invention, the picture quality that terminal exports can be improved.
As shown in Figure 6, the embodiment of the present invention provides a kind of Video service device, and this Video service device comprises: treater 30, storer 31, system bus 32 and communication interface 33.
Described storer 31 is for storing computer executed instructions, described treater 30 is connected by described system bus 32 with described storer 31, when described Video service device runs, described treater 30 performs the described computer executed instructions that described storer 31 stores, so that described Video service device performs frame processing method as described in Figure 3. Concrete frame processing method see the associated description in above-mentioned embodiment as shown in Figure 3, can repeat no more herein.
Wherein, described treater 30 can be central processing unit (English: centralprocessingunit, abbreviation: CPU). Described treater 30 can also be other general procedure devices, digital signal processor (English: digitalsignalprocessing, be called for short DSP), application specific integrated circuit (English: applicationspecificintegratedcircuit, be called for short ASIC), field-programmable gate array (English: field-programmablegatearray, it is called for short FPGA) or the chip such as other programmable logic device parts, discrete gate or transistor logical device, discrete hardware components, or the combination of above-mentioned all kinds of chip. General procedure device can be microprocessor or this treater can also be the treater etc. of any routine.
Described treater 30 can be application specific processor, and this application specific processor can comprise at least one in baseband processing chip, radio frequency processing chip etc. Further, this application specific processor can also comprise the chip with other dedicated processes functions of Video service device.
Described storer 31 can comprise volatile memory, and (English: volatilememory), such as (English: random-accessmemory, abridges: RAM) random access memory; It is (English: non-volatilememory) that described storer 31 can also comprise nonvolatile memory, such as read-only storage is (English: read-onlymemory, abbreviation: ROM), flash device is (English: flashmemory), hard disk is (English: harddiskdrive, abbreviation: HDD) or solid state hard disc (English: solid-statedrive, abbreviation: SSD); Described storer 31 can also comprise the combination of the storer of mentioned kind.
Described system bus 32 can comprise data bus, power supply bus, control bus and signal status bus etc. In order to clearly demonstrate in the present embodiment, in the figure 7 various bus is all illustrated as system bus 32.
In specific implementation process, the computer executed instructions of the software form that each step in above-mentioned method flow as shown in Figure 3 all can perform in storer 31 by the treater 30 of example, in hardware to store realizes. For avoiding repetition, repeat no more herein.
The embodiment of the present invention provides a kind of Video service device, it is possible to subsidiary frame sequence in getting frame sequence and the first residual frame sequence between slotting frame sequence, by the error of this first residual frame sequence to reduce between subsidiary frame sequence and slotting frame sequence. it is specifically as follows Video service device by this first residual frame sequence is sent to terminal after coding, so that after terminal gets this first residual frame sequence of reconstruct, the slotting frame sequence that can obtain according to the first residual frame sequence of reconstruct and terminal, obtain the subsidiary frame sequence of reconstruct, due to reconstruct subsidiary frame sequence and frame sequence in subsidiary frame sequence between error, the error being less than between slotting frame sequence and subsidiary frame sequence, therefore, the quality of the image that terminal presents after the main frame sequence of the subsidiary frame sequence of described reconstruct and described reconstruct is merged output by terminal, higher than terminal in prior art, slotting frame sequence is merged with the main frame sequence of reconstruct the image exporting rear terminal and presenting, namely by method for converting video frame rate provided by the invention, the picture quality that terminal exports can be improved.
As shown in Figure 7, the embodiment of the present invention provides a kind of terminal, and this terminal comprises: treater 40, storer 41, system bus 42 and communication interface 43.
Described storer 41 is for storing computer executed instructions, described treater 40 is connected by described system bus 42 with described storer 41, when described terminal operating, described treater 40 performs the described computer executed instructions that described storer 41 stores, so that described terminal performs frame processing method as described in Figure 3. Concrete frame processing method see the associated description in above-mentioned embodiment as indicated at 3, can repeat no more herein.
Wherein, described treater 40 can be CPU. Described treater 40 can also be the chips such as other general procedure devices, DSP, ASIC, FPGA or other programmable logic device parts, discrete gate or transistor logical device, discrete hardware components, or the combination of above-mentioned all kinds of chip. General procedure device can be microprocessor or this treater can also be the treater etc. of any routine.
Described treater 40 can be application specific processor, and this application specific processor can comprise at least one in baseband processing chip, radio frequency processing chip etc. Further, this application specific processor can also comprise the chip with other dedicated processes functions of terminal.
Described storer 41 can comprise volatile memory, such as random access memory ram; Described storer 41 can also comprise nonvolatile memory, such as read only memory ROM, flash device, HDD or SSD; Described storer 41 can also comprise the combination of the storer of mentioned kind.
Described system bus 42 can comprise data bus, power supply bus, control bus and signal status bus etc. In order to clearly demonstrate in the present embodiment, in the figure 7 various bus is all illustrated as system bus 42.
In specific implementation process, the computer executed instructions of the software form that each step in above-mentioned method flow as shown in Figure 3 all can perform in storer 41 by the treater 40 of example, in hardware to store realizes. For avoiding repetition, repeat no more herein.
Optionally, under a kind of special scene, when terminal is TV, for the main frame sequence of reconstruct is carried out in frame per second conversion FRUC chip may and demoder not on the same chip. Concrete, when terminal is TV, described treater 40 possibility two portions, a part is that master control chip in demoder place is (English: systemonchip, abbreviation: SoC), GPU) this SoC specifically comprises CPU and graphic process unit, and (English: graphicsprocessingunit, abbreviation: the treater such as, for completing the application such as menu, audio/video decoding, network and game. Another part is timing/counting control device (English: timecontrol, abbreviation: TCON), FRUC chip can be integrated in TCON, or is an independent process chip on the chip of TCON place. TCON mainly completes sequential control.
Owing to FRUC chip is not in the SoC chip at demoder place, therefore when TV terminal receives video transmission code stream, and decode the first residual frame sequence of reconstruct, and after the main frame sequence of reconstruct, it is necessary to by the first residual frame sequence of interface transmission reconstruct between SoC and FRUC chip and the main frame sequence of reconstruct. It is described for Vx1 interface conventional at present below.
In embodiments of the present invention, the first residual frame sequence of reconstruct and the main frame sequence of reconstruct can alternately be transmitted. Exemplary, when the resolving power of the first residual frame reconstructed and the prime frame of reconstruct is 3840 �� 2160, Vx1 interface sequence is as shown in Fig. 8 (a), and when the resolving power of the first residual frame of reconstruct is reduced into 1920x1080, then Vx1 interface sequence is as shown in Fig. 8 (b).Piece size according to FRUC definition, each frame waiting for transmission is put into block (such as the scope that Fig. 8 (a) and figure dotted line as shown in Fig. 8 (b) are confined) transmit, wherein, in each block, part region except the frame that need to transmit is blanking district, synchronizing signal, private information etc. can be deposited, it is also possible to deposit the scaling in above-described embodiment, noise figure, quantized value, and/or MV etc.
Assume for MV, if the MV that terminal adopts domestic video server to be provided seeks slotting frame sequence, then can deposit MV in definition part region, this blanking district, exemplary, when this region definition depositing MV is when the left side of frame waiting for transmission, Vx1 interface sequence is as shown in Fig. 8 (c).
The present invention provides a kind of terminal, the first residual frame sequence of the reconstruct of the first residual frame sequence can be obtained according to the attached code stream decoding that Video service device sends, and after terminal obtains slotting frame sequence, terminal can further according to the first residual frame sequence of reconstruct and the subsidiary frame sequence of this slotting frame retrieval reconstruct, owing to the first residual frame sequence is that Video service device is according to the subsidiary frame sequence in frame sequence and slotting frame retrieval, therefore, reconstruct subsidiary frame sequence and frame sequence in subsidiary frame sequence between error, the error being less than between slotting frame sequence and subsidiary frame sequence, therefore, the quality of the image that terminal presents after the main frame sequence of the subsidiary frame sequence of described reconstruct and described reconstruct is merged output by terminal, higher than terminal in prior art, slotting frame sequence is merged with the main frame sequence of reconstruct the image exporting rear terminal and presenting, namely by method for converting video frame rate provided by the invention, the picture quality that terminal exports can be improved.
Through the above description of the embodiments, the technician of art can be well understood to, for convenience and simplicity of description, only it is illustrated with the division of above-mentioned each function module, in practical application, can complete by different function modules as required and by above-mentioned functions distribution, be divided into different function modules by the internal structure of device, to complete all or part of function described above. The concrete working process of the system of foregoing description, device and unit, it is possible to reference to the corresponding process in aforementioned embodiment of the method, do not repeat them here.
In several embodiments that the application provides, it should be appreciated that, disclosed system, device and method, it is possible to realize by another way. Such as, device embodiment described above is only schematic, such as, the division of described module or unit, being only a kind of logic function to divide, actual can have other dividing mode when realizing, such as multiple unit or assembly can in conjunction with or another system can be integrated into, or some features can ignore, or do not perform. Another point, shown or discussed coupling each other or directly coupling or communication connection can be the indirect coupling by some interfaces, device or unit or communication connection, it is possible to be electrical, machinery or other form.
The described unit illustrated as separating component or can may not be and physically separates, and the parts as unit display can be or may not be physical location, namely can be positioned at a place, or can also be distributed on multiple NE. Some or all of unit wherein can be selected according to the actual needs to realize the object of the present embodiment scheme.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, it is also possible to is that the independent physics of each unit exists, it is also possible to two or more unit are in a unit integrated.Above-mentioned integrated unit both can adopt the form of hardware to realize, it is also possible to adopts the form of software functional unit to realize.
If described integrated unit realize using the form of software functional unit and as independent production marketing or when using, it is possible to be stored in a computer read/write memory medium. Based on such understanding, the technical scheme of the present invention in essence or says that part prior art contributed or all or part of of this technical scheme can embody with the form of software product, this computer software product is stored in a storage media, comprise some instructions with so that a computer equipment (can be Personal Computer, server, or the network equipment etc.) or treater (processor) perform all or part of step of method described in each embodiment of the present invention. And aforesaid storage media comprises: USB flash disk, portable hard drive, read-only storage (ROM, Read-OnlyMemory), random access memory (RAM, RandomAccessMemory), magnetic disc or CD etc. various can be program code stored medium.
The above; it is only the specific embodiment of the present invention, but protection scope of the present invention is not limited thereto, any it is familiar with those skilled in the art in the technical scope that the present invention discloses; change can be expected easily or replace, all should be encompassed within protection scope of the present invention. Therefore, protection scope of the present invention should be as the criterion with the protection domain of described claim.

Claims (30)

1. a frame processing method, it is characterised in that, comprising:
Frame sequence is separated into main frame sequence and subsidiary frame sequence;
Obtain the slotting frame sequence of described main frame sequence;
Subtract each other to obtain the first residual frame sequence by described subsidiary frame sequence and described slotting frame sequence;
Merge to form transmission code stream by the main bit stream formed after described main frame sequence being encoded and the attached code stream that formed after described first residual frame sequence being encoded;
Described transmission code stream is sent to terminal, so that described terminal recovers described frame sequence according to described transmission code stream.
2. frame processing method according to claim 1, it is characterised in that, described subtract each other to obtain the first residual frame sequence by described subsidiary frame sequence and described slotting frame sequence, comprising:
The residual frame sequence that described subsidiary frame sequence and described slotting frame sequence are subtracted each other rear gained is directly as the first residual frame sequence.
3. frame processing method according to claim 1, it is characterised in that, described subtract each other to obtain the first residual frame sequence by described subsidiary frame sequence and described slotting frame sequence, comprising:
The residual frame sequence that described subsidiary frame sequence and described slotting frame sequence subtract each other rear gained is carried out pre-treatment, using pretreated described residual frame sequence as the first residual frame sequence.
4. frame processing method according to claim 3, it is characterised in that, described residual frame sequence is carried out that pre-treatment comprises in following four kinds of modes one or more:
By the chromatic(ity)component zero setting of each residual frame in described residual frame sequence;
According to default scaling, the resolving power of each residual frame in described residual frame sequence is reduced;
According to default noise figure, the pixel of each residual frame in described residual frame sequence is carried out denoising; And
According to default quantized value, the pixel of each residual frame in described residual frame sequence is quantized.
5. frame processing method according to claim 4, it is characterised in that, described method also comprises:
Being merged in described transmission code stream by the parameter adopted when described residual frame sequence is carried out pre-treatment and be sent to described terminal, described parameter comprises at least one in described scaling, described noise figure and described quantized value.
6. frame processing method according to claim 1, it is characterised in that, described subtract each other to obtain the first residual frame sequence by described subsidiary frame sequence and described slotting frame sequence, comprising:
After the sequence belonging to the designated area of described frame sequence in described subsidiary frame sequence and the sequence that belongs to described designated area in described slotting frame sequence being subtracted each other, the residual frame sequence of gained is as the first residual frame sequence.
7. frame processing method according to claim 1, it is characterised in that, described subtract each other to obtain the first residual frame sequence by described subsidiary frame sequence and described slotting frame sequence, comprising:
After the sequence belonging to the designated area of described frame sequence in described subsidiary frame sequence and the sequence that belongs to described designated area in described slotting frame sequence being subtracted each other, the residual frame sequence of gained carries out pre-treatment, will carry out pretreated described residual frame sequence as the first residual frame sequence.
8. frame processing method according to claim 6 or 7, it is characterised in that, described method also comprises:
The information of described designated area is merged in described transmission code stream and is sent to described terminal.
9. frame processing method according to the arbitrary item of claim 1-8, it is characterised in that, the slotting frame sequence of the described main frame sequence of described acquisition comprises:
Carrying out changing in frame per second to obtain described slotting frame sequence and motion vector MV to described main frame sequence by transfer algorithm in frame per second, the described main frame sequence of reconstruct is carried out changing in frame per second by described MV for described terminal;
Described attached code stream also comprises described MV.
10. a frame processing method, it is characterised in that, comprising:
Receive transmission code stream, and described transmission code stream is separated into main bit stream and attached code stream;
Described main bit stream and described attached code stream decode to obtain the main frame sequence of reconstruct and the first residual frame sequence of reconstruct respectively;
Obtain the slotting frame sequence of the main frame sequence of described reconstruct;
The subsidiary frame sequence of the first residual frame sequence according to described reconstruct and the reconstruct of described slotting frame retrieval;
The subsidiary frame sequence of the main frame sequence of described reconstruct and described reconstruct is carried out the frame sequence merging to form reconstruct.
11. frame processing methods according to claim 10, it is characterised in that, the subsidiary frame sequence of described the first residual frame sequence according to described reconstruct and the reconstruct of described slotting frame retrieval, comprising:
Described slotting frame sequence is added, with the first residual frame sequence of described reconstruct, the subsidiary frame sequence obtaining described reconstruct.
12. frame processing methods according to claim 10, it is characterised in that, the subsidiary frame sequence of described the first residual frame sequence according to described reconstruct and the reconstruct of described slotting frame retrieval, comprising:
First residual frame sequence of described reconstruct is carried out pre-treatment;
Described slotting frame sequence is added, with the first residual frame sequence carrying out pretreated described reconstruct, the subsidiary frame sequence obtaining described reconstruct.
13. frame processing methods according to claim 12, it is characterised in that, described transmission code stream also comprises scaling, noise figure, and at least one in quantized value,
Described the first residual frame sequence to described reconstruct carries out one or more in the following four kinds of modes of pre-treatment:
By the chromatic(ity)component zero setting of each residual frame in the first residual frame sequence of described reconstruct;
According to default scaling, the resolving power of the first residual frame of each reconstruct in the first residual frame sequence of described reconstruct is amplified;
According to default noise figure, the pixel of the first residual frame of each reconstruct in the first residual frame sequence of described reconstruct is carried out anti-denoising; And
According to default quantized value, the pixel of the first residual frame of each reconstruct in the first residual frame sequence of described reconstruct is carried out inverse quantization.
14. frame processing methods according to claim 10, it is characterised in that, described transmission code stream also comprises the information of designated area,
Described described slotting frame sequence is added, with the first residual frame sequence of described reconstruct, the subsidiary frame sequence obtaining described reconstruct, comprising:
Information according to described designated area determines to belong in described slotting frame sequence the sequence of described designated area;
First residual frame sequence of the sequence and described reconstruct that belong to described designated area in described slotting frame sequence is added the subsidiary frame sequence obtaining described reconstruct.
15. frame processing methods according to claim 10, it is characterised in that, described transmission code stream also comprises the information of designated area,
The subsidiary frame sequence of described the first residual frame sequence according to described reconstruct and the reconstruct of described slotting frame retrieval, comprising:
First residual frame sequence of described reconstruct is carried out pre-treatment;
Information according to described designated area determines to belong in described slotting frame sequence the sequence of described designated area;
The sequence belonging to described designated area in described slotting frame sequence is added, with the first residual frame sequence carrying out pretreated described reconstruct, the subsidiary frame sequence obtaining described reconstruct.
16. frame processing methods according to the arbitrary item of claim 10-15, it is characterised in that, described attached code stream also comprises motion vector MV,
The slotting frame sequence of the described main frame sequence to obtaining described reconstruct, comprising:
According to described MV, carry out frame per second being changed obtain described slotting frame sequence to the main frame sequence of described reconstruct by transfer algorithm in frame per second.
17. 1 kinds of Video service devices, it is characterised in that, comprising:
Separating unit, for being separated into main frame sequence and subsidiary frame sequence by frame sequence;
Acquiring unit, for obtaining the slotting frame sequence of the described main frame sequence of described separating unit separation;
Described acquiring unit, also for subtracting each other to obtain the first residual frame sequence by described subsidiary frame sequence and described slotting frame sequence;
Merge cells, for merging to form transmission code stream by the main bit stream formed after described main frame sequence being encoded and the attached code stream that formed after described first residual frame sequence being encoded;
Send unit, for described transmission code stream is sent to terminal, so that described terminal recovers described frame sequence according to described transmission code stream.
18. Video service devices according to claim 17, it is characterised in that,
Described acquiring unit, specifically for described subsidiary frame sequence and described slotting frame sequence are subtracted each other rear gained residual frame sequence directly as the first residual frame sequence.
19. Video service devices according to claim 17, it is characterised in that,
Described acquiring unit, specifically for carrying out pre-treatment by the residual frame sequence that described subsidiary frame sequence and described slotting frame sequence subtract each other rear gained, using pretreated described residual frame sequence as the first residual frame sequence.
20. Video service devices according to claim 19, it is characterised in that,
Described merge cells, also for the parameter adopted when described residual frame sequence is carried out pre-treatment being merged in described transmission code stream.
21. Video service devices according to claim 17, it is characterised in that,
Described acquiring unit, after subtracting each other specifically for the sequence of the designated area using belonging to described frame sequence in described subsidiary frame sequence and the sequence that belongs to described designated area in described slotting frame sequence, the residual frame sequence of gained is as the first residual frame sequence.
22. Video service devices according to claim 17, it is characterised in that,
Described acquiring unit, after subtracting each other specifically for the sequence of the designated area by belonging to described frame sequence in described subsidiary frame sequence and the sequence that belongs to described designated area in described slotting frame sequence, the residual frame sequence of gained carries out pre-treatment, will carry out pretreated described residual frame sequence as the first residual frame sequence.
The 23. Video service devices according to claim 21 or 22, it is characterised in that,
Described merge cells, also for the information of described designated area being merged in described transmission code stream.
The 24. Video service devices according to the arbitrary item of claim 17-23, it is characterised in that,
Described acquiring unit, specifically for carrying out changing in frame per second to obtain described slotting frame sequence and motion vector MV to described main frame sequence by transfer algorithm in frame per second, the described main frame sequence of reconstruct is carried out changing in frame per second by described MV for described terminal;
Described attached code stream also comprises described MV.
25. 1 kinds of terminals, it is characterised in that, comprising:
Receive unit, for receiving transmission code stream, and described transmission code stream is separated into main bit stream and attached code stream;
Decoding unit, for decoding to obtain the main frame sequence of reconstruct and the first residual frame sequence of reconstruct respectively to the described main bit stream and described attached code stream that receive unit separation;
Acquiring unit, for obtaining the slotting frame sequence of the main frame sequence of the described reconstruct that decoding unit obtains;
Described acquiring unit, also for the subsidiary frame sequence of the first residual frame sequence according to described reconstruct and the reconstruct of described slotting frame retrieval;
Merge cells, the subsidiary frame sequence for the described reconstruct main frame sequence of the described reconstruct of decoding unit acquisition and described acquiring unit obtained carries out merging the frame sequence forming reconstruct.
26. terminals according to claim 25, it is characterised in that,
Described acquiring unit, specifically for being added, with the first residual frame sequence of described reconstruct, the subsidiary frame sequence obtaining described reconstruct by described slotting frame sequence.
27. terminals according to claim 25, it is characterised in that,
Described acquiring unit, specifically for carrying out pre-treatment to the first residual frame sequence of described reconstruct;
Described acquiring unit, also for described slotting frame sequence is added the subsidiary frame sequence obtaining described reconstruct with the first residual frame sequence carrying out pretreated described reconstruct.
28. terminals according to claim 25, it is characterised in that, the described transmission code stream that described reception unit receives also comprises the information of designated area,
Described acquiring unit, specifically for determining to belong in described slotting frame sequence the sequence of described designated area according to the information of described designated area;
Described acquiring unit, also for the first residual frame sequence of the sequence and described reconstruct that belong to described designated area in described slotting frame sequence is added the subsidiary frame sequence obtaining described reconstruct.
29. terminals according to claim 25, it is characterised in that, the described transmission code stream that described reception unit receives also comprises the information of designated area,
Described acquiring unit, specifically for carrying out pre-treatment to the first residual frame sequence of described reconstruct;
Described acquiring unit, also for determining to belong in described slotting frame sequence the sequence of described designated area according to the information of described designated area;
Described acquiring unit, also for the sequence belonging to described designated area in described slotting frame sequence is added the subsidiary frame sequence obtaining described reconstruct with the first residual frame sequence carrying out pretreated described reconstruct.
30. terminals according to the arbitrary item of claim 25-29, it is characterised in that, the described attached code stream of described reception unit separation also comprises motion vector MV,
Described acquiring unit, specifically for according to described MV, carrying out changing in frame per second to obtain described slotting frame sequence to the main frame sequence of described reconstruct by transfer algorithm in frame per second.
CN201511024885.XA 2015-12-29 2015-12-29 Frame processing method and device Pending CN105657541A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201511024885.XA CN105657541A (en) 2015-12-29 2015-12-29 Frame processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201511024885.XA CN105657541A (en) 2015-12-29 2015-12-29 Frame processing method and device

Publications (1)

Publication Number Publication Date
CN105657541A true CN105657541A (en) 2016-06-08

Family

ID=56490803

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201511024885.XA Pending CN105657541A (en) 2015-12-29 2015-12-29 Frame processing method and device

Country Status (1)

Country Link
CN (1) CN105657541A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107948628A (en) * 2016-10-12 2018-04-20 阿里巴巴集团控股有限公司 A kind of coding, coding/decoding method and the device of multidimensional video data
CN109891891A (en) * 2016-11-08 2019-06-14 Ati科技无限责任公司 It is converted using the video frame rate of streaming metadata
WO2019192490A1 (en) * 2018-04-02 2019-10-10 Huawei Technologies Co., Ltd. Adaptive quantization in video coding

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1195994A2 (en) * 2000-09-29 2002-04-10 Robert Bosch Gmbh Image sequence coding and decoding method and apparatus
CN1484452A (en) * 2003-08-07 2004-03-24 西安电子科技大学 enhanced type fineness extensible video coding structure
CN1964460A (en) * 2005-11-07 2007-05-16 索尼株式会社 Recording and playback apparatus and recording and playback method, recording apparatus and recording method, playback apparatus and playback method, and program
CN101309356A (en) * 2007-05-16 2008-11-19 奇景光电股份有限公司 Apparatus and method for frame rate up conversion
CN101360238A (en) * 2007-07-13 2009-02-04 索尼株式会社 Image processing device, image processing method and program
CN101621690A (en) * 2009-07-24 2010-01-06 北京交通大学 Two-description video coding method based on Wyner-Ziv principle
CN101938658A (en) * 2009-07-02 2011-01-05 承景科技股份有限公司 Picture rate conversion device and method
CN102111613A (en) * 2009-12-28 2011-06-29 中国移动通信集团公司 Image processing method and device
CN102630008A (en) * 2011-09-29 2012-08-08 北京京东方光电科技有限公司 Method and terminal for wireless video transmission
CN102630012A (en) * 2012-03-30 2012-08-08 北京交通大学 Coding and decoding method, device and system based on multiple description videos
JP2012175526A (en) * 2011-02-23 2012-09-10 Nippon Telegr & Teleph Corp <Ntt> Decoding device and decoding method
CN102665061A (en) * 2012-04-27 2012-09-12 中山大学 Motion vector processing-based frame rate up-conversion method and device
CN102685438A (en) * 2012-05-08 2012-09-19 清华大学 Up-conversion method of video frame rate based on time-domain evolution
CN103297782A (en) * 2013-06-08 2013-09-11 河海大学常州校区 Area-partition-based reconstruction method in distributed video compression sensing (CS) system
CN103501441A (en) * 2013-09-11 2014-01-08 北京交通大学长三角研究院 Multiple-description video coding method based on human visual system

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1195994A2 (en) * 2000-09-29 2002-04-10 Robert Bosch Gmbh Image sequence coding and decoding method and apparatus
CN1484452A (en) * 2003-08-07 2004-03-24 西安电子科技大学 enhanced type fineness extensible video coding structure
CN1964460A (en) * 2005-11-07 2007-05-16 索尼株式会社 Recording and playback apparatus and recording and playback method, recording apparatus and recording method, playback apparatus and playback method, and program
CN101309356A (en) * 2007-05-16 2008-11-19 奇景光电股份有限公司 Apparatus and method for frame rate up conversion
CN101360238A (en) * 2007-07-13 2009-02-04 索尼株式会社 Image processing device, image processing method and program
CN101938658A (en) * 2009-07-02 2011-01-05 承景科技股份有限公司 Picture rate conversion device and method
CN101621690A (en) * 2009-07-24 2010-01-06 北京交通大学 Two-description video coding method based on Wyner-Ziv principle
CN102111613A (en) * 2009-12-28 2011-06-29 中国移动通信集团公司 Image processing method and device
JP2012175526A (en) * 2011-02-23 2012-09-10 Nippon Telegr & Teleph Corp <Ntt> Decoding device and decoding method
CN102630008A (en) * 2011-09-29 2012-08-08 北京京东方光电科技有限公司 Method and terminal for wireless video transmission
CN102630012A (en) * 2012-03-30 2012-08-08 北京交通大学 Coding and decoding method, device and system based on multiple description videos
CN102665061A (en) * 2012-04-27 2012-09-12 中山大学 Motion vector processing-based frame rate up-conversion method and device
CN102685438A (en) * 2012-05-08 2012-09-19 清华大学 Up-conversion method of video frame rate based on time-domain evolution
CN103297782A (en) * 2013-06-08 2013-09-11 河海大学常州校区 Area-partition-based reconstruction method in distributed video compression sensing (CS) system
CN103501441A (en) * 2013-09-11 2014-01-08 北京交通大学长三角研究院 Multiple-description video coding method based on human visual system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
石晨阳: "视频帧率转换专利技术综述", 《数学技术与应用》 *
黄昆仑: "视频帧率上转换技术综述", 《技术分析》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107948628A (en) * 2016-10-12 2018-04-20 阿里巴巴集团控股有限公司 A kind of coding, coding/decoding method and the device of multidimensional video data
CN107948628B (en) * 2016-10-12 2019-09-03 阿里巴巴集团控股有限公司 A kind of coding, coding/decoding method and the device of multidimensional video data
CN109891891A (en) * 2016-11-08 2019-06-14 Ati科技无限责任公司 It is converted using the video frame rate of streaming metadata
WO2019192490A1 (en) * 2018-04-02 2019-10-10 Huawei Technologies Co., Ltd. Adaptive quantization in video coding

Similar Documents

Publication Publication Date Title
EP2201784B1 (en) Method and device for processing a depth-map
CN107257485B (en) Decoder, encoder, decoding method, and encoding method
US8446958B2 (en) Image coding apparatus, image decoding apparatus, image coding method, and image decoding method
KR100990565B1 (en) Systems and methods for processing multiple projections of video data in a single video file
JP6095067B2 (en) Stereoscopic video encoding apparatus, stereoscopic video decoding apparatus, stereoscopic video encoding method, stereoscopic video decoding method, stereoscopic video encoding program, and stereoscopic video decoding program
JP5735181B2 (en) Dual layer frame compatible full resolution stereoscopic 3D video delivery
CN105828183A (en) Video frame processing method, video processing chip and motion estimate and motion compensation MEMC chip
EP2942953A1 (en) Stereoscopic video encoding device, stereoscopic video decoding device, stereoscopic video encoding method, stereoscopic video decoding method, stereoscopic video encoding program, and stereoscopic video decoding program
US9473788B2 (en) Frame-compatible full resolution stereoscopic 3D compression and decompression
TW200539705A (en) Image transmitting device, image receiving device and image transceiving system
US20100013988A1 (en) Method and apparatus for transmitting and using picture descriptive information in a frame rate conversion processor
TWI539790B (en) Apparatus, method and software product for generating and rebuilding a video stream
CN105657541A (en) Frame processing method and device
US20110122227A1 (en) 3d image display apparatus and display method
WO2014088446A1 (en) Recovering motion vectors from lost spatial scalability layers
WO2011039927A1 (en) Three-dimensional video processing device and three-dimensional video processing method
JP2012522285A (en) System and format for encoding data and 3D rendering
CN112136330A (en) Video decoder chipset
US8330799B2 (en) Image output apparatus and image output method
US20120050465A1 (en) Image processing apparatus and method using 3D image format
WO2013021023A1 (en) View synthesis compliant signal codec
US11979606B2 (en) Conditional recolor for video based point cloud coding
EP3035688B1 (en) Encoding and decoding of 3d hdr images using a tapestry representation
Didier et al. The Use of a Dense Disparity Map to Enhance Quality of Experience in Stereoscopic 3D Content
US20050259750A1 (en) Method and encoder for encoding a digital video signal

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20160608