Summary of the invention
The purpose of the embodiment of the invention is to provide a kind of image interpolation method, is intended to solve the hardware cost big and chip occupying area big problem of existing image interpolation method in realization B frame and the computing of S frame interpolation.
The embodiment of the invention is achieved in that a kind of image interpolation method, and described interpolation method may further comprise the steps:
Read forward reference frame data and back to reference frame data;
To carry out transform vector to the corresponding motion vector of reference frame data with described forward reference frame data and described back respectively according to the anglec of rotation handles;
Motion vector according to forward reference frame data correspondence carries out the result that transform vector is handled simultaneously, the forward reference frame data are carried out the forward interpolation computing, with the result who carries out the transform vector processing to the motion vector of reference frame data correspondence according to the back, the backward interpolation computing is carried out to reference frame data in the back.
Another purpose of the embodiment of the invention is to provide a kind of mobile multimedia processor, and described mobile multimedia processor comprises:
The first interpolation arithmetic unit is used for reading the row interpolation computing of going forward side by side of forward reference frame data from the system reference buffer area;
The second interpolation arithmetic unit is used for reading the back to the reference frame data row interpolation computing of going forward side by side from the system reference buffer area;
The first motion vector converter unit, be used for will be corresponding according to the image anglec of rotation with the forward reference frame data motion vector carry out conversion process, and export the conversion process result to the described first interpolation arithmetic unit; And
The second motion vector converter unit is used for will carrying out conversion process to the corresponding motion vector of reference frame data with the back according to the image anglec of rotation, and exports the conversion process result to the described second interpolation arithmetic unit;
First timing sequencer is used to control the described first interpolation arithmetic unit and carries out reading of data;
Second timing sequencer is used to control the described second interpolation arithmetic unit and carries out reading of data; And
The interpolation arithmetic trigger element, be used for simultaneously sending the interpolation arithmetic triggering signal, start simultaneously that the described first interpolation arithmetic unit carries out the forward interpolation computing and the described second interpolation arithmetic unit carries out the backward interpolation computing to described first interpolation arithmetic unit and the described second interpolation arithmetic unit.
Another purpose of the embodiment of the invention is to provide a kind of employing multimedia play terminal of mobile multimedia processor as mentioned above.
In the embodiment of the invention, from the system reference buffer area, read forward reference frame data and back respectively to reference frame data, according to rotation respectively will with the forward reference frame data that read and read afterwards carry out transform vector to the corresponding motion vector of reference frame data and handle, and forward interpolation computing and backward interpolation computing are carried out in startup simultaneously, thereby saved the metadata cache of depositing the forward prediction value, than before the interpolation method of inter prediction, the embodiment of the invention is supported the interpolation arithmetic of B frame and S frame, the speed that can effectively reduce area and effectively improve interpolation.
Embodiment
In order to make purpose of the present invention, technical scheme and advantage clearer,, the present invention is further elaborated below in conjunction with drawings and Examples.Should be appreciated that specific embodiment described herein only in order to explanation the present invention, and be not used in qualification the present invention.
In the embodiment of the invention, from the system reference buffer area, read forward reference frame data and back respectively to reference frame data, and forward interpolation computing and backward interpolation computing are carried out in startup simultaneously, thereby saved the metadata cache of depositing the forward prediction value, reduced cost, reduced interpolation arithmetic and partly take area of chip.
Fig. 1 shows the realization flow of the image interpolation method that the embodiment of the invention provides, and details are as follows:
In step S101, read forward reference frame data and back to reference frame data;
In step S102, carry out the forward interpolation computing and carry out the backward interpolation computing to reference frame data according to described forward reference frame data simultaneously according to described back;
In step S103, the back forecast value that forward prediction value that the forward interpolation computing is obtained and backward interpolation computing obtain computes weighted, and is produced final pixel value mutually with the system residual values again.
The image interpolation method that the embodiment of the invention provides also can be supported spinfunction, before carrying out interpolation arithmetic, to carry out transform vector to the corresponding motion vector of reference frame data with described forward reference frame data and described back according to the anglec of rotation earlier and handle, concrete processing procedure is as shown in the table:
The angle of rotation |
Directions X |
The Y direction |
0 |
Mv_dx |
Mv_dy |
90 |
2-Mv_dy |
Mv_dx |
180 |
2-Mv_dx |
2-Mv_dy |
270 |
Mv_dy |
2-Mv_dx |
Wherein, Mv_dx represents to rotate the motion vector of 0 fraction pixel on directions X when spending, and Mv_dy represents to rotate the motion vector of 0 fraction pixel on the Y direction when spending.The first interpolation arithmetic unit and the second interpolation arithmetic unit carry out interpolation arithmetic to forward reference frame data and back to reference frame data according to the conversion process result of motion vector respectively respectively.
Fig. 2 shows the basic sequential of the first interpolation arithmetic unit and the second interpolation arithmetic unit, this sequential is at having only forward interpolation in B frame interpolation and the B frame or having only the situation of backward interpolation, with reference to Fig. 2, data block with 8 * 8 is an example, when clock signal Read_pre1 saltus step is high level, the first interpolation arithmetic unit or the second interpolation arithmetic unit read earlier data line from buffer area, when clock signal Read_pre2 saltus step is high level, the first interpolation arithmetic unit or the second interpolation arithmetic unit read the next line data again from buffer area, the first interpolation arithmetic unit or the second interpolation arithmetic unit begin interpolation arithmetic when interpolation arithmetic triggering signal intp saltus step is high level then, interpolation goes out 8 pixels of delegation needs 8 clock cycle, start by clock signal Read_pre2 in the 8th clock cycle of interpolation simultaneously and begin to read the next line data, and then interpolation next line data, from sequential as can be seen after this first interpolating unit or second interpolating unit be in the process of interpolation always, finish until one 8 * 8 data block interpolation.
Fig. 3 shows the sequential of the two-way interpolation macro block of B frame, still be example with 8 * 8 data block, as can be seen, when doing two-way interpolation, read two required line data of forward interpolation earlier, and then read two required line data of backward interpolation, carry out sagittal data interpolating then simultaneously, interpolation process also is to need 8 clock cycle, the 8th clock cycle in interpolation starts clock signal Read_pre1 and Read_pre2, read the next line data and carry out the interpolation process of next line data, until this end of data block.Wherein reference data deposit form as shown in Figure 4, be example with YUV color form, what deposited address 0~18 is four pieces of the luminance Y component of forward direction reference data, what deposited address 18~28 is colourity U component, what deposited address 28~38 is colourity V component; And the back luminance Y component to reference frame that deposit address 38~56, what deposited address 56~66 is back to the colourity U of reference frame component, what deposited address 66~76 is that the back is to the colourity V of reference frame component.
Fig. 5 shows that the embodiment of the invention provides is used to realize the structural principle of the device of above-mentioned image interpolation method for convenience of description, only show the part relevant with the embodiment of the invention, and this device can be integrated in the mobile multimedia processor.
At first, first timing sequencer, 502 controls, the first interpolation arithmetic unit 501 reads the forward reference frame data from system reference buffer area 508, second timing sequencer, 504 controls, the second interpolation arithmetic unit 503 reads the back to reference frame data from reference buffer 508 then, interpolation arithmetic trigger element 505 sends the interpolation arithmetic triggering signal to the first interpolation arithmetic unit 501 and the second interpolation arithmetic unit 503 simultaneously, start simultaneously that the first interpolation arithmetic unit 501 carries out the forward interpolation computing and the second interpolation arithmetic unit 503 carries out the backward interpolation computing, computing finishes and obtains forward prediction value and back forecast value respectively, ranking operation unit 509 computes weighted forward prediction value and back forecast value, result and the last pixel value of the final generation of system's residual values addition that pixel value creating unit 510 obtains 509 computings of ranking operation unit.The basic sequential of wherein controlling the first interpolation arithmetic unit 501 and the second interpolation arithmetic unit 503 is as indicated above, repeats no more herein.
The mobile multimedia processor that the embodiment of the invention provides can also be supported spinfunction, as shown in Figure 5, the first motion vector converter unit 506 according to the image anglec of rotation will be corresponding with the forward reference frame data motion vector carry out conversion process, and the result after will handling exports the first interpolation arithmetic unit 501 to, the first interpolation arithmetic the unit 501 and then forward reference frame data are carried out interpolation arithmetic according to the motion vector transformation results of the first motion vector converter unit 506, in like manner, the second motion vector converter unit 507 will carry out conversion process to the corresponding motion vector of reference frame data with the back according to the image anglec of rotation, and the result after will handling exports the second interpolation arithmetic unit 503 to, the second interpolation arithmetic unit 503 and then interpolation arithmetic is carried out to reference frame data in the back according to the motion vector transformation results of the second motion vector converter unit 507, it is as indicated above that wherein the first motion vector converter unit 506 and the second motion vector converter unit 507 advance the principle of transform vector, repeats no more herein.
The mobile multimedia processor that the embodiment of the invention provides is also supported half-pix interpolation and GMC interpolative mode (being the S frame pattern).
Fig. 6 shows the interpolation principle for half-pix, and wherein A, B, C, D are whole pixel position, and a, b, c are half-pix point position, and be as follows for the interpolation calculation of half-pix point:
a=(A+B+1-rounding)/2
b=(A+C+1-rounding)/2
c=(A+B+C+D+2-rounding)/4
Wherein rounding is the correction value that rounds up of interpolation calculation, and this device can be supported spinfunction equally when carrying out the half-pix interpolation, repeat no more.
For the GMC interpolative mode, use the realization image interpolation method that the embodiment of the invention provides device carry out the GMC interpolation arithmetic time at first the GMC interpolation formula is carried out abbreviation, the interpolation parameter of representing directions X and Y direction with ri and rj respectively, represent that with f0 (high 16 is the value of a pixel for the value of two pixels, low 16 is the value of another one pixel), wherein ri is relevant with the position of reference point with rj, is example explanation abbreviation process with ri * f0, ri=(16-i) * 2
16+ i, f0=x1 * 2
16+ x0, then
Ri * f0=((16-i) * 2
16+ i) * (x1 * 2
16+ x0), obtain after the expansion
ri×f0=x0×i+((16-i)×x0+x1×i)×2
16+(16-i)×x1×2
32
With The above results and 0x0fff0000 with, then
Ri * f0=((16-i) * x0+x1 * i), wherein the scope of x0 and x1 is 0~255, the scope of i is 0~16,0x0fff0000 represents to take out 16 to 24 data wherein, as can be seen ri * f0 abbreviation be the multiply-add operation of (16-i) * x0 and x1 * i, and then can use first arithmetic element and second arithmetic element in the mobile multimedia processor that the embodiment of the invention provides to carry out this multiply-add operation of two respectively, thereby the quantity that finally participates in the multiplier of GMC interpolation arithmetic reduces to some extent, partly takies area of chip at the minimizing interpolation arithmetic and has also reduced cost simultaneously.
The mobile multimedia processor that the embodiment of the invention provides is provided with modification and can also be used for other video algorithm by parameter.
The embodiment of the invention also provides a kind of multimedia play terminal, and this multimedia play terminal comprises a mobile multimedia processor, and this mobile multimedia processor adopts above-mentioned mobile multimedia processor, specifically repeats no more.
In the embodiment of the invention, from the system reference buffer area, read forward reference frame data and back respectively to reference frame data, and forward interpolation computing and backward interpolation computing are carried out in startup simultaneously, thereby saved the metadata cache of depositing the forward prediction value, than before the interpolation method of inter prediction, the speed that the embodiment of the invention can effectively reduce area and effectively improve interpolation for the interpolation method of B frame and S frame.Embodiment of the invention another one advantage is can compatible a lot of video standard, can effectively save R﹠D costs.
The above only is preferred embodiment of the present invention, not in order to restriction the present invention, all any modifications of being done within the spirit and principles in the present invention, is equal to and replaces and improvement etc., all should be included within protection scope of the present invention.