Summary of the invention
Technical problem to be solved by this invention is to provide a kind of loop filter based on multistage parallel pipeline mode, utilize the multistage parallel pipeline structure to finish the loop filtering of each macroblock boundaries of video image, guarantee the real-time of image loop filtering operation in the image coding and decoding process, reduce the pressure of access external memory, and controlled the complexity of whole hardware configuration, thereby helped design and realization.
Technical problem to be solved by this invention is achieved by the following technical solution:
A kind of loop filter based on multistage parallel pipeline mode, it comprises the three class pipeline module: macroblock boundaries computing module, vertical/horizontal filtration module and output module; The three class pipeline module is moved simultaneously by the flowing water order;
Described macroblock boundaries computing module comprises filtering boundary strength calculator, filtering boundary threshold calculations device and first in first out data buffer; Described filtering boundary strength calculator and filtering boundary threshold calculations device extract the data boundary that needs filtered macroblock from external memory storage respectively, calculate the boundary intensity and the threshold value of each piece in the whole macro block, deposit the first in first out data buffer in; Described first in first out data buffer is input to the vertical/horizontal filtration module by the order of first in first out with the boundary intensity and the threshold value of macro block filtering;
Described vertical/horizontal filtration module comprises vertical/horizontal filter, arrangement machine, intermediate data buffer, selector and loop filtering controller; Described selector connects external memory storage, and under the control of loop filtering controller, described selector extracts the macroblock boundaries data that need filtering in the external memory storage, deposits the intermediate data buffer in; Described intermediate data buffer is connected with arrangement machine, and this arrangement machine also is input to the vertical/horizontal filter with the macroblock boundaries data successively by the ranks sequence arrangement; The raw column data that described vertical/horizontal filter transmits arrangement machine under the control of loop filtering controller is carried out the vertical/horizontal boundary filtering, with needing the data of filtering once more to write back to the intermediate data buffer among the filtered result, the data of finishing filtering are write output module;
Described output module comprises the dateout buffer and writes external memory interface.The data of finishing filtering that the storage of described dateout buffer is transmitted from the vertical/horizontal filter, and with transfer of data to writing external memory interface; The described external memory interface of writing will be finished the data of filtering and write external memory storage under the control of loop filtering controller.
Whole device is divided into three class pipeline and finishes loop filtering operation: first order streamline calculates boundary intensity and boundary threshold that will each filtering boundary of filtered macroblock; Second level streamline will be to carrying out vertical filtering and horizontal filtering by each bar filtering boundary of filtered macroblock; Third level streamline outputs to external memory storage to the macro block data of finishing filtering.By the three class pipeline operation, each process of loop filtering operation is assigned to different pipeline step, promptly reduced the complexity of every level production line operation, improved the concurrency of whole loop filtering operation again.
In whole loop filter, horizontal boundary and vertical boundary filtering also are designed to the multi-stage pipeline mode in the pile line operation of the second level, utilize two parallel pipeline organizations like this, can satisfy the requirement of in the high definition video real time codec process loop filtering being operated fully.
Described vertical/horizontal filter comprises filtering condition decision device module and filtering calculator modules;
Described filtering condition decision device module comprises filtering condition decision device and data buffer, the boundary intensity and the threshold value of the raw column data that described filtering condition decision device reception arrangement machine transmits, the macro block filtering that the first in first out data buffer transmits, calculate the boundary filtering judgment condition, decision will be carried out the edge pixel point of filtering operation, and edge pixel point is input to down level production line by data buffer will carry out filtering;
Described filtering calculator modules comprises the filtering calculator, and it carries out filtering calculating to each picture element, and with the picture element output after the filtering calculating.
Described filtering condition decision device is divided into two-stage or the above pipeline module of two-stage again, and the above pipeline module of two-stage or two-stage moves simultaneously by the flowing water order.
Described filtering calculator is divided into two-stage or the above pipeline module of two-stage again, and the above pipeline module of two-stage or two-stage moves simultaneously by the flowing water order.
The present invention finishes the filtering to macroblock boundaries through the following steps:
Step 1, under the control of loop filtering controller, selector extracts the macroblock boundaries data that need filtering in the external memory storage, deposit the intermediate data buffer in, by arrangement machine the macroblock boundaries data are pressed the ranks sequence arrangement again, and be input to the vertical/horizontal filter successively; Filtering boundary strength calculator and filtering boundary threshold calculations device extract the data boundary that needs filtered macroblock from external memory storage respectively simultaneously, calculate the boundary intensity and the threshold value of each piece in the whole macro block, deposit the first in first out data buffer in, the first in first out data buffer is input to the vertical/horizontal filter by the order of first in first out with block boundary intensity and threshold value.
The block boundary filtering strength of step 2, filter controller use upper level operational computations and boundary threshold are to whole macroblock boundaries filtering, and the loop filtering controller is controlled the vertical/horizontal filtering of whole macroblock boundaries according to piece level production line mode; With needing the data of filtering once more to write back to the intermediate data buffer among the filtered result, the data of finishing filtering are write output module.
Step 3, finish to the filtering operation of a blocks of data simultaneously, the row/columns that from middle data buffer the next one is needed the block boundary of filtering is sent into the vertical/horizontal filter successively according to above method and is carried out filtering operation according to reading;
Step 4, the operation of execution in step two and step 3 repeatedly are up to the filtration of finishing current whole macroblock boundaries data;
Step 5, simultaneously to each blocks of data filtering of current macro, the final data of vertical/horizontal filter output writes the dateout buffer memory, waits external memory interface to be written to write the operation of external memory storage.
The present invention utilizes the multistage parallel pipeline structure to finish the loop filtering of each macroblock boundaries of video image, the reference frame storing process in the middle of the codec design and the actual interface problem of loop filtering have been considered simultaneously, guaranteed the real-time of image loop filtering operation in the image coding and decoding process, reduced the pressure of access external memory, and controlled the complexity of whole hardware configuration, thereby helped design and realization.
Embodiment
Below in conjunction with the drawings and specific embodiments technical scheme of the present invention is further specified:
First embodiment:
A kind of based on multistage parallel pipeline mode loop filter and realize the step of the loop filtering algorithm of AVS standard:
As shown in Figure 3, a kind of loop filter based on multistage parallel pipeline mode, it comprises the three class pipeline module: macroblock boundaries computing module, vertical/horizontal filtration module and output module; The three class pipeline module is moved simultaneously by the flowing water order;
Described macroblock boundaries computing module comprises filtering boundary strength calculator, filtering boundary threshold calculations device and FIFO (first in first out) data buffer; Described filtering boundary strength calculator and filtering boundary threshold calculations device extract the data boundary that needs filtered macroblock from external memory storage respectively, calculate the boundary intensity and the threshold value of each piece in the whole macro block, deposit the data fifo buffer in; Described data fifo buffer is input to the vertical/horizontal filtration module by the order of first in first out with the boundary intensity and the threshold value of macro block filtering;
Described vertical/horizontal filtration module comprises vertical/horizontal filter, arrangement machine, intermediate data buffer, selector and loop filtering controller; Described selector connects external memory storage, and under the control of loop filtering controller, described selector extracts the macroblock boundaries data that need filtering in the external memory storage, deposits the intermediate data buffer in; Described intermediate data buffer is connected with arrangement machine, and this arrangement machine also is input to the vertical/horizontal filter with the macroblock boundaries data successively by the ranks sequence arrangement; The raw column data that described vertical/horizontal filter transmits arrangement machine under the control of loop filtering controller is carried out the vertical/horizontal boundary filtering, with needing the data of filtering once more to write back to the intermediate data buffer among the filtered result, the data of finishing filtering are write output module.
Described output module comprises the dateout buffer and writes external memory interface; The data of finishing filtering that the storage of described dateout buffer is transmitted from the vertical/horizontal boundary filtering, and with transfer of data to writing external memory interface; The described external memory interface of writing will be finished the data of filtering and write external memory storage under the control of loop filtering controller.
Whole device is divided into three class pipeline and finishes loop filtering operation: first order streamline calculates boundary intensity and boundary threshold that will each bar filtering boundary of filtered macroblock; Second level streamline will be to carrying out vertical filtering and horizontal filtering by each bar filtering boundary of filtered macroblock; Third level streamline outputs to external memory storage to the macro block data of finishing filtering.By the three class pipeline operation, each process of loop filtering operation is assigned to different pipeline step, promptly reduced the complexity of every level production line operation, improved the concurrency of whole loop filtering operation again.
The present invention realizes the loop filtering algorithm of AVS standard through the following steps:
The boundary filtering strength of 8x8 blocks of data and boundary threshold in step 1, the computing macro block;
Step 2, from external memory, read in the data of the left side, top and the current block that need filtering simultaneously, these data are put into the intermediate data buffer;
After the blocks of data of step 3, current macro, boundary filtering strength and boundary threshold are ready to and left side piece treat filtering data through the intermediate data buffer, send into the data arrangement device simultaneously, send into successively in the vertical/horizontal filter according to the row order then;
By pipeline organization the vertical boundary data are carried out filtering in step 4, the vertical/horizontal filter, filtered data write back to the intermediate data buffer successively simultaneously;
Step 5, finish the vertical boundary filtering of a blocks of data of current macro after, the data of the left side macro block that can export are write the dateout buffer, the notice output interface is write the external memory operation, begins the next block boundary vertical filtering of current macro simultaneously;
Step 6, the operation of execution in step three, step 4 or step 5 repeatedly are up to the vertical filtering of finishing all block boundaries;
Step 7, the data of taking out wait horizontal filtering in top and the current macro border from middle buffer are sent into the data arrangement device;
The blocks of data process intermediate data buffer of filtering is treated in the blocks of data of step 8, current macro and top, sends into the data arrangement device simultaneously, sends into the vertical/horizontal filter successively according to the row order then;
By pipeline organization the horizontal boundary data are carried out filtering in step 9, the vertical/horizontal filter, filtered data write back to the intermediate data buffer successively simultaneously;
Step 10, finish the horizontal boundary filtering of a piece of current macro after, the data that can export in the macro block of top are write the dateout buffer, the notice output interface is write the external memory operation, begins the next block boundary horizontal filtering of current macro simultaneously;
Step 11, the operation of execution in step eight, step 9 or step 10 repeatedly are up to the horizontal filtering of finishing all block boundaries;
Step 12, according to finishing the loop filtering of each macro block in the frame with upper type and outputing to external memory.
Embodiment two:
A kind of based on multistage parallel pipeline mode loop filter and realize the step of the loop filtering algorithm of MPEG4-part10/h.264 standard:
As shown in Figure 3, a kind of loop filter based on multistage parallel pipeline mode, it comprises the three class pipeline module: macroblock boundaries computing module, vertical/horizontal filtration module and output module; The three class pipeline module is moved simultaneously by the flowing water order;
Described macroblock boundaries computing module comprises filtering boundary strength calculator, filtering boundary threshold calculations device and FIFO (first in first out) data buffer; Described filtering boundary strength calculator and filtering boundary threshold calculations device extract the data boundary that needs filtered macroblock from external memory storage respectively, calculate the boundary intensity and the threshold value of each piece in the whole macro block, deposit the data fifo buffer in; Described data fifo buffer is input to the vertical/horizontal filtration module by the order of first in first out with the boundary intensity and the threshold value of macro block filtering;
Described vertical/horizontal filtration module comprises vertical/horizontal filter, arrangement machine, intermediate data buffer, selector and loop filtering controller; Described selector connects external memory storage, and under the control of loop filtering controller, described selector extracts the macroblock boundaries data that need filtering in the external memory storage, deposits the intermediate data buffer in; Described intermediate data buffer is connected with arrangement machine, and this arrangement machine also is input to the vertical/horizontal filter with the macroblock boundaries data successively by the ranks sequence arrangement; The raw column data that described vertical/horizontal filter transmits arrangement machine under the control of loop filtering controller is carried out the vertical/horizontal boundary filtering, with needing the data of filtering once more to write back to the intermediate data buffer among the filtered result, the data of finishing filtering are write output module.
Described output module comprises the dateout buffer and writes external memory interface; The data of finishing filtering that the storage of described dateout buffer is transmitted from the vertical/horizontal boundary filtering, and with transfer of data to writing external memory interface; The described external memory interface of writing will be finished the data of filtering and write external memory storage under the control of loop filtering controller.
Whole device is divided into three class pipeline and finishes loop filtering operation: first order streamline calculates boundary intensity and boundary threshold that will each bar filtering boundary of filtered macroblock; Second level streamline will be to carrying out vertical filtering and horizontal filtering by each bar filtering boundary of filtered macroblock; Third level streamline outputs to external memory storage to the macro block data of finishing filtering.By the three class pipeline operation, each process of loop filtering operation is assigned to different pipeline step, promptly reduced the complexity of every level production line operation, improved the concurrency of whole loop filtering operation again.
As shown in Figure 4, in whole loop filter, horizontal boundary and vertical boundary filtering also are designed to the multi-stage pipeline mode in the pile line operation of the second level, utilize two parallel pipeline organizations like this, can satisfy the requirement of in the high definition video real time codec process loop filtering being operated fully.
Wherein, the vertical/horizontal filter comprises filtering condition decision device module and filtering calculator modules; Described filtering condition decision device module comprises filtering condition decision device and data buffer, the boundary intensity and the threshold value of the raw column data that described filtering condition decision device reception arrangement machine transmits, the macro block filtering that the first in first out data buffer transmits, calculate the boundary filtering judgment condition, decision will be carried out the edge pixel point of filtering operation, and edge pixel point is input to down level production line by data buffer will carry out filtering; Described filtering calculator modules comprises the filtering calculator, and it carries out filtering calculating to each picture element, and with the picture element output after the filtering calculating.
The filtering condition decision device can be divided into two-stage or the above pipeline module of two-stage, and the above pipeline module of two-stage or two-stage moves simultaneously by the flowing water order.The filtering calculator can be divided into two-stage or the above pipeline module of two-stage, and the above pipeline module of two-stage or two-stage moves simultaneously by the flowing water order.
The present invention realizes the loop filtering algorithm of MPEG4-part 10/h.264 standard through the following steps:
The boundary filtering strength of 4x4 blocks of data and boundary threshold in step 1, the computing macro block; Compare with the AVS standard, need the piece number of filtering many, but the size of the intermediate buffer that needs can reduce;
Step 2, from external memory, read in the data of the left side, top and the current block that need filtering simultaneously, these data are put into the intermediate data buffer;
After the blocks of data of step 3, current macro, boundary filtering strength and boundary threshold are ready to and left side piece treat filtering data through the intermediate data buffer, send into the data arrangement device simultaneously, send into successively in the vertical/horizontal filter according to the row order then;
By pipeline organization the vertical boundary data are carried out filtering in step 4, the vertical/horizontal filter, filtered data write back to the intermediate data buffer successively simultaneously;
Step 5, finish the vertical boundary filtering of a blocks of data of current macro after, the data of the left side macro block that can export are write the dateout buffer, the notice output interface is write the external memory operation, begins the next block boundary vertical filtering of current macro simultaneously;
Step 6, the operation of execution in step three, step 4 or step 5 repeatedly are up to the vertical filtering of finishing all block boundaries;
Step 7, the data of taking out wait horizontal filtering in top and the current macro border from middle buffer are sent into the data arrangement device;
The blocks of data process intermediate data buffer of filtering is treated in the blocks of data of step 8, current macro and top, sends into the data arrangement device simultaneously, sends into the vertical/horizontal filter successively according to the row order then;
By pipeline organization the horizontal boundary data are carried out filtering in step 9, the vertical/horizontal filter, filtered data write back to the intermediate data buffer successively simultaneously;
Step 10, finish the horizontal boundary filtering of a piece of current macro after, the data that can export in the macro block of top are write the dateout buffer, the notice output interface is write the external memory operation, begins the next block boundary horizontal filtering of current macro simultaneously;
Step 11, the operation of execution in step eight, step 9 or step 10 repeatedly are up to the horizontal filtering of finishing all block boundaries;
Step 12, according to finishing the loop filtering of each macro block in the frame with upper type and outputing to external memory.
The present invention utilizes the multistage parallel pipeline structure to finish the loop filtering of each macroblock boundaries of video image, guaranteed the real-time of image loop filtering operation in the image coding and decoding process, reduced the pressure of access external memory, and controlled the complexity of whole hardware configuration, thereby helped design and realization.
It should be noted last that, above embodiment is only unrestricted in order to technical scheme of the present invention to be described, although the present invention is had been described in detail with reference to preferred embodiment, those of ordinary skill in the art is to be understood that, can make amendment or be equal to replacement technical scheme of the present invention, and not breaking away from the spirit and scope of technical solution of the present invention, it all should be encompassed in the middle of the claim scope of the present invention.