JP4686048B2 - Pixel arithmetic unit - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a pixel arithmetic device including a filtering circuit for resizing an image.
[0002]
[Prior art]
In recent years, technological progress of digital video equipment has been remarkable, and so-called media processors that handle video compression / expansion processing, resizing, and the like have been put into practical use.
For image resizing, an FIR (Finite Impulse Response) filter is often used.
[0003]
FIG. 1 is a block diagram showing an example of a circuit that performs FIR filter processing in the prior art. This figure shows an FIR filter with 7 taps and symmetrical coefficients.
In the figure, data input in time series from the data input terminal 1001 is sequentially transferred to the delay units 1002, 1003, 1004, 1005, 1006, and 1007 in this order. When the filter coefficients are symmetric, that is, when the coefficients corresponding to the input of the data input terminal and the output of each delay unit (called a tap) are symmetric with respect to the center tap (the output of the delay device 1004), Instead of multiplying the tap data by the filter coefficient, the tap data having the same coefficient is added to each other and then multiplied by the coefficient.
[0004]
For example, the input data of the data input unit 1001 and the output data of the delay unit 1007 are added by the adder 1008, and the multiplier 1008 multiplies the addition result by the coefficient h0. The output of the delay unit 1002 and the output of the delay unit 1006 are added by the adder 1009, and the multiplier 1009 multiplies the addition result by the coefficient h1.
Output data of the multipliers 1011 to 1014 are added by an adder 1015. The output data of the adder 1015 is output from the data output terminal 1016 in time series as a filter processing result. The coefficients h0 to h3 are determined according to the image reduction ratio. For example, if the reduction ratio is 1/2, a reduced image can be obtained by thinning out the time series output data to 1/2.
[0005]
Also, the filter coefficients are selected symmetrically because the visual phase of the image is preferable because a linear phase (the phase characteristic becomes a straight line with respect to the frequency) is obtained.
[0006]
[Problems to be solved by the invention]
However, in the above conventional method, when filtering processing is performed on image data, pixel data constituting an image is sequentially input from the end due to the circuit configuration, so that one pixel data can be input in one clock. Therefore, it is necessary to increase the operating frequency in order to increase the processing speed. There is a problem that operation at a high operating frequency increases cost and power consumption.
[0007]
In addition, since the circuit differs depending on the number of taps in the conventional method, there is no degree of freedom. If a circuit is provided for each number of taps, enormous costs are required.
A first object of the present invention is to provide a pixel calculation device that can perform a filtering process that can change the number of taps and increase the processing speed without increasing the frequency.
[0008]
A second object of the present invention is to provide a pixel arithmetic device that can be used not only for filtering processing but also for MC (motion compensation) processing, and which has a reduced circuit scale.
A third object is to provide a pixel arithmetic device that can be used not only for filtering processing but also for ME (motion prediction) processing, and which has a reduced circuit scale.
[0009]
A fourth object is to provide a pixel arithmetic device that is not only used for filtering processing but also used for OSD (On Screen Display) processing in digital video equipment and has a reduced circuit scale.
[0010]
[Means for Solving the Problems]
A pixel arithmetic device that achieves the first object is a pixel arithmetic device that performs filter processing, and includes N pixel processing means, N pixel data and a supply means for supplying filter coefficients, and N pieces of pixel arithmetic means. Control means for operating the pixel processing means in parallel.
Each pixel processing means calculates using the pixel data supplied to the supplying means and the filter coefficient, then acquires pixel data from the pixel processing means adjacent to each pixel processing means, and uses the acquired pixel data And accumulate the calculation results. The control means controls the N pixel processing means so as to repeat the acquisition of the pixel data from the adjacent pixel processing means and the calculation and accumulation using the acquired pixel data for the number of taps.
[0011]
Here, the N pixel processing units form a first shifter that shifts N pixel data to the right and a second shifter that shifts N pixel data to the left. Each pixel processing means performs an operation using two pixel data shifted out from two adjacent pixel processing means.
The pixel calculation device that achieves the second object supplies pixel data of a difference image and pixel data of a reference frame as pixel data from a supply unit.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
The pixel arithmetic unit of the present invention mainly includes (a) filter processing used for enlargement / reduction of an image, (b) motion compensation (hereinafter referred to as ME) processing, (c) OSD (On Screen Display) processing, ( d) It is configured to selectively execute a motion estimation (hereinafter referred to as ME) process. (A) Regarding the filter processing, the pixel operation unit makes the number of taps variable without fixing, and processes a plurality of pixels (for example, 16 pixels) that are continuous in the horizontal direction or the vertical direction in parallel. Further, the filtering process in the vertical direction is performed in synchronization with the decompression process of the compressed moving image data.
[0013]
Hereinafter, the pixel operation unit in the embodiment of the present invention will be described in the following order.
1 Media processor configuration
1.1 Configuration of pixel operation unit
1.2 Configuration of pixel parallel processing unit
2.1 Filter processing
2.2 MC (motion compensation) processing
2.3 OSD (On Screen Display) processing
2.4 ME (motion prediction) processing
3.1 Vertical filter processing (1)
3.1.1 1/2 reduction
3.1.2 1/4 reduction
3.2 Vertical filter processing (part 2)
3.2.1 1/2 reduction
3.2.2 1/4 reduction
4 Modifications
<1 Media processor configuration>
A case will be described below where the pixel arithmetic unit in the present embodiment is incorporated in a media processor that performs media processing (compression audio video data decompression processing, audio video data compression processing, etc.). The media processor is mounted on, for example, a set top box that receives digital TV broadcasts, a television receiver, a DVD recording / playback apparatus, and the like.
[0014]
FIG. 2 is a block diagram illustrating a configuration of a media processor including a pixel operation unit. In the figure, a media processor 200 includes a dual port memory 100, a stream unit 201, an input / output buffer (hereinafter abbreviated as I / O buffer) 202, a setup processor 203, a bit stream FIFO 204, a variable length code decoding unit (VLD) 205, a variable. A long code decoding unit 205, a transfer engine (hereinafter referred to as TE) 206, a pixel operation unit A (hereinafter referred to as POUA) 207, a pixel operation unit B (hereinafter referred to as POUB) 208, a POUC 209, an audio unit 210, an IOP 211, an input / output processor (hereinafter referred to as “input processor”). IOP) 211, video buffer memory 212, video unit 213, host unit 214, RE 215, and filter unit 216.
[0015]
The dual port memory 100 includes an input / output port (hereinafter referred to as an external port) for the external memory 220, an input / output (hereinafter referred to as an internal port) for the media processor 200, and a cache memory. Among them, an access request from a component (hereinafter referred to as a master device) that reads / writes data to / from the external memory 220 is received from the internal port, and the external memory 220 is accessed according to the received access request. At that time, the dual port memory 100 caches a part of the data of the external memory 220 in the internal cache memory. The external memory 220 is a memory such as SDRAM or RDRAM, and temporarily stores compressed moving image data, compressed audio data, decoded audio data, decoded moving image data, and the like.
[0016]
The stream unit 201 inputs stream data (a so-called MPEG stream) from the outside, separates the input stream data into a video elementary stream and an audio elementary stream, and writes them into the I / O buffer 202.
The I / O buffer 202 is a buffer memory that temporarily holds a video elementary stream, an audio elementary stream, and audio data (expanded audio data). The video elementary stream and the audio elementary stream are respectively stored in the I / O buffer 202 from the stream unit 201 and further stored in the external memory 220 via the dual port memory 100 under the control of the IOP 211. Audio data is stored in the I / O buffer 202 from the external memory 220 via the dual port memory 100 under the control of the IOP 211.
[0017]
The setup processor 203 performs decoding (decompression) of the audio elementary stream and header analysis of the macro block of the video elementary stream. The audio elementary stream and the video elementary stream are transferred from the external memory 220 to the bit stream FIFO 204 via the dual port memory 100 under the control of the IOP 211. The setup processor 203 reads and decodes the audio elementary stream from the bit stream FIFO 204 and stores the decoded audio data in the setup memory 217. Audio data in the setup memory 217 is transferred to the external memory 220 via the dual port memory 100 by the IOP 211. Also, the setup processor 203 reads the video elementary stream from the bit stream FIFO 204, analyzes the macroblock header, and notifies the VLD 205 of the analysis result.
[0018]
The bit stream FIFO 204 is a FIFO memory for supplying a video elementary stream to the variable length code decoding unit 205 and an audio elementary stream to the setup processor 203. The video elementary stream and the audio elementary stream are transferred from the external memory 220 to the bit stream FIFO 204 via the dual port memory 100 under the control of the IOP 211.
[0019]
The VLD 205 decodes a variable length code included in the video elementary stream supplied from the bit stream FIFO 204. This decoding result is a DCT coefficient group in units of macroblocks.
The TE 206 performs IQ (inverse quantization) processing and IDCT (inverse DCT) processing for each macroblock on the decoding result of the VLD 205. These processing results are macroblocks. One macro block is composed of four luminance blocks (Y1 to Y4) and two color difference blocks (Cb, Cr). One block is 8 × 8 pixels. However, for a P picture and a B picture, one block is output from the TE 206 as 8 × 8 difference values. The TE 206 stores the decoding result in the external memory 220 via the dual port memory 100.
[0020]
The POUA 207 selectively executes mainly (a) filter processing, (b) MC processing, (c) OSD processing, (d) motion estimation processing, and the like.
In the filtering process (a), the POUA 207 filters 16 pixel data included in the video data (frame data) stored in the external memory 220 in parallel, and thins out or interpolates the 16 pixels after filtering. To reduce or enlarge. The data after the reduced word is stored in the external memory 220 via the dual port memory 100 under the control of the POUC 209.
[0021]
In the MC processing of (b), the POUA 207 uses the IQ and IDCT processing results (that is, the difference value of the pixel data) for the P picture and B picture stored in the external memory 220 by the TE 206 and the pixel data in the reference frame. Add in 16 parallels. The 16 sets of difference values and pixel data are input to the POUA 207 by the POUC 209 in accordance with the motion vector detected by the macroblock header analysis in the setup processor 203.
[0022]
(C) In OSD processing, the POUA 207 inputs an OSD image (still image) stored in the external memory 220 or the like via the dual port memory 100 and overwrites the display frame data in the external memory 220. Here, the OSD image means a menu image displayed according to a user's remote control operation, a time display, a channel number display, or the like.
[0023]
The ME process of (d) is a search for a highly correlated rectangular area in a reference frame for a macroblock to be encoded in uncompressed frame data, and a correlation is detected from the macroblock to be encoded. This is a process for obtaining a motion vector indicating the highest rectangular area. The POUA 207 calculates 16 differences in parallel between the pixel of the macroblock to be encoded and the pixel of the rectangular area in the search area.
[0024]
The POUB 208 has the same configuration as the POUA 207, and dynamically shares the processes (a) to (d).
The POUC 209 controls the supply of pixel data groups to the POUA 207 and the POUB 208 and the transfer of processing results to the external memory 220.
The audio unit 210 outputs audio data stored in the I / O buffer 202.
[0025]
The IOP 211 controls data input / output (data transfer) in the media processor 200. There are the following types of data transfer. The first is to transfer the stream data stored in the I / O buffer 202 to the stream buffer area in the external memory 220 via the dual port memory 100. Secondly, the video elementary stream and the audio elementary stream stored in the external memory 220 are transferred to the bit stream FIFO 204 via the dual port memory 100. Thirdly, the audio data stored in the external memory 220 is transferred to the I / O buffer 202 via the dual port memory 100.
[0026]
The video unit 213 reads pixel data for a few lines from the video data (image frame) in the external memory 220, stores it in the video buffer memory 212, and converts the pixel data for the lines 2-3 into a video signal. Then, the data is output to a display device such as a television receiver connected to the outside.
A host unit (HOST) 214 receives an instruction from an external host microcomputer, and controls start / end of MPEG decoding, MPEG encoding, OSD processing, reduction / enlargement processing, and the like according to the instruction.
[0027]
A rendering engine (RE) 215 is a master device and performs rendering processing in computer graphics. Data input / output is performed when the dedicated LSI 218 is connected to the outside.
The filter 216 performs still image data enlargement / reduction processing. Data input / output is performed when the dedicated LSI 218 is connected to the outside.
[0028]
In the above description, the case where the media processor inputs stream data from the stream unit 201 and decodes (decompresses) it has been mainly described. However, the reverse flow occurs when encoding (compressing) uncompressed video data and audio data. It becomes. At that time, POUA 207 (or POUB 208) performs ME processing, TE 206 performs DCT processing and Q (quantization) processing, and VLD 205 performs variable length coding.
<1.1 Pixel Arithmetic Unit Configuration>
FIG. 3 is a block diagram illustrating a configuration of the pixel calculation unit.
[0029]
Since the POUA 207 and the POUB 208 have the same configuration, the POUA 207 will be described here.
As shown in the figure, the POUA 207 includes a pixel parallel processing unit 21, an input buffer group 22, an output buffer group 23, an instruction memory 24, an instruction decoder 25, an instruction circuit 26, and a DDA circuit 27.
[0030]
The pixel parallel processing unit 21 includes a pixel transfer unit 17, 16 pixel processing units 1 to 16, and a pixel transfer unit 18, and targets the plurality of pixels input from the input buffer group 22 (a). Filter processing, (b) MC processing, (c) OSD processing, and (d) ME) processing are performed and output to the output buffer group 23. Each processing of (a) to (d) is completed by repeating a macroblock unit, that is, 16 pixels 16 times (for 16 lines). The activation of each process is controlled by the POUC 209. Further, the pixel transfer unit 17 holds a plurality of pixels (eight pixels in this case) on the left side (or the upper side) of the 16 pixels in the filter processing, and right shifts every clock. The pixel transfer unit 18 holds a plurality of pixels (here, 8 pixels) on the right side (or lower side) of the 16 pixels in the filter processing, and shifts out to the left for each clock.
[0031]
The input buffer group 22 holds a plurality of pixels to be processed transferred from the dual port memory 100 under the control of the POUC 209, and further holds a filter coefficient in the filter processing.
The output buffer group 23 arbitrarily changes the arrangement of the processing results (16 processing results corresponding to 16 pixels) by the pixel parallel processing unit 21 and temporarily holds them. In the filter processing, the pixel arrangement is changed and held to perform pixel thinning (during reduction) or interpolation (during enlargement).
[0032]
The instruction memory 24 stores a microprogram for filter processing (filter μP), a microprogram for MC processing (MCμP), a microprogram for OSD processing (OSDμP), and a microprogram for ME processing (MEμP). . In addition to this, the instruction memory 24 stores a microprogram for macroblock format conversion, a microprogram for converting the numerical expression of pixels, and the like. Here, the format of the macroblock is a pixel of Y, Cb, Cr blocks such as “4: 2: 0”, “4: 2: 2”, “4: 4: 4”, etc. defined in the MPEG standard. Sampling rate ratio. The numerical representation of the pixel includes a case where the pixel can be represented by 0 to 255 (general MPEG data or the like) and a case of -128 to 127 (DV camera or the like).
[0033]
The instruction decoder 25 sequentially reads and decodes the microcode in the microprogram from the instruction memory 24, and controls each part in the POUA 207 according to the decoding result.
The instruction circuit 26 receives an instruction (start address or the like) from the POUC 209 about which microprogram in the instruction memory 24 should be activated, and activates the designated microprogram.
[0034]
The DDA circuit 27 performs selection control of the filter coefficient group held in the input buffer group 22 in the filter processing.
<1.2 Configuration of Pixel Parallel Processing Unit>
4 and 5 are block diagrams showing detailed configurations of the left half and the right half of the pixel parallel processing unit.
[0035]
In FIG. 4, a pixel transfer unit 17 includes eight input ports A1701 to H1708, eight delay units A1701 to delayer H1709 that hold pixel data and delay one clock time, input port pixel data, and left delayer output. It is composed of seven selection units A1717 to G1723 for selecting one of them, and 8 pixels input in parallel from the input buffer group 22 are held in 8 delay units, and the pixels held in the 8 delay units are clock-synchronized. Functions as a right shifter that shifts to the right.
[0036]
In FIG. 5, the pixel transfer unit 18 is different from the pixel transfer unit 17 in that the shift direction is to the left.
Since the 16 pixel processing units 1 to 16 in FIGS. 4 and 5 have the same configuration, the pixel processing unit 2 will be described as a representative.
The pixel processing unit 2 includes an input port A201 to an input port C203, selection units A204 and B205, delay units A206 to D209, an adder A120, a multiplier A211, an adder B212, and an output port D213.
[0037]
The selection unit A204 selects one of the pixel data input from the input port A201 and the pixel data output from the pixel transfer unit 17 on the left side.
The selection unit A204 and the delay unit A206 also function to shift pixel data input from the right adjacent pixel processing unit 3 to the left adjacent pixel processing unit 1.
The selection unit B205 selects one of the pixel data input from the input port B202 and the pixel data shifted out from the right adjacent external memory 220.
[0038]
The selection unit B205 and the delay unit B207 also perform a function of shifting and outputting the pixel data input from the left adjacent pixel processing unit 1 to the right adjacent pixel processing unit 3.
The delay unit A206 and the delay unit B207 hold the selected pixel data in the selection unit A204 and the selection unit B205, respectively.
The delay device B207 holds the pixel data from the input port C203.
The adder A120 adds the pixel data output from the delay device A206 and the delay device B207.
[0039]
Multiplier A211 multiplies the addition result of adder A120 and the pixel data from delayer C208. The multiplier A211 is used for multiplication of pixel data and a filter coefficient in the filter processing.
The adder B212 adds the multiplication result of the multiplier A211 to the data of the delay unit D209.
[0040]
The delay unit D109 accumulates the addition result of the adder B212.
The pixel processing unit 2 executes the above (a) filter processing, (b) MC processing, (c) OSD processing, and (d) ME processing by selectively operating these components. The operation of selectively combining these components is performed by microprogram control by the instruction memory 24 and the instruction decoder 25.
[0041]
FIG. 6A is a block diagram showing a detailed configuration of the input buffer group 22.
As shown in the figure, the input buffer group 22 includes eight latches 221 that supply pixel data to the pixel transfer unit 17, 16 latch units 222 that supply pixel data to the pixel processing units 1 to 16, and pixel transfer. 8 latches 223 for supplying pixel data to the unit 18. In these, pixel data groups are transferred from the external memory 220 via the dual port memory 100 under the control of the POUC 209.
[0042]
Each latch unit 222 includes two latches that supply pixel data to the input ports A and B of the pixel processing unit, and a selection unit 224 that supplies pixel data or a filter coefficient to the input port C of the pixel processing unit.
FIG. 6B is a block diagram illustrating a detailed configuration of the selection unit 224.
As shown in the figure, the selection unit 224 includes eight latches 224a to 224h and a selector 224i that selects any one of data from the eight latches.
[0043]
The latches 224a to 224h hold the filter coefficients a0 to a7 (or a0 / 2, a1 to a7) in the filtering process. These filter coefficients are transferred from the external memory 220 to the latches 224a to 224h by the POUC 209 via the dual port memory 100.
The selector 224 i is sequentially selected from the latches 224 a to 224 h in synchronization with the clock under the control of the DDA circuit 27. In this way, the supply of the filter coefficient to the pixel processing unit is not directly controlled by the microcode, but is controlled by the hardware by the DDA circuit 27, so that the speed is increased.
[0044]
FIG. 7 is a block diagram showing the configuration of the output buffer group 23.
As shown in the figure, the output buffer group 23 includes 16 selectors 24a to 24p and 16 latches 23a to 23p.
Each of the selectors 24a to 24p receives 16 processing results of the pixel processing units 1 to 16, and selects one of them. This selection control is performed by the instruction decoder 25.
[0045]
The latches 23a to 23p hold the selection results of the selectors 24a to 24p, respectively.
For example, when the filter processing result is reduced to ½, the pixel processing units 1, 3, 5,... 15 out of the 16 processing results of the pixel processing units 1 to 16 for 16 pixels. Are selected by the eight selectors 24a to 24h and stored in the latches 23a to 23h. Further, out of the 16 processing results of the pixel processing units 1 to 16 for the next 16 pixels, the pixel processing unit The processing results of 2, 4, 6,... 16 are selected by the eight selectors 24i to 24p and stored in the latches 23i to 23p. In this way, the pixel data is thinned out and 16 pixel data reduced by 1/2 is held in the output buffer group 23 and further transferred to the external memory 220 via the dual port memory 100 under the control of the POUC 209.
<2.1 Filter processing>
Details of the filter processing in the pixel operation unit will be described.
[0046]
The POUC 209 identifies a macroblock to be filtered, transfers 32 pixel data and filter coefficients a0 / 2, a1 to a7 to the input buffer group 22 as initial values for the POUA 207 or POUB 208, and further indicates an instruction circuit. 26 is instructed to start the filtering process together with the notification of the number of taps.
FIG. 8 is a diagram illustrating initial input values of pixel data when the pixel processing unit (POUA 207) performs filter processing. In the figure, the input port column means each input port shown in FIGS. The input pixel column means pixel data supplied from the input buffer group 22 to each input port. The output port column means the output port D (adder B output) shown in FIGS. 4 and 5, and the output pixel column means the output value.
[0047]
In the input buffer group 22 for supplying pixel data to the input port, 32 pieces of pixel data X1 to X32 continuous in the horizontal direction are transferred and held by the POUC 209 as shown in FIG. The object of the filtering process here is 16 pixel data of X9 to X24. As shown in FIG. 8, the pixel data X9 to X24 are supplied to the input ports A and B of the pixel processing units 1 to 16, and the filter coefficient a0 / 2 selected by the input buffer group 22 is supplied to the input port C as an initial value. Is done.
[0048]
Further, after the initial input value is supplied from the input buffer group 22 to the pixel parallel processing unit 21, the filtering process is performed by the number of clock inputs corresponding to the desired number of taps as the filtering process.
FIG. 10 is an explanatory diagram showing a calculation process of the pixel processing unit 1 as a representative of the 16 pixel processing units. In the figure, the contents held by the delay units A to D in the pixel processing unit 1 and the output value of the adder B are shown for each number of input clocks. FIG. 11 is a diagram illustrating an output value of the output port D (adder B output) for each clock input of the pixel processing unit 1.
In the pixel processing unit 1, delay devices A and B hold pixel data X 9 as an initial input value by the first clock input (CLK 1), the delay device D holds the filter coefficient a 0/2, and the delay device D is cleared to zero. At this time, the selection units A and B both select the input port. As a result, the adder A outputs (X9 + X9), the multiplier A outputs (X9 + X9) * a0 / 2, and the adder B outputs (X9 * a0 / 2 + 0 (that is, a0 * X9) ( (See FIG. 11).
[0049]
After the second clock input (CLK2), the selection units A and B select the shift output from the adjacent pixel processing unit or pixel transfer unit instead of the input ports A and B.
The pixel data X10 and X8 and the filter coefficients a1 and a0 * X9 are held in the delay devices A to D by the second clock input (CLK2). As a result, the adder B outputs a0 * X9 + a1 (X10 + X8) (see FIG. 11). In this way, the second time, the multiplier A multiplies the filter coefficient a1 (delay device C) and the sum of pixel data shifted out from both sides (adder A). The adder B adds the multiplication result and the accumulated value of the delay unit D.
[0050]
At the third clock input (CLK3), the pixel processing unit 1 operates in the same manner as the second clock input, thereby adding a0 * X9 + a1 (X10 + x8) + a2 (X11 + X7) from the adder B. Is output.
By performing the same operation at the fourth to ninth clock inputs (CLK4 to CLK9), the adder B outputs the output values shown in FIG.
[0051]
In this way, when the filer processing result (output data) of the pixel processing unit 1 is 9 clocks,
a0 ・ X9 + a1 (X10 + X8) + a2 (X11 + X7) + a3 (X12 + X6)
+ a4 (X13 + X5) + a5 (X14 + X4) + a6 (X15 + X3) + a7 (X16 + X2) + a8 (X17 + X1)
It becomes.
[0052]
10 and 11 show the processing steps up to CLK9, but the number of input clocks is terminated by the control of the instruction decoder 25 in accordance with the number of taps notified from the POUC 209. That is, each pixel processing unit ends the filtering process at CLK2 when the number of taps is 3, ends at CLK3 when the number of taps is 5, and ends the filtering process at CLK4 when the number of taps is 7. In other words, the filter processing with the number of taps (2n-1) ends with n clock inputs.
[0053]
The instruction decoder 25 repeats 16 pixel parallel processing for 16 lines, thereby completing the filter processing for 4 blocks. At this time, the 16 filter processing results are reduced or enlarged by being thinned or interpolated in the output buffer group 23. Every time 16 pixel groups after reduction or enlargement of the output buffer group 23 are held, they are transferred to the external memory 220 via the dual port memory 100 under the control of the POUC 209. Further, the instruction decoder 25 notifies the POUC 209 of the end when the 16th line ends. The POUC 209 instructs the POUA 207 to supply the initial input value and the filter coefficient and start the filter process in the same manner as described above for the next macroblock.
[0054]
Note that the filter processing result of the pixel processing unit 2 is represented by the following expression when 9 clocks are used.
a0 ・ X10 + a1 (X11 + X9) + a2 (X12 + X8) + a3 (X13 + X7)
+ a4 (X14 + X6) + a5 (X15 + X5) + a6 (X16 + X4) + a7 (X17 + X3) + a8 (X18 + X2)
The filter processing result of the pixel processing unit 3 is expressed by the following equation when 9 clocks are used.
a0 ・ X11 + a1 (X12 + X10) + a2 (X13 + X9) + a3 (X14 + X8)
+ a4 (X15 + X7) + a5 (X16 + X6) + a6 (X17 + X5) + a7 (X18 + X4) + a8 (X19 + X3)
The filter processing results of the pixel processing units 4 to 16 are the same except that the pixel positions are different.
[0055]
As described above, the pixel parallel processing unit 21 can perform the filter processing on the 16 input pixels in parallel, and can arbitrarily set the number of taps by controlling the number of input clocks.
In FIG. 8, the input pixels of the input ports A, B, and C of the pixel processing unit 1 are (X9, X9, a0 / 2), but (X9, 0, a0) or (0, X9, a0) It is good. The pixel processing units 2 to 16 may be the same except that the target pixels are different.
<2.2 MC (Motion Compensation) Processing>
Details of MC processing when the decoding target frame is a P picture will be described.
[0056]
The POUC 209 instructs the instruction circuit 26 to start MC processing, specifies a macro block (difference value) in the decoding target frame to be MC processed, and a rectangular area pointed to by the motion vector in the reference frame, and the POUA 207 Alternatively, 16 difference values D 1 to D 16 and 16 pixel data P 1 to P 16 in the rectangular area are set in the input buffer group 22 for the POUB 208.
[0057]
FIG. 12 is a diagram showing input / output pixel data when MC processing (P picture) is performed in the pixel calculation unit. In the figure, the input port column means the input ports of the pixel transfer unit 17, the pixel processing units 1 to 16, and the pixel transfer unit 18 shown in FIGS. 4 and 5. The input pixel column means pixel data input to the input port. Since the pixel transfer units 17 and 18 are not used in the MC process, the input pixel may be anything (don't care). The output port column means the output port D (adder B output) shown in FIGS. 4 and 5, and the output pixel column means the output value.
[0058]
FIG. 13 is an explanatory diagram of input pixels to the pixel processing units 1 to 16 in the MC processing. As shown in the figure, D1 to D16 are 16 difference values in the macroblock (MB) of the decoding target frame. P1 to P16 are 16 pieces of pixel data in the rectangular area indicated by the motion vector in the reference frame.
In the MC processing, the selection units A and B in the pixel processing units 1 to 16 always select the input ports A and B, respectively. Thereby, the pixel data from the input port A and the difference value from the input port B are input to the delay units A and B via the selection units A and B, held, and further added by the adder A. The addition result is multiplied by 1 by the multiplier, 0 is added by the adder B, and the result is output from the output port D. That is, the pixel data from the input port A and the difference value from the input port B are simply added and output from the output port D.
[0059]
Further, the 16 addition results are stored in the output buffer group 23 and written back to the decoding target frame in the external memory 220 via the dual port memory 100 by the POUC 209.
MC processing is performed by repeating the above processing in units of 16 pixels of the decoding target frame. Each pixel processing unit simply performs addition, and an addition result of 16 pixels can be obtained for each clock.
[0060]
Next, MC processing when the decoding target frame is a B picture will be described.
FIG. 14 is a diagram showing input / output pixel data when MC processing (B picture) is performed in the pixel calculation unit. In the figure, the input port column, input pixel column, output port column, and output pixel column are the same as in FIG. However, the input pixel column is different from FIG. 12 in that the first clock (CLK1) and the second clock (CLK2) are input in two steps.
[0061]
P1 to P16 and B1 to B16 are 16 pieces of pixel data in a rectangular area indicated by a motion vector in two different reference frames.
In the MC processing, the selection units A and B in the pixel processing units 1 to 16 always select the input ports A and B, respectively. In the first clock (CLK1), P1 and B1 are held in the delay units A and B from the input ports A and B via the selection units A and B, and are simultaneously held in the constant 1/2 delay unit C from the input port C. . As a result, (P1 + B1) / 2 is obtained from the multiplier A. In the second clock (CLK2), the multiplication result (P1 + B1) / 2 is held in the delay device D, and at the same time, (1,0, D1) from the input ports A, B, C are the delay devices A, B, C. Therefore, D1 from the multiplier A and (P1 + B1) / 2 from the delay unit D are added by the adder B. As a result, (P1 + B1) / 2 + D1 is output from the output port.
[0062]
Further, the 16 addition results are stored in the output buffer group 23 and written back to the decoding target frame in the external memory 220 via the dual port memory 100 by the POUC 209.
MC processing for the B picture is performed by repeating the above processing in units of 16 pixels of the decoding target frame.
<2.3 OSD (On Screen Display) Processing>
The POUC 209 instructs the instruction circuit 26 to start OSD processing, and sequentially reads 16 pixel data X1 to X16 from the OSD image held in the external memory 220 and sets them in the input buffer group 22.
[0063]
FIG. 15 is a diagram showing input / output pixel data when OSD (on-screen display) processing is performed in the pixel arithmetic unit.
In the figure, the pixel transfer units 17 and 18 are not used. Pixel data X1 to X16 are input to the input port A of the pixel processing units 1 to 16 from the input buffer group 22, 0 is input to the input port B, and 1 is input to the input port C, respectively. FIG. 16 shows a state in which 16 pixels in the OSD image are sequentially written in the input buffer group 22.
[0064]
The selection units A and B in the pixel processing units 1 to 16 always select an input port in the OSD process. For example, in the pixel processing unit 1, pixel data X1 of the input port A and “0” of the input port B are held in the delay units A and B, respectively, and further added by the adder A (X1 + 0 = X1). . The addition result is multiplied by “1” input from the input port C by the multiplier A, and “0” is added by the adder B. As a result, the pixel data X1 of the input port A is output from the adder B as it is. Similarly, the pixel data X2 to X16 of the input port A from the pixel processing unit 2 to the pixel processing unit 16 are output from the adder B as they are.
[0065]
The pixel data X1 to X16 output from the adder B are stored in the output buffer group 23, and further overwritten on the display frame data in the external memory 220 via the dual port memory 100 by the POUC 209.
As shown in FIG. 16, the above processing is repeated for the entire OSD image, whereby the OSD image in the external memory 220 is overwritten and copied to the display frame data. This is the simplest of the OSD processes, and the POUA 207 or the POUB 208 simply relays the OSD image in units of 16 pixels.
[0066]
As another form of OSD processing, (1) OSD images and display frame data may be blended. When the blend rate is 0.5, the pixel data of the OSD image is input from the input buffer group 22 to each input port A of the pixel processing unit 1 to the pixel processing unit 16, and the pixel data of display frame data is input to each input port B. What is necessary is just to supply.
When the blend ratio is α: (1−α), (OSD image pixel data, 0, α) is input from the input buffer group 22 to the input ports A, B, and C of each pixel processing unit in the first clock. In the second clock, (0, pixel data of display frame data, 1-α) may be supplied.
[0067]
Further, when the OSD image is displayed in a reduced size, the above-described filtering process may be applied to the OSD image from the input buffer group 22 and overwritten from the output buffer group 23 to the position to be reduced and displayed in the display frame data.
Furthermore, the OSD image may be reduced by filtering and then blended.
<2.4 ME (Motion Prediction) Processing>
FIG. 17 is a diagram illustrating input / output pixel data when ME (motion prediction) processing is performed in the pixel arithmetic unit. In the input pixel column of the figure, X1 to X16 are 16 pixels of the macroblock in the encoding target frame, and R1 to R16 are 16 pixels in a 16 × 16 pixel rectangular area in the reference frame. FIG. 18 is an explanatory diagram showing the relationship between these pixels. The reference frame motion vector (MV) search range in FIG. 6 is a range to be searched for motion vectors around the same position as the encoding target macroblock (for example, +16 pixels to −16 pixels in the horizontal and vertical directions). It is. In this MV search range, a 16 × 16 rectangular area is present at 16 × 16 positions for pixel-by-pixel search, and 32 × 32 for half-pel (1 / 2-pixel) search. Located in the street position. FIG. 13 shows only the upper left rectangular area in the MV search range.
[0068]
In the ME processing, the sum of differences between pixels is obtained between each rectangular area in the MV search range and the macroblock to be encoded, and the rectangular area with the smallest sum (that is, the highest correlation). The displacement of the relative position between the rectangular area) and the encoding target macroblock is determined as a motion vector. The encoding target block is compared with the rectangular region having the highest correlation.
[0069]
Under the control of the POUC 209, the pixel data X1 to X16 to be encoded and the pixel data R1 to R16 of one rectangular area are transferred to the input buffer group 22. As for the pixel data R1 to R16 in the rectangular area, one line in the rectangular area is transferred for each clock. Accordingly, R1 to R16 for 16 lines are transferred for one rectangular area.
According to FIG. 17, for example, the pixel processing unit 1 shown in FIG. 4 performs subtraction and absolute value conversion between the pixel data X1 of the input port A and the pixel data R1 of the input port B in the adder A at the first clock. And passes through the multiplier A (multiplied by 1). The adder B outputs an addition value between the multiplier output and the data held in the delay unit D. In the first clock, the adder B outputs | X1-R1 | on the first line.
[0070]
In the second clock, the first line | X1-R1 | is held in the delay unit. Therefore, the adder B is held in the second line | X1-R1 | from the multiplier A and the delay unit D. Add | X1-R1 | on the first line.
In the third clock, the first and second lines | X1-R1 | are accumulated in the delay unit, so that the adder B holds the third line | X1-R1 | from the multiplier A and the delay unit D. Then, | X1-R1 | in the first line is added.
[0071]
In the 16th clock, the adder B outputs the cumulative value (Σ | X1-R1 |) of | X1-R1 |
Accumulated values (Σ | X1-R1 |) to (Σ | X16-R16 |) are also output for the pixel processing units 2 to 16, respectively.
These 16 accumulated values are held in the output buffer group 23 at the 17th clock, taken out by the POUC 209, and the sum of the 16 accumulated values is calculated and stored in the work area in the external memory 220.
[0072]
This completes the calculation of the sum of the differences between the pixel data of one rectangular area and the encoding target macroblock.
Thereafter, the sum of differences is calculated in the same manner for other rectangular areas in the MV search range. When the sum of differences is calculated for all rectangular areas (or necessary rectangular areas) within the MV search range, the rectangular area having the smallest value is determined as the rectangular area having the highest correlation, and a motion vector is generated. The
[0073]
In the ME process, the total of 16 accumulated values from the pixel processing unit is separately performed, but the total of 16 accumulated values may be calculated in the pixel processing units 1 to 16. In this case, the 16 accumulated values for one rectangular area are stored as they are from the output buffer group 23 in the work area of the external memory 220, and the accumulated value groups for 16 or more rectangular areas are stored in this work area. In this case, each of the pixel processing units 1 to 16 may share one rectangular area and sequentially accumulate the 16 accumulated values to obtain the sum of the differences.
[0074]
In the ME process, the difference is calculated in units of pixels, but may be calculated in units of half pels. In that case, of the half line and the real line, | X1-R1 | is calculated for one line as described above for the real line, and for half lines, for example, one half of the two clocks is half-pel. The pixel value ((R1 + R1 ′) / 2) may be calculated, and the difference | X1− (R1 + R1 ′) / 2 | may be calculated in the next one clock. Alternatively, half-pel pixel values ((R1 + R1 ′ + R2 + R2 ′) / 4) may be calculated in 4 clocks out of 5 clocks, and the difference may be calculated in the next 1 clock.
<3.1 Vertical Filter Processing (Part 1)>
FIG. 19 is a schematic block diagram of the media processor showing the flow of data when performing vertical filter processing in the media processor shown in FIG.
[0075]
In the figure, a decoder unit 301 corresponds to the VLD 205, TE 206, and POUA 207 (MC processing) for decoding (decompressing) the video elementary stream in FIG. 2, and decodes (decompresses) the video elementary stream.
The frame memory 302 corresponds to the external memory 220 and holds video data (frame data) as a decoding result.
The vertical filter 303 corresponds to the POUB 208 and performs vertical reduction by vertical filter processing.
The buffer memory 304 corresponds to the external memory 220 and holds reduced video data (frame data for display).
[0076]
The image output unit 305 corresponds to the video buffer memory 212 and the video unit 213, converts display frame data into a video signal, and outputs the video signal.
The POUA 207 is responsible for MC processing and the POUB 208 is responsible for vertical filter processing. Further, it is assumed that the horizontal reduction by the horizontal filter processing is performed by one of the POUA 207 and the POUB 208 with respect to the decoded frame data of the frame memory 302.
<3.1.1 1/2 reduction>
FIG. 20 is a diagram showing temporal changes in the data supply states of the frame memory 302 and the buffer memory 304 when the 1/2 reduction process is performed in FIG.
[0077]
In FIG. 20, the vertical axes of the graphs 701 to 703 indicate time in units of the period V of the vertical synchronization signal of the field. In the figure, five periods are shown, and the time axes of the graphs 70 to 703 are the same. The horizontal axis of the graph 701 indicates the amount of data in the frame memory 302. The horizontal axis of the graph 7702 indicates the amount of data in the buffer memory 304. A graph 703 shows a frame (field) being output by the image output unit 305.
[0078]
A solid line 704 in the graph 701 indicates the amount of frame data supplied from the decoder unit 301 to the frame memory 302. A broken line 705 indicates the amount of frame data supplied from the frame memory 302 to the vertical filter unit 303.
A broken line 706 in the graph 702 indicates the supply amount of the 1st field reduced image from the vertical filter unit 303 to the buffer memory 304. An alternate long and short dash line 707 indicates the supply amount of the 2nd field reduced image from the vertical filter unit 303 to the buffer memory 304.
[0079]
A solid line 708 in the graph 702 indicates the supply state of 1st field reduced image data from the buffer memory 304 to the image output unit 305. In the case of 1/2 reduction, since the display position of the reduced image can be taken from the upper half position to the lower half position of the frame, the solid line 709 in the figure differs in timing depending on the display position. Similarly, a solid line 709 indicates a supply state of 2nd field reduced image data from the buffer memory 304 to the image output unit 305.
[0080]
As shown by the graph 701, the supply of the frame data of n frames from the decoder unit 301 to the frame memory 302 starts immediately after the supply of the 2nd field of the n-1 frame from the frame memory 302 to the vertical filter unit 303 is started. Control is performed so that the supply of frame data of n frames from the memory 302 to the vertical filter unit 303 is completed immediately before the supply from the frame memory 302 of the 1st field of n frames to the vertical filter unit 303 is completed.
[0081]
As shown by the graph 702, the supply of the frame data of the 1st field of n frames from the vertical filter unit 303 to the buffer memory 304 is the display of the frame data of the 2nd field of n frames while the 2nd field of the n-1 frame is displayed. Control is performed so that each is completed during display of the 1st field of n frames.
By controlling the apparatus in this manner, it is sufficient that the decoder unit 301 and the frame memory 302 have the ability to transfer one frame of frame data in a 2V period. Between the frame memory 302 and the vertical filter unit 303, it is sufficient to have the ability to transfer 1/2 frame data in a 1V period. It is sufficient that the decoder unit 301 has an arithmetic capability for generating one frame of frame data in a 2V period, and the vertical filter unit 303 has an arithmetic capability to filter 1/2 frame data in a 1V period. Between the vertical filter unit 303 and the buffer memory 304, it is sufficient to have the ability to transfer ¼ frame data in a 1V period. Between the buffer memory 304 and the image output unit 305, it is sufficient to have the ability to transfer 1/4 frame data in a 1V period. The frame memory 302 holds one frame of frame data, and the buffer memory 304 only needs to have a capacity for holding 1/2 frame data.
[0082]
Next, for comparison with FIG. 20, FIG. 21 shows a time change in the data supply state when the buffer memory 304 is not provided.
When the reduction process is not performed, the supply of digital image data of n frames to the frame memory 302 has started to be supplied to the vertical filter unit 303 of the 2nd field of the n-1 frame indicated by the broken line 507, as indicated by the solid line 506. It starts from the time and ends before the supply to the vertical filter unit 303 of the 1st field of the n frame indicated by the broken line 508 is completed. Therefore, one frame of digital image data is supplied at a constant rate during the 2V period shown on the graph of FIG.
[0083]
Also, the supply of digital image data from the frame memory 302 of the 1st field of n frames to the vertical filter unit 303 ends the supply of the digital image data of n frames to the frame memory 302 indicated by the solid line 511 as indicated by a broken line 508. The process is completed immediately after, and then the processing of the 2nd field is started. For this reason, digital image data is supplied from the frame memory 302 to the vertical filter unit 303 by supplying one field of digital image data at a constant rate during a period of 1V shown in the graph of FIG.
[0084]
However, when the 1/2 reduction process is performed, the timing at which the supply of n frames of digital image data to the frame memory 302 can be started differs depending on the display position of the 2nd field of the n-1 frame. Depending on the display position of the 2nd field of n−1 frame, the digital image data is supplied from the frame memory 302 to the vertical filter unit 303 somewhere between the broken lines 509 to 510, and n frames of digital data are supplied to the frame memory 302. The timing at which the supply of image data can be started is most delayed in the case of the display position indicated by the broken line 510. In this case, the 1/2 reduced image is output to the lower half of the image output unit 501. Further, the supply of n frames of digital image data to the frame memory 302 must be completed before the supply to the vertical filter unit 303 of the 1st field of the n frames indicated by the broken line 511 is completed. For this reason, it is necessary to supply one frame of digital image data at a constant speed during the 1 V period shown in the graph of FIG. 21, and twice the supply capability is required as compared with the case where no reduction is performed.
[0085]
In addition, the supply of digital image data from the frame memory 302 of the 1st field of n frames to the vertical filter unit 303 ends the supply of the digital image data of n frames to the frame memory 302 indicated by the solid line 512, as indicated by a broken line 511. The process is completed immediately after, and then the processing of the 2nd field is started. Therefore, it is necessary to supply one field of digital image data at a constant speed during the period of 1 / 2V shown in the graph of FIG. 5, and twice the supply capability is required as compared with the case where no reduction is performed. . Since the vertical filter unit 303 is also required to have performance corresponding to the supplied digital image data, it requires twice the computing power as compared with the case where no reduction is performed.
[0086]
For comparison with FIG. 20, FIG. 23 shows a time change in the data supply state when the 1/4 reduction process is performed when the buffer memory 304 is not provided.
FIG. 23 shows a graph when the 1/4 reduction processing is performed. For the same reason as described above, the supply capability of digital image data to the frame memory 302, the supply capability from the frame memory 302 to the vertical filter unit 303, and the calculation capability of the vertical filter unit are four times that when the reduction process is not performed. Necessary. Thus, when the buffer memory 304 is not provided, the required peak performance increases as the reduction ratio increases.
<3.1.2 1/4 reduction>
FIG. 22 is a diagram showing a data supply state of each unit and its change over time when 1/4 reduction is performed by the media processor shown in FIG.
[0087]
In FIG. 22, the horizontal and vertical axes of the graph are the same as those in FIG.
A solid line 804 on the graph indicates a supply state of frame data from the decoder unit 301 to the frame memory 302. A broken line 805 on the graph indicates a supply state of frame data from the frame memory 302 to the vertical filter unit 303. A broken line 806 on the graph indicates a supply state of 1st field reduced image data from the vertical filter unit 303 to the buffer memory 304. A broken line 807 on the graph indicates a supply state of 2nd field reduced image data from the vertical filter unit 303 to the buffer memory 304. A solid line 808 on the graph indicates a supply state of 1st field reduced image data from the buffer memory 304 to the image output unit 305. A solid line 809 on the graph indicates a supply state of 2nd field reduced image data from the buffer memory 304 to the image output unit 305.
[0088]
As shown in the figure, between the decoder unit 301 and the frame memory 302, it is sufficient if there is an ability to transfer one frame of frame data in a 2V period. Between the frame memory 302 and the vertical filter unit 303, it is sufficient to have the ability to transfer frame data of 1/2 frame in a 1V period. It suffices that the decoder unit 301 has a calculation capability for generating one frame of frame data in a 2V period. The vertical filter unit 303 is capable of filtering 1/2 frame data in a 1V period. Between the vertical filter unit 303 and the buffer memory 304 is 1/8 frame data transfer capability in a 1V period. Between 304 and the image output unit 305, a frame data transfer capability of 1/8 frame is sufficient for a period of 1V. A frame memory 302 that can hold one frame of frame data and a buffer memory 304 that can hold ¼ frame data are required.
[0089]
Each of these required performances is an average ability in a period of 1 V at the shortest, and even if the reduction ratio increases, a large peak performance is not required in a short period. The processing performance is most required when there is no reduction. In this case, the frame data transfer capability of 1 frame is sufficient between the decoder unit 301 and the frame memory 302 for a period of 2V. Between the frame memory 302 and the vertical filter unit 303, a frame data transfer capability of 1/2 frame is sufficient for a period of 1V. The decoder unit 301 only needs a calculation capability to generate frame data of one frame in a 2V period. The vertical filter unit 303 only needs to have a computing capability to filter 1/2 frame data in a 1V period. Between the vertical filter unit 303 and the buffer memory 304, a frame data transfer capability of 1/2 frame is sufficient for a period of 1V. Between the buffer memory 304 and the image output unit 305, a frame data transfer capacity of 1/2 frame is sufficient for a period of 1V. The frame memory 302 can hold one frame of frame data, and the buffer memory 304 only needs to hold one frame of frame data. With this ability, you can perform any vertical reduction process. As a result, the circuit scale can be reduced and the operation clock can be lowered.
<3.2 Vertical Filter Processing (Part 2)>
FIG. 24 is a schematic block diagram showing the flow of data when vertical filtering is performed in the media processor.
[0090]
The figure includes a decoding unit 401, a buffer memory 402, a vertical filter unit 403, a buffer memory 404, a video output unit 405, and a control unit 406. Compared with FIG. 19, the decoding unit 401, the vertical filter unit 403, the buffer memory 404, and the video output unit 405 are the same as those in FIG. Therefore, the description of the same points will be omitted, and different points will be mainly described.
[0091]
The buffer memory 402 is different from the frame memory 302 in that the capacity may be smaller than the storage capacity for one frame.
The vertical filter unit 403 is different from the vertical filter unit 303 in that the vertical filter unit 403 notifies the control unit 406 of the fact (filter state) every time filtering processing of 64 lines in the vertical direction (4 macroblock lines in the frame before processing) is completed. Different. The unit of notification may be a unit of 2-3 macroblock lines.
[0092]
The decoding unit 401 is different from the decoding unit 301 in that the decoding unit 401 notifies the control unit 406 to that effect (decoding state) every time decoding of 64 lines is completed. The unit of notification may be a unit of 16 lines.
The control unit 406 corresponds to the IOP 211 in FIG. 2 and monitors the operation states of the decoding unit 401 and the vertical filter unit 403 based on notifications from each, so that the vertical filter processing does not exceed the decoding processing. In addition, the decoding unit 401 and the vertical filter unit 403 are controlled so that the decoding process does not overtake the vertical filter process. That is, the following two of the control unit 406 are controlled. One is that the pixel data of the macroblock line to be filtered is not written in the buffer memory 402 by the decoding unit 401, but the pixel of the macroblock line of the previous frame (or field) is written by the vertical filter unit 403. This is to prevent the filtering process from being performed on the data group. The other is to prevent the decoding unit 401 from overwriting the pixel data group of the next frame with respect to a macroblock line which is subject to vertical filtering by the vertical filter unit 403 but has not been processed.
[0093]
FIG. 25 is an explanatory diagram showing the control contents in the control unit 406.
The horizontal axis in the figure is time, and shows the operations of the control unit 406, VSYNC (vertical synchronization signal), decoding unit 401, vertical filter unit 403, and video output unit 405.
As shown in the figure, the decoding unit 401 notifies the control unit 406 every time 64 lines have been decoded, and the vertical filter unit 403 notifies the control unit 406 every time 64 lines have been filtered. To do. Under these notifications, the control unit 406 holds and updates the line number Nd for which decoding has been completed and the line number Nf for which filtering has been completed, and Nd (current frame)> Nf (current frame), Nd ( The decoding unit 401 and the vertical filter unit 403 are controlled so as to satisfy (next frame) <Nf (current frame). Specifically, the control unit 406 temporarily stops one of the decoding unit 401 and the vertical filter unit 403 when Nd and Nf approach each other (when the difference becomes equal to or less than a threshold value). Nd and Nf may be macroblock line numbers.
[0094]
Also, when Nd and Nf approach, one of the decoding unit 401 and the vertical filter unit 403 is temporarily stopped by the control unit 406 under the control of the control unit 406, but whether or not Nd and Nf approach each other The determination and the control for temporarily stopping the decoding unit 401 or the vertical filter unit 403 may be configured so as to be in charge of other than the control unit 406.
[0095]
For example, the vertical filter unit 403 notifies the decoding unit 401 of the filter state, and the decoding unit 401 determines whether Nd and Nf have approached according to the notification of the filter state and the internal decoding state. The decoding operation may be temporarily stopped or the vertical filter unit 403 may be temporarily stopped according to the determination result.
[0096]
Or, conversely, the decoding unit 401 notifies the vertical filter unit 403 of the decoding state, and the vertical filtering unit 403 determines whether Nd and Nf approach each other according to the notification of the decoding state and the internal filter state. It is good also as a structure which determines whether or not and stops the filter process temporarily according to the determination result, or stops the decoding part 401 temporarily.
<3.2.1 1/2 reduction>
FIG. 26 is a diagram showing the supply data amount of each part when the 1/2 reduction process is performed in FIG.
[0097]
The horizontal axis of the graph 901 indicates the amount of frame data on the buffer memory 402, and the vertical axis indicates time. The horizontal axis of the graph 902 indicates the amount of frame data on the buffer memory 404, and the vertical axis indicates time. A graph 903 shows the state of the image output unit 405 arranged in time series, and the time axis matches the vertical axes of the graphs 901 and 902.
[0098]
A solid line 904 on the graph indicates a supply state of frame data from the decoder unit 401 to the buffer memory 402. A broken line 905 on the graph indicates a frame data supply state from the buffer memory 402 to the vertical filter unit 403. A broken line 906 on the graph indicates a supply state of 1st field reduced image data from the vertical filter unit 403 to the buffer memory 404. A broken line 907 on the graph indicates a supply state of the 2nd field reduced image data from the vertical filter unit 403 to the buffer memory 404. A solid line 908 on the graph indicates a supply state of 1st field reduced image data from the buffer memory 404 to the image output unit 405. A solid line 909 on the graph indicates a supply state of the 2nd field reduced image data from the buffer memory 404 to the image output unit 405.
[0099]
As shown in the graph 901, immediately after the supply of n frame data from the decoder unit 401 to the buffer memory 402 is started, the supply of n frame data from the buffer memory 402 to the vertical filter unit 403 is started. Control is performed so that the supply of n frame data from the buffer memory 402 to the vertical filter unit 403 is completed immediately after the supply of n frame data from the decoder unit 401 to the buffer memory 402 is completed. As shown by a graph 902, control is performed so that the supply of frame data of n frames from the vertical filter unit 403 to the buffer memory 404 is completed during n−1 frame display.
[0100]
By controlling the apparatus in this manner, the frame data transfer capability between the decoder unit 401 and the buffer memory 402 is 1 frame in a period of 2V, and the frame data transfer capability between the buffer memory 402 and the vertical filter unit 403 is 1 frame in a period of 2V. From the vertical filter unit 403, the frame data transfer capability, the decoder unit 401 is an arithmetic capability to generate frame data of one frame in a 2V period, and the vertical filter unit 403 is an arithmetic capability to filter frame data of one frame in a 2V period. Between the buffer memories 404, a frame data transfer capability of 1/2 frame in a 2V period, and between the buffer memory 404 and the image output unit 405, a frame data transfer capability of 1/4 frame in a 1V period, a frame for several lines Buffer memory 402 that can hold data, frame data 1 frame A buffer memory 404 that can be lifting is required, respectively.
<3.2.2 1/4 reduction>
FIG. 27 is a diagram showing the data supply amount of each part when 1/4 reduction is performed in FIG.
[0101]
The horizontal axis of the graph 1001 indicates the amount of frame data on the buffer memory 402, and the vertical axis indicates time. The horizontal axis of the graph 1002 indicates the amount of frame data on the buffer memory 404, and the vertical axis indicates time. A graph 1003 is a graph in which the states of the image output unit 405 are arranged in time series, and the time axis matches the vertical axes of the graphs 1001 and 1002.
[0102]
A solid line 1004 on the graph indicates a frame data supply state from the decoder unit 401 to the buffer memory 402. A broken line 1005 on the graph indicates a supply state of frame data from the buffer memory 402 to the vertical filter unit 403. A broken line 1006 on the graph indicates a supply state of 1st field reduced image data from the vertical filter unit 403 to the buffer memory 404. A broken line 1007 on the graph indicates a supply state of the 2nd field reduced image data from the vertical filter unit 403 to the buffer memory 404. A solid line 1008 on the graph indicates a supply state of 1st field reduced image data from the buffer memory 404 to the image output unit 405. A solid line 1009 on the graph indicates a supply state of the 2nd field reduced image data from the buffer memory 404 to the image output unit 405.
[0103]
By controlling the apparatus in this manner, a frame data transfer capability of 1 frame is sufficient between the decoder unit 401 and the buffer memory 402 in a 2V period, and between the buffer memory 402 and the vertical filter unit 403 is 1 in a 2V period. The frame frame data transfer capability is sufficient, the decoder unit 401 is sufficient to generate frame data of one frame in a 2V period, and the vertical filter unit 403 is calculation capability of filtering one frame of frame data in a 2V period. Therefore, a frame data transfer capacity of 1/4 frame is sufficient between the vertical filter unit 403 and the buffer memory 404 in a 2V period, and 1/8 frame is sufficient between the buffer memory 404 and the image output unit 405 in a 1V period. Frame data transfer capability is sufficient. The buffer memory 402 can hold frame data for several lines, and the buffer memory 404 only needs to hold 1/2 frame data.
[0104]
Each of these required performances is an average ability in a period of 1V at the shortest, and a large peak performance is not required in a period where the reduction rate is short.
In the case where the processing performance is required most but no reduction is required, the frame data transfer capability of one frame and the buffer memory are required between the decoder unit 401 and the buffer memory 402 in the period of 2V. Between 402 and the vertical filter unit 403, a frame data transfer capability of 1 frame in a 2V period, a decoder unit 401 has an arithmetic capability to generate frame data of 1 frame in a 2V period, and the vertical filter unit 403 has a 1 in 2V period. Computational capability for filtering frame data of a frame, between the vertical filter unit 403 and the buffer memory 404, a frame data transfer capability of 1 frame in a 2V period, and between the buffer memory 404 and the image output unit 405 1 in a 1V period / 2 frame data transfer capability, buffer that can hold frame data for several lines The memory 402 is a buffer memory 404 that can hold two frames of frame data. With this capability, any vertical reduction processing can be performed. As a result, the circuit scale can be reduced and the operation clock can be lowered.
<4. Modification>
28 and 29 are diagrams illustrating a first modification of the left half and the right half of the pixel parallel processing unit. In these drawings, the same constituent elements as those in FIGS. 3 and 4 are denoted by the same reference numerals, and the description thereof will be omitted.
[0105]
28 and 29 include pixel processing units 1 a to 16 a instead of the pixel processing units 1 to 16 of FIGS. 3 and 4, and pixel transfer units 17 a and 18 b instead of the pixel transfer units 17 and 18. Since the pixel processing units 1a to 16a have the same configuration, the pixel processing unit 1a will be described as a representative.
The pixel processing unit 1a includes a selection unit A104a and a selection unit B105a instead of the selection unit A104 and the selection unit B105 in the pixel processing unit 1.
[0106]
The selection unit A104a is different from the selection unit A104 in that the number of input is changed from two to three. That is, the selection unit A 104a has more pixel data inputs than the delay unit (delay unit B) of the two adjacent pixel transfer units (or pixel processing units).
Similarly, in the selection unit B105a, the pixel data input of the delay unit (delay unit B) of the two adjacent pixel transfer units (or pixel processing units) is increased.
[0107]
The pixel transfer unit 17a includes a selection unit B1703a to a selection unit G1708a instead of the selection unit B1703 to the selection unit G1708. Selection unit B1703a to selection unit G1708a each have three inputs instead of two inputs. The increasing input is the pixel data input from the two left delay units.
The pixel transfer unit 18a includes a selection unit B1803a to a selection unit G1808a instead of the selection unit B1803 to the selection unit G1808. Selection unit B1803a to selection unit G1808a each have three inputs instead of two inputs. The increasing input is the pixel data input from the two right delay devices.
[0108]
According to this configuration, it is possible to perform a filter process that uses a pixel to be processed and two pixels adjacent to the left and right of the pixel in order.
For example, the pixel processing unit 1a can calculate the following equation.
a0 ・ X9 + a1 (X11 + X7) + a2 (X13 + X5) + a3 (X15 + X3)
30 and 31 are diagrams illustrating a second modification of the left half and the right half of the pixel parallel processing unit.
[0109]
30 and 31 include a pixel processing unit 1b and a pixel processing unit 16b instead of the pixel processing unit 1 and the pixel processing unit 16 of FIGS.
The pixel processing unit 1b includes a selection unit b105b instead of the selection unit B105 in the pixel processing unit 1. Selection unit B105b differs from selection unit B105 in that it has a feedback input from delay unit B107.
[0110]
The pixel processing unit 16b includes a selection unit A1604b instead of the selection unit A1604 in the pixel processing unit 16. The selection unit A1604b differs from the selection unit A1605 in that it has a feedback input from the delay unit A1606.
According to this configuration, the pixel processing unit 1b performs, for example, the following calculation.
a3 * X6 + a2 * X7 + a1 * X8 + a0 * X9 + a1 * X10 + a2 * X11 + a3 * X12
At this time, the output of the pixel processing unit 2 is as follows.
[0111]
a3 * X20 + a2 * X21 + a1 * X22 + a0 * X23 + a1 * X24 + a2 * X24 + a3 * X24
At this time, the output of the pixel processing unit 16b is as follows.
a3 * X21 + a2 * X22 + a1 * X23 + a0 * X24 + a1 * X24 + a2 * X24 + a3 * X24
As described above, in FIGS. 30 and 31, when the leftmost pixel data of the data string is transferred to the leftmost pixel processing unit 1b, the selection unit B105b receives the feedback input from the delay unit B in the pixel processing unit 1b. select. When the rightmost pixel data of the data string is transferred to the rightmost pixel processing unit 16b, the selection unit A1604b selects the feedback input from the delay device A1606.
[0112]
32 and 33 are diagrams illustrating a second modification of the left half and the right half of the pixel parallel processing unit.
32 and 33 include pixel processing units 1c to 16c instead of the pixel processing units 1 to 16 of FIGS. 3 and 4, and pixel transfer units 17c and 18c instead of the pixel transfer units 17 and 18. Since the pixel processing units 1c to 16c have the same configuration, the pixel processing unit 1a will be described as a representative.
[0113]
The pixel processing unit 1c includes a selection unit A104c and a selection unit B105c instead of the selection unit A104 and the selection unit B105 in the pixel processing unit 1.
The selection unit A104c is different from the selection unit A104 in that the number of input is changed from two to three. That is, in the selection unit A104c, the pixel data input of the delay unit (delay unit B) of the two adjacent pixel transfer units (or pixel processing units) is increased.
[0114]
In the selection unit B105c, the pixel data input of the delay unit (delay unit B) of the two adjacent pixel transfer units (or pixel processing units) and the feedback input from the delay unit B107 are increased.
Similarly to the pixel transfer units 17a and 18a shown in FIGS. 28 and 29, the pixel transfer units 17c and 18c have three inputs instead of two inputs.
[0115]
According to this configuration, the pixel processing unit 1c performs, for example, the following calculation.
a3 * X9 + a2 * X9 + a1 * X9 + a0 * X9 + a1 * X11 + a2 * X13 + a3 * X15
At this time, the output of the pixel processing unit 2c is as follows.
a3 * X10 + a2 * X10 + a1 * X10 + a0 * X10 + a1 * X12 + a2 * X14 + a3 * X16
At this time, the output of the pixel processing unit 15c is as follows.
a3 * X17 + a2 * X19 + a1 * X21 + a0 * X23 + a1 * X23 + a2 * X23 + a3 * X23
At this time, the output of the pixel processing unit 16c is as follows.
a3 * X18 + a2 * X20 + a1 * X22 + a0 * X24 + a1 * X24 + a2 * X24 + a3 * X24
FIG. 34 is a diagram showing a modification of the POUA 207.
[0116]
The POUA 207 shown in the figure has an upsampling circuit 22a and a downsampling circuit 23a added to FIG. The description of the same points as in FIG. 2 will be omitted, and different points will be mainly described.
The upsampling circuit 22a expands the pixel data group input from the input buffer group 22 in the vertical direction. For example, in order to interpolate the pixel data so that the pixel data group input from the input buffer group 22 is doubled in the vertical direction, the same pixel data group for one input of the pixel data group from the input buffer group 22 Is output to the pixel parallel processing unit 21 twice.
[0117]
The downsampling circuit 23a reduces the pixel data group input from the pixel parallel processing unit 21 in the vertical direction. For example, the pixel data is thinned out so that the pixel data group input from the pixel parallel processing unit 21 is halved in the vertical direction. That is, with respect to two input of the pixel data group from the pixel parallel processing unit 21, one time is discarded and one time is output.
[0118]
According to this configuration, the pixel parallel processing unit 21 doubles in the vertical direction on the input side and halves in the vertical direction on the output side, so the data amount per frame in the external memory 220 is 1 / vertical in the vertical direction. As a result, the amount of data transferred from the POUC 209 to the POUA 207 can be halved. As a result, the bus neck can be eliminated when accesses to the internal ports of the dual port memory 100 are concentrated.
[0119]
In addition, since the pixel arithmetic device of the present invention performs the filtering process for resizing an image on a plurality of pixels in parallel, it can be applied to digital video equipment such as a media processor for handling moving picture compression / expansion processing, resizing, etc. Used.
[0120]
【The invention's effect】
A pixel arithmetic device according to the present invention is a pixel arithmetic device that performs filter processing, and includes N pixel processing means, supply means for supplying N pixel data and filter coefficients, and N pixel processing means in parallel. And control means for operating the device.
Each pixel processing means calculates using the pixel data supplied to the supplying means and the filter coefficient, then acquires pixel data from the pixel processing means adjacent to each pixel processing means, and uses the acquired pixel data And accumulate the calculation results. The control means controls the N pixel processing means so as to repeat the acquisition of the pixel data from the adjacent pixel processing means and the calculation and accumulation using the acquired pixel data for the number of taps.
[0121]
Here, the N pixel processing units form a first shifter that shifts N pixel data to the right and a second shifter that shifts N pixel data to the left. Each pixel processing means performs an operation using two pixel data shifted out from two adjacent pixel processing means.
According to this configuration, it is possible to make the number of taps variable, and it is possible to perform a filtering process that speeds up the process without increasing the frequency.
[0122]
In addition, the pixel arithmetic device of the present invention supplies pixel data of a difference image and pixel data of a reference frame as pixel data from a supply unit.
According to this configuration, it can be used not only for the filtering process but also for the MC (motion compensation) process, and it is not necessary to provide the filter device and the MC circuit independently, so that the circuit scale can be reduced. There is.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an example of a circuit that performs FIR filter processing in the prior art.
FIG. 2 is a block diagram illustrating a configuration of a media processor including a pixel operation unit.
FIG. 3 is a block diagram showing a configuration of a pixel operation unit (POUA, POUB).
FIG. 4 is a block diagram illustrating a configuration of a left half of a pixel parallel processing unit.
FIG. 5 is a block diagram showing a configuration of a right half of a pixel parallel processing unit.
6A is a block diagram illustrating a detailed configuration of an input buffer group 22. FIG.
FIG. 2B is a block diagram illustrating a detailed configuration of the selection unit in the input buffer group 22.
7 is a block diagram showing a configuration of an output buffer group 23. FIG.
FIG. 8 is a diagram illustrating initial input values of pixel data when filter processing is performed in the pixel arithmetic unit.
9 is an explanatory diagram showing initial input values of pixel data to the pixel processing unit 1. FIG.
10 is a diagram illustrating a calculation process in filter processing in the pixel processing unit 1. FIG.
FIG. 11 is an explanatory diagram showing calculation contents of filter processing in the pixel processing unit 1;
FIG. 12 is a diagram showing input / output pixel data when MC (motion compensation) processing (P picture) is performed in the pixel operation unit.
FIG. 13 is an explanatory diagram showing a decoding target frame and a reference frame in MC processing.
FIG. 14 is a diagram showing input / output pixel data when MC processing (B picture) is performed in the pixel operation unit.
FIG. 15 is a diagram showing input / output pixel data when OSD (on-screen display) processing is performed in the pixel calculation unit.
FIG. 16 is an explanatory diagram of OSD (on-screen display) processing in the pixel calculation unit.
FIG. 17 is a diagram illustrating input / output pixel data when ME (motion prediction) processing is performed in the pixel calculation unit.
FIG. 18 is an explanatory diagram of ME (motion prediction) in the pixel calculation unit.
FIG. 19 is a schematic block diagram illustrating a data flow when performing vertical filter processing in a media processor.
FIG. 20 is an explanatory diagram when performing vertical ½ reduction.
FIG. 21 is an explanatory diagram when performing vertical ½ reduction in the prior art.
FIG. 22 is an explanatory diagram when vertical ¼ reduction is performed.
FIG. 23 is an explanatory diagram when vertical ¼ reduction is performed in the prior art.
FIG. 24 is another schematic block diagram showing the flow of data when vertical filtering is performed in the media processor.
FIG. 25 is an explanatory diagram showing timings of decoding processing and vertical filter processing;
FIG. 26 is an explanatory diagram when vertical ½ reduction is performed.
FIG. 27 is an explanatory diagram when vertical ¼ reduction is performed.
FIG. 28 is a diagram illustrating a first modification of the left half of the pixel parallel processing unit.
FIG. 29 is a diagram illustrating a first modification of the right half of the pixel parallel processing unit.
FIG. 30 is a diagram illustrating a second modification of the left half of the pixel parallel processing unit.
FIG. 31 is a diagram illustrating a second modification of the right half of the pixel parallel processing unit.
FIG. 32 is a diagram illustrating a third modification of the left half of the pixel parallel processing unit.
FIG. 33 is a diagram illustrating a third modification of the right half of the pixel parallel processing unit.
FIG. 34 is a diagram showing a modification of the pixel processing unit.
[Explanation of symbols]
1 Pixel processing unit
1 to 16 pixel processing unit
17 Pixel transfer unit
18 pixel transfer section
21 Pixel parallel processing unit
22 Input buffers
22a Upsampling circuit
23 Output buffer group
23a Downsampling circuit
23a-23h Latch
23a-23p latch
23i-23p latch
24 Instruction memory
24a-24h selector
24a-24p selector
24i-24p selector
25 Instruction decoder
26 Indicator circuit
27 DDA circuit
100 dual port memory
104 Selection part A
104a Selection part A
104c Selection part A
105 Selector B
105a Selection part B
105b Selection part B
105c Selection part B
107 Delayer B
109 Delay D
120 Adder A
200 Media processor
201 Stream unit
201 Input port A
202 I / O buffer
202 Input port B
203 Setup processor
203 Input port C
204 bitstream FIFO
205 Variable length code decoding unit
206 TE
207 POUA
208 POUB
209 POUC
210 Audio unit
211 IOP
212 Video buffer memory
213 video unit
214 Host unit
215 RE
216 Filter section
217 Setup memory
218 Dedicated LSI
220 External memory
401 Decoder unit
402 Buffer memory
403 Vertical filter section
404 Buffer memory
405 Video output unit
406 control unit

Claims (2)

A pixel arithmetic device that performs filter processing,
N pixel processing means;
Supply means for supplying N pixel data and filter coefficients;
Control means for operating N pixel processing means in parallel,
The N pixel processing means form a first shifter that shifts N pixel data to the right for each clock input, and a second shifter that shifts N pixel data to the left for each clock input,
The first shifter shifts the pixel data supplied to the supply unit to the right, and then right-shifts the pixel data shifted right from the pixel processing unit adjacent to the left in the previous clock input,
The second shifter shifts the pixel data supplied to the supply means to the left, and then shifts the pixel data left-shifted from the right pixel processing means at the previous clock input to the left,
Each pixel processing means calculates using the pixel data supplied to the supply means and the filter coefficient, and then shifts two pixels that are shifted for each clock input from two pixel processing means adjacent to each pixel processing means. acquires data for each clock input, calculated for each clock input using the acquired pixel data, the operation result pixel calculation apparatus characterized by accumulating every clock input. It further comprises means for specifying the number of taps for filtering,
  The pixel processing means repeats acquisition of pixel data from adjacent pixel processing means, and calculation and accumulation using the acquired pixel data, the number of clock inputs corresponding to a specified number of taps. Item 2. A pixel processing apparatus according to Item 1.

Images