JP2002245448A - Arithmetic unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an arithmetic unit which performs a series of processes, such as inputting, decoding and outputting of compressed images and compressed sound data and has high throughput even though the arithmetic unit is not operated in a high frequency. SOLUTION: This arithmetic device has first and second control/storage parts 506 and 507 for respectively storing a microprogram, a first program counter 504 for designating a first read address in the first control/storage part, a second program counter 505 for designating a second read address, a selector 508 for selecting either the first read address or the second read address and outputting the selected read address to the second control/storage part, and an execution unit 501 for performing an arithmetic by microprogram control by the first and second control/storage parts.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention belongs to the technical field of digital signal processing, and more particularly to video / audio processing for expanding compressed video and audio data, compressing video and audio data, and performing graphics processing. Equipment (arithmetic unit)
About.

[0002]

2. Description of the Related Art In recent years, a digital video data compression / decompression technology has been established and an LSI technology has been improved. Various video and audio processing devices such as an encoder for compression and a graphics process for performing a graphics process are regarded as important.

As a first conventional technique, MPEG (Moving)
(Video Experts Group) Standard video / audio decoder for expanding compressed video and audio data
29). This video / audio decoder performs both video decoding and audio decoding using one signal processing unit. FIG. 1 is an explanatory diagram of a decoding process by the video / audio decoder. In the figure, the vertical axis represents time, and the horizontal axis represents the amount of calculation.

When viewed along the vertical axis, video decoding and audio decoding are performed alternately. This is for decoding both video and audio with common hardware. As shown in the figure, video decoding is divided into sequential processing and block processing. Sequential processing is processing that requires a wide variety of condition determinations, such as decoding of blocks other than blocks, that is, analysis of the header of an MPEG stream, and the amount of calculation is small. Block decoding decodes the variable-length code of the MPEG stream and dequantizes it in block units.
This is a process of performing inverse DCT (discrete cosine transform), and the amount of calculation is large. As shown in the figure, audio decoding is also divided into sequential processing similar to the above, which requires a variety of condition judgments, and decoding processing of audio data itself. The decoding process of the audio data itself requires higher precision than the image data and must be processed within a limited time. Therefore, it is necessary to perform the processing with high precision and high speed, and the amount of calculation is large.

[0005] As described above, the first conventional technique enables one chip, and realizes efficient audio / video decoding with hardware as small as one chip. As a second conventional technique, there is a decoder having a two-chip configuration. One chip is used as a video decoder, and the other chip is used as an audio decoder. FIG. 2 is an explanatory diagram of a decoding process by a two-chip decoder. Both the video decoder and the audio decoder perform sequential processing including a large number of condition determinations such as header analysis, and block decoding processing mainly for decoding the data body. Since both the video decoder and the audio decoder process independently, the performance of each chip may be lower than in the first prior art.

[0006]

However, according to the above prior art, there are the following problems. According to the first prior art, the signal processing unit must decode both video and audio, so that high processing performance is required.
That is, it is necessary to operate using a high-speed clock of 100 MHz or more, and there is a problem that the cost is high as a semiconductor for consumer use. Also, in order to increase processing capacity without using a high-speed clock, VLIW (Very Long Instruction
Although it is conceivable to use a Word) processor or the like, there is a problem that the cost of the VLIW processor itself is high and the overall processing becomes inefficient unless a processor that performs sequential processing is used separately.

According to the second prior art, there is a problem that the cost is high because two processors are used. That is, neither the video processor nor the audio processor can use a general-purpose and inexpensive processor with low processing capability. This is because a video processor is required to be capable of processing a large amount of image data in real time. Also, although the processor for audio does not require as much computational complexity as the processor for video, the audio data is required to have higher accuracy than the image data. Therefore, an inexpensive or low-performance processor does not satisfy the required processing capability for both video and audio.

Further, a tuner for digital (satellite) broadcasting (called STB (Set Top Box)) or a DVD (D
In the case where the video / audio processing apparatus is used in an AV decoder used in an apparatus such as a digital versatile / video disc (playback) apparatus, an MPEG stream received from a broadcast wave or read from a disc is input and its MP
Decode the EG stream and finally display,
A series of processing amounts required until outputting a video signal and an audio signal to a speaker or the like becomes enormous. Recently, there has been an increasing demand for a video / audio processing device (arithmetic device) that efficiently performs such a series of enormous processes.

The present invention performs a series of processing of inputting, decoding, and outputting stream data representing compressed image and compressed audio data, has a high processing capability without operating at a high frequency, and reduces manufacturing costs. It is an object of the present invention to provide an arithmetic device capable of performing the above. It is another object of the present invention to provide an arithmetic unit that realizes decoding of compressed video data, encoding of video data, and graphics processing at low cost.

[0010]

According to the present invention, there is provided a video / audio processing apparatus for inputting a data stream including compressed audio data and compressed video data from the outside.
An input / output processing means for performing input / output processing which is asynchronously generated due to an external factor, and data stored in a memory in parallel with the input / output processing. Decoding processing means for performing decoding processing mainly for decoding a stream, wherein the video data decoded by the decoding processing means and the decoded audio data are stored in a memory;
The input / output processing is to input the data stream asynchronously input from the outside, further store the data in the memory, supply the data stream stored in the memory to the decoding processing unit, and an external display device; Reading from a memory in accordance with the output rate of each audio output device and outputting to them are performed as input / output processing.

According to this configuration, in addition to the input / output processing means and the decoding processing means operating in parallel in a pipeline, the asynchronous processing and the decoding processing are shared between the input / output processing means and the decoding processing means. Therefore, the decoding processing means can be released from the processing that occurs asynchronously and can exclusively use the decoding processing. As a result, the video and audio processing apparatus efficiently executes a series of processing of stream data input, decode, and output, thereby enabling full decoding of stream data (without dropping frames) without using a high-speed operation clock. ing.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of a video and audio processing apparatus according to the present invention will be described in the following sections. 1 First embodiment 1.1 Schematic configuration of video / audio processing device 1.1.1 Input / output processing unit 1.1.2 Decoding processing unit 1.1.2.1 Sequential processing unit 1.1.2.2 Standard processing unit 1.2 Configuration of video / audio processing device 1.2.1 Input Configuration of output processing unit 1.2.2 Decode processing unit 1.2.2.1 Sequential processing unit 1.2.2.2 Routine processing unit 1.3 Detailed configuration of each unit 1.3.1 Processor 7 (sequential processing unit) 1.3.2 Routine processing unit 1.3.2.1 Code conversion unit 1.3.2.2 Pixel operation unit 1.3.2.3 Pixel read / write unit 1.3.3 Input / output processing unit 1.3.3.1 IO processor 1.3.3.1.1 Instruction readout circuit 1.3.3.1.2 Task management unit 1.4 Operation description 2 Second embodiment 2.1 Configuration of Video / Audio Processing Device 2.1.1 Pixel Calculation Unit <1. First Embodiment> The video / audio processing device in the present embodiment is a satellite broadcast receiving device (called STB: Set TopBox), DVD (Digital Versatile Disc). Playback device,
It is provided in a DVD-RAM recording / reproducing device, etc., and receives compressed video / audio data from satellite broadcasts or from a DVD.
The EG stream is input, decompressed (hereinafter, simply referred to as decoding), and a video signal and an audio signal are output to an external output device. <1.1 Schematic Configuration of Video / Audio Processing Device> FIG. 3 is a block diagram showing a schematic configuration of a video / audio processing device according to the first embodiment of the present invention.

The video / audio processing apparatus 1000 includes an input / output processing unit 1001, a decoding processing unit 1002, and a memory controller 6, and is configured to perform input / output processing and decoding processing separately and in parallel. The external memory 3 is used as a working memory for temporarily storing an MPEG stream and audio data after decoding, and a frame memory for storing video data after decoding. <1.1.1 Input / Output Processing Unit> The input / output processing unit 1001 performs input / output processing that occurs asynchronously with the internal operation of the video / audio processing apparatus 1000. This input / output processing includes (a) inputting an MPEG stream asynchronously input from the outside and temporarily storing the same in the external memory 3;
Processing unit 10 decodes the MPEG stream stored in
02, and (c) read the decoded video data and audio data from the external memory 3 and output them in accordance with the output rates of the external display device and audio output device (not shown). . <1.1.2 Decoding Processing Unit> The decoding processing unit 1002
In parallel with the operation of the input / output processing unit 1001, the MPEG stream supplied by the input / output processing unit 1001 is decoded, and the decoded video data and audio data are stored in the external memory 3. Since the decoding processing of the MPEG stream requires a large amount of computation and a wide variety of processing contents, the decoding processing unit 1002 includes a sequential processing unit 1003,
A routine processing unit 1004 is provided to separate serial processing mainly based on a wide variety of conditional judgments and routine processing mainly based on a large number of routine operations and suitable for parallel operations so as to be executed in parallel. It is configured. Here, the sequential processing is MPEG
This is a header analysis of a stream, and includes a number of condition determinations such as header detection and header content determination. In addition, since the routine processing needs to perform various operations on a block unit including a predetermined number of pixels, it is suitable for parallel processing in a pipeline manner, and performs exactly the same operation on different data (pixels). Suitable for parallel processing such as vector operation. <1.1.2.1 Sequential Processing Unit> The sequential processing unit 1003 performs header analysis of the compressed audio data and compressed video data supplied from the input / output processing unit 1001, control to activate the standard processing unit 1004 for each macroblock, and compression. The decoding of the audio data is performed as the above-described sequential processing. The header analysis includes analysis of a macroblock header in an MPEG stream and decoding of a motion vector. Here, the block represents an image composed of 8 * 8 pixels. The macro block is composed of four luminance blocks and two color difference blocks. The motion vector is a vector indicating a rectangular area of 8 * 8 pixels in the reference frame, and indicates a difference between the block and the rectangular area in the reference frame. <1.1.2.2 Routine Processing Unit> The routine processing unit 1004 receives the decoding start instruction for each macroblock from the sequential processing unit 1003, and performs the decoding processing of the macroblock in parallel with the audio decoding processing of the sequential processing unit 1003. Perform as routine processing. This decoding processing includes decoding of a variable length code (VLD: Variable Length code Decoding), inverse quantization (IQ: Inverse Quantization), and inverse discrete cosine transform (I
The content is to perform DCT (Inverse Discrete Cosine Transform) and motion compensation (MC: Motion Compensation) in the same order. In the motion compensation, the routine processing unit 1004
The decoded block is stored in the external memory 3 as a frame memory via the memory controller 6. <1.2 Configuration of Video / Audio Processing Device> FIG. 4 is a block diagram showing a more detailed configuration of the video / audio processing device 1000. <1.2.1 Configuration of Input / Output Processing Unit> In the figure, an input / output processing unit 1001 includes a stream input unit 1, a buffer memory 2, an input / output processor 5 (hereinafter abbreviated as an IO processor 5), a DMAC (Direct Memory Access Controller).
5a, video output unit 12, audio output unit 13, host I / F
And a unit 14.

The stream input unit 1 converts an externally input MPEG data stream into parallel data (hereinafter referred to as MPEG data). At this time, the stream input unit 1 includes one GOP (Group Of Picture: I picture) from the MPEG data stream,
The start code of the MPEG data stream corresponding to a moving image for about 0.5 seconds) is detected, and the IO processor 5 is notified of the start code. The MPEG data converted by this notification is stored in the buffer memory 2 under the control of the IO processor 5.
Is forwarded to

The buffer memory 2 includes a stream input unit 1
Is a buffer memory for temporarily holding the MPEG data transferred from. MPE held in buffer memory 2
The G data is further transferred to the external memory 3 via the memory controller 6 under the control of the input / output processor 5. The external memory 3 is an SDRAM (Synchronous Dynami
c Random Access Memory) chip, and temporarily holds the MPEG data transferred from the buffer memory 2 via the memory controller 6. Further, the external memory 3 holds decoded video data (hereinafter also referred to as frame data) and decoded audio data.

The input / output processor 5 controls data input / output between the stream input unit 1, the buffer memory 2, the external memory 3 (with the memory controller 6 interposed), and the FIFO memory 4. That is, it controls the data transfer (DMA transfer) of the paths shown in the following (1) to (4). (1) Stream input unit 1 → buffer memory 2 → memory controller 6 → external memory 3 (2) external memory 3 → memory controller 6 → FIFO memory 4 (3) external memory 3 → memory controller 6 → buffer memory 2 → video output Unit 12 (4) External memory 3 → Memory controller 6 → Buffer memory 2 → Audio output unit 13 In these routes, the input / output processor 5 controls the transfer of video data and audio data in MPEG data independently. . (1) and (2) are transfer paths of MPEG data before decoding. In the transfer paths (1) and (2), the input / output processor 5 transfers the compressed video data and the compressed audio data separately. (3) and (4) are transfer paths for decoded video and audio data, respectively. The decoded video and audio data are transferred in accordance with the output rates of the external display device (not shown) and the audio output device (not shown).

The DMAC 5a includes a DMA transfer between the stream input unit 1, the video output unit 12, the audio output unit 13 and the buffer memory 2, a DMA transfer between the buffer memory 2 and the external memory 3, and a DMA transfer between the external memory 3 and the FIFO. The DMA transfer between the memories 4 is executed under the control of the IO processor 5. The video output unit 12 is connected to an external display device (CRT)
Etc.) in accordance with the output rate (for example, the period of the horizontal synchronization signal Hsync), and issues a data request to the input / output processor 5.
The video data input by the input / output processor 5 through the transfer path (3) is output to the display device.

The audio output unit 13 issues a data request to the input / output processor 5 in accordance with the output rate of the external audio output device, and converts the audio data input by the input / output processor 5 through the transfer path (4) into audio data. Output to an output device (D / A converter, audio amplifier, combination of speakers, etc.). The host I / F unit 14 includes an external host processor,
For example, in the case of a DVD playback device, this is an interface for performing communication with a processor that performs overall control. In this communication, instructions such as decoding start, stop, fast-forward playback, and reverse playback of the MPEG stream are sent from the host processor. <1.2.2 Decoding unit> Decoding unit 10 in FIG.
02 includes a FIFO memory 4, a sequential processing unit 1003, and a standard processing unit 1004.
The decoding processing of the MPEG data supplied via the FO memory 4 is performed. Further, the sequential processing unit 1003 includes a processor 7 and an internal memory 8. Standard processing unit 1004
Includes a code conversion unit 9, a pixel operation unit 10, a pixel read / write unit 11, a buffer 200, and a buffer 201.

The FIFO memory 4 is composed of two FIFOs (hereinafter referred to as a video FIFO and an audio FIFO). Remember in the formula. <1.2.2.1 Sequential Processing Unit> The processor 7 controls the reading of the compressed video data and the compressed audio data from the FIFO memory 4 and performs a partial decoding process on the compressed video data and a full decoding process on the compressed audio data. Do. Decoding of part of compressed video data
It includes analysis of header information in MPEG data, calculation of motion vectors, and control of compressed video decoding processing. this is,
The processor 7 performs all decoding processing of the compressed video data,
This is because the sharing is performed with the standard processing unit 1004. That is, the processor 7 shares the sequential processing requiring various condition judgments, and the routine processing unit 1004 shares a large amount of routine arithmetic processing. On the other hand, the processor 7 is in charge of the audio decoding since the amount of calculation is smaller than that of the video decoding.

The function of the processor 7 will be specifically described with reference to FIG. FIG. 5 shows the operation timing of each section of the video / audio processing apparatus together with the MPEG stream in a hierarchical manner. In the figure, the horizontal axis is the time axis. The first level is M
5 shows the flow of a PEG stream. An MPEG stream for one second as in the second layer includes a plurality of frames (I, P,
B picture). As in the third layer, one frame is
Includes a picture header and multiple slices. As in the fourth layer, one slice includes a slice header and a plurality of macroblocks. As in the fifth layer, one macroblock includes a macroblock header and six blocks.

The data structure of the first to fifth layers shown in FIG. 1 is described in detail in a well-known document, for example, ASCII Corporation "Point Illustrated Latest MPEG Textbook". As shown in the fifth and lower layers of FIG.
The header analysis up to the macroblock layer in the G stream and the decoding of the compressed audio data are performed. At that time, the processor 7
Is a code conversion unit 9, a pixel operation unit 10, and a pixel read / write unit 11 according to a header analysis result in macroblock units.
To start decoding the macroblock, and the code conversion unit 9, the pixel operation unit 10, and the pixel read / write unit 11
While the macroblock is being decoded by
The compressed audio data is read from the FIFO memory 4 and decoded. When the decoding of the macroblock is completed by the code conversion unit 9, the pixel operation unit 10, and the pixel read / write unit 11, the processor 7 receives a notification to that effect by an interrupt signal, suspends the decoding of the compressed audio data, and Start parsing the macroblock header.

The internal memory 8 is a work memory of the processor 7, and temporarily stores the decoded audio data. The held audio data is transferred to the external memory 3 by the input / output processor 5 through the path (4). <1.2.2.2 Routine processing unit> The code conversion unit 9 is a FIFO
The compressed video data read from the memory 4 is subjected to variable length decoding (VLD). As shown in FIG.
Transfers the header information and the information on the motion vector (broken line section in the figure) of the decoded data to the processor 7, and the macro block (six blocks including the luminance blocks Y0 to Y3 and the color difference blocks Cb and Cr) (The solid line section in the figure) is transferred to the pixel operation unit 10 via the buffer 200. The macroblock data decoded by the code conversion unit 9 is data representing a spatial frequency component.

The buffer 200 holds data representing the spatial frequency components for one block (8 × 8 pixels) written by the code converter 9. The pixel operation unit 10 performs an inverse quantization process (IQ) and an inverse discrete cosine transform (IDCT) on the block data transferred from the code conversion unit 9 via the buffer 200 in block units. The processing result by the pixel operation unit 10 is data representing the luminance value of a pixel or its difference in the case of a luminance block, and data representing the color difference or its difference in the case of a chrominance block. The data is transferred to the read / write unit 11.

The buffer 201 holds one block (8 × 8 pixels) of pixel data. Pixel read / write unit 11
Performs motion compensation on the processing result of the pixel operation unit 10 in block units. That is, for the P picture and the B picture, a rectangular area indicated by the motion vector is cut out from the decoded reference frame in the external memory 3 via the memory controller 6 and synthesized with the block of the processing result of the pixel operation unit 10. To decode the original block image. The decoding result by the pixel read / write unit 11 is stored in the external memory 3 via the memory controller 6.

Since the contents of the motion compensation, IQ, and IDCT are well-known techniques, detailed descriptions thereof are omitted (see the above-mentioned document). <1.3 Detailed Configuration of Each Unit> Next, the video / audio processing device 10
The detailed configuration of each main part of 00 will be described. <1.3.1 Processor 7 (Sequential Processing Unit)> FIG. 6 is a diagram showing the analysis of the macroblock header by the processor 7 and the contents of control to other units. First, the respective data in the macroblock header indicated by the abbreviations in the figure have been described in the above-mentioned documents and the like, and thus description thereof is omitted here.

As shown in the figure, the processor 7 issues a command to the code converter 9 and sequentially obtains the data of the header portion subjected to the variable length decoding, and according to the contents thereof, the code converter 9, the pixel operation unit 10, the pixel Data necessary for decoding a macroblock is set in the read / write unit 11. Specifically, the processor 7 first sends the code conversion unit 9 the MBA
A command for acquiring I (Macro Block Address Increment) is issued (S101), and the MBAI is acquired from the code conversion unit 9. If the macroblock data is a skip macroblock based on this MBAI (if the macroblock to be decoded is the same as the previous macroblock), the macroblock data has been omitted, so the process proceeds to S117 and must be a skip macroblock. If so, the header analysis is continued (S102, 103).

Next, the processor 7 sets the MBT (Macro Bl
A command for acquiring the lock type is issued to acquire the MBT from the code converter 9. From the MBT, it is determined whether the scan type of the block is a zigzag scan or an alternate scan.
00 is designated (S104). further,
The processor 7 performs the STWC from the already obtained header data.
It is determined whether (Spartial Temporal Weight Code) exists (S105), and if it exists, a command is issued and acquired (S106).

Similarly, the processor 7 sets the FrMT (Fr
ame Motion Type), FiMT (Field Motion Type),
DT (DCT type), QSC (Quantizer Scale Code),
MV (Motion Vector), CBP (Coded Block Patter
n) is acquired (S107-116). At this time, the processor 7 notifies the pixel read / write unit 11 of the analysis results of FrMT, FiMT, and DT, notifies the pixel operation unit 10 of the analysis result of QSC, and notifies the code conversion unit 9 of the analysis result of CBP. As a result, information necessary for IQ, IDCT, and motion compensation is set in the code conversion unit 9, the pixel operation unit 10, and the pixel read / write unit 11.

The two-processor configuration has a redundant configuration because each processor individually performs the above-described sequential processing requiring a variety of condition judgments. Next, the processor 7 issues a macroblock decoding start instruction to the code converter 9 (S117). Thus, the code conversion unit 9 starts VLD for each block in the macroblock, and outputs the result of VLD to the pixel operation unit 10 via the buffer 200. Processor 7
Calculates a motion vector based on the MV data (S1
18), and notifies the calculation result to the pixel read / write unit 11 (S119).

In the above processing, regarding the motion vector, the data (MV) of the motion vector is obtained (S113).
Then, a series of processes of calculating a motion vector (S118) and setting the motion vector in the pixel read / write unit 11 (S119) are required. At this point, the processor 7 does not calculate and set the motion vector (S118, 119) immediately after acquiring the motion vector data (MV) (S113, 119), Vectors are calculated and set. Thereby, the motion vector calculation and setting processing of the processor 7 and the decoding processing to the routine processing unit 1004 are processed in parallel. That is, the routine processing unit 1
The decoding start timing of 004 is advanced.

As described above, since the header analysis of the compressed video data for one macroblock is completed, the processor 7 acquires the compressed audio data from the FIFO memory 4 and starts the audio decoding process (S120). . The audio decoding process is continued until an interrupt signal indicating that macroblock decoding has been completed is input from the code converter 9. In response to this interrupt signal, the processor 7 starts the header analysis for the next macroblock. <1.3.2 Routine processing unit> Next, the routine processing unit 1004
The code conversion unit 9 converts the six blocks in the macro block into
The decoding process is performed by operating the pixel operation unit 10 and the pixel read / write unit 11 in parallel (in a pipeline). Here, the configurations of the pixel operation unit 10, the pixel read / write unit 11, and the code conversion unit 9 will be described in detail in this order. <1.3.2.1 Code Conversion Unit 9> FIG. 19 is a block diagram showing the configuration of the code conversion unit 9.

The code converter 9 shown in FIG.
1, counter 902, incrementer 903, selector 904, scan table 905, scan table 9
06, a flip-flop (hereinafter abbreviated as FF) 907, and a selector 908, and are configured to write the result of the variable length decoding (VLD) to the buffer 200 in a block unit so as to be arranged in the order of zigzag scan or alternate scan. I have.

The VLD unit 901 performs variable length decoding (VLD) on the compressed video data read from the FIFO memory 4.
Then, of the decoded data, the header information and the information on the motion vector (broken line section in FIG. 5) are transferred to the processor 7, and the macro blocks (the six blocks including the luminance blocks Y0 to Y3 and the color difference blocks Cb and Cr) are transferred. ) Is output to the buffer 200 in block (64 spatial frequency data) units.

A counter 902, an incrementer 903,
The circuit portion including the selector 904 repeatedly counts from 0 to 63 in synchronization with the output of the spatial frequency data from the VLD unit 901. Scan table 905
Is a table storing the addresses of the block storage areas of the buffer 200 in the order of the zigzag scan. The output values (0 to 63) of the counter 902 are sequentially input, and the addresses are sequentially output. FIG. 20 shows a block storage area for storing 8 × 8 pieces of spatial frequency data in the buffer 200 and a zigzag scan route. The scan table 905 sequentially outputs the pixel addresses in the route shown in FIG.

The scan table 906 is stored in the buffer 20
This is a table storing the addresses of the block storage areas of 0 in the order of the alternate scan.
Output values (0 to 63) are sequentially input, and the addresses are sequentially output. FIG. 21 shows a block storage area in the buffer 200 for storing 8 × 8 pieces of spatial frequency data and a route of the alternate scan. Scan table 90
Reference numeral 5 sequentially outputs the pixel addresses in the route shown in FIG.

The FF 907 holds a flag indicating a scan type (zigzag scan or alternate scan). This flag is set by the processor 7. The selector 908 selects an address output from the scan table 905 and the scan table 906 according to the flag of the FF 907 and outputs the address to the buffer 200 as a write address. <1.3.2.2 Pixel Operation Unit> FIG. 7 is a block diagram showing a configuration of the pixel operation unit 10.

As shown in the figure, the pixel operation unit 10 includes a multiplier 5
02, an execution unit 501 including an adder / subtractor 503, a first program counter (hereinafter abbreviated as a first PC) 504,
Second program counter (hereinafter abbreviated as second PC) 50
5, a first instruction memory 506, and a second instruction memory 507
And a selector 508 so that the IQ and a part of the IDCT can be overlapped and executed in parallel. .

The execution unit 501 includes a first instruction memory 506,
The access and operation of the buffers 200 and 201 are executed according to the micro-instructions sequentially output from the second instruction memory 507. First instruction memory 506, second instruction memory 5
Reference numeral 07 denotes a control storage for storing a microprogram for realizing IQ and IDCT for the block (frequency component) held in the buffer 200. FIG. 8 shows an example of the microprogram stored in the first instruction memory 506 and the second instruction memory 507.

In the figure, the first instruction memory 506 stores I
The DCT1A microprogram and the IQ microprogram are stored, and a read address is specified by the first PC 504. The IQ microprogram is an arithmetic process mainly including reading of the buffer 200 and multiplication, and does not use the adder / subtractor 503. Second instruction memory 507
Stores an IDCT1B microprogram and an IDCT2 microprogram, and the read address is specified by the first PC 504 or the second PC 505 via the selector 508. Here, IDCT1 means processing of the first half of IDCT mainly including multiplication and addition / subtraction.
The 1A microprogram and the IDCT1B microprogram are simultaneously read and executed using the entire execution unit 501. Also, IDCT2 means processing of the latter half of IDCT mainly including addition and subtraction and processing of writing to the buffer 201, and IDCT2 of the second instruction memory 507
The CT2 microprogram is read and executed using the adder / subtractor 503.

Since IQ is processed by the multiplier 502 and IDCT2 is processed by the adder / subtractor 503, they can be operated in parallel. FIG. 9 shows the relationship between the I
The operation timing chart of Q, IDCT1, and IDCT2 is shown. 9, when the code conversion unit 9 writes the data of the luminance block Y0 into the buffer 200 (at timing t)
0), the control signal 102 notifies the pixel operation unit 10 of the fact. The pixel operation unit 10 uses a QS (Quantizer Scale) value set at the time of header analysis of the processor 7,
By reading the IQ microprogram in the first instruction memory 506 in accordance with the address specification of the first PC 504, IQ is performed on the data in the buffer 200. At this time, the selector 508 selects the first PC 504 (timing t1).

Further, the pixel operation section 10 includes a first PC 50
IDCT1A and IDCT1 according to the address designation
The IDCT1 is performed on the data in the buffer 200 by reading the B microprogram. At this time,
Since the selector 508 selects the first PC 504, the first
Both the instruction memory 506 and the second instruction memory 507 have the first
An address from PC 504 is specified (at timing t
2).

Next, the pixel operation unit 10 executes the QS (Quan
Using the tizer scale) value, the IQ microprogram in the first instruction memory 506 is read according to the address designation of the first PC 504 to perform IQ on the data in the block Y1 of the buffer 200, and at the same time, the second PC
The latter half of the IDCT processing is performed on the block Y0 by reading the IDCT2 microprogram in the second instruction memory 507 according to the address specification of 505. At this time, the selector 508 selects the second PC 505. The first PC 504 and the second PC 505 specify addresses independently (timing t3).

Thereafter, similarly, the pixel operation section 10 continues the processing in block units (after timing t4). <1.3.2.3 Pixel Read / Write Unit> FIG. 10 is a block diagram showing a detailed configuration of the pixel read / write unit 11. As shown in the figure, the pixel read / write unit 11 includes buffers 71 to 74 (hereinafter, buffers A to D) and a half-pel interpolator 75.
, A combining unit 76, selectors 77 and 78, and a read / write control unit 79.

The read / write control unit 79 controls the buffer A for the block data input through the buffer 201.
To D, and the final decoded image is transferred to the external memory 3 in units of two blocks. More specifically, the memory controller 6 is controlled so that a rectangular area corresponding to two blocks is read from the reference frame in the external memory 3 according to the motion vector set at the time of the header analysis of the processor 7. As a result, data of a rectangular area for two blocks indicated by the motion vector is stored in the buffer A or the buffer B. Then, the picture type (I
Or P picture or B picture), the combining unit 76 performs half-pel interpolation of a rectangular area of two blocks. Further, by combining (adding) the block data input via the buffer 201 and the rectangular area after the half-pel interpolation, the pixel value of the block is calculated and stored in the buffer B. The final decoded block stored in the buffer B is transferred to the external memory 3 via the memory controller 6. <1.3.3 Input / Output Processing Unit> The input / output processing unit 1001 switches a plurality of tasks sharing various data transfers without overhead in order to execute a large number of data inputs / outputs (data transfers) as described above. In addition, the configuration is such that a response delay to a data input / output request is not caused. The overhead referred to here is the saving and restoring of the context that occurs at the time of task switching. That is, the input / output processor 5 is configured to eliminate the overhead caused by saving and restoring the instruction address and the register data of the program counter to and from the memory (stack area). Here, the detailed configuration will be described. <1.3.3.1 IO Processor> FIG. 11 is a block diagram showing a configuration of the IO processor 5. In FIG.
The O processor 5 includes a state monitoring register 51, an instruction memory 52, an instruction reading circuit 53, an instruction register 54, a decoder 55, an operation execution unit 56, a general-purpose register set group 57, and a task management unit 58. In order to respond to an event, the system is configured to execute a task while switching tasks at an extremely short cycle (for example, four instruction cycles).

The state monitoring register 51 is provided in the register CR1.
To CR3, and the IO processor 5 holds various state data (flags, etc.) for monitoring various input / output states. For example, the state monitoring register 51 includes a state of the stream input unit 1 (start code detection flag in the MPEG stream), a state of the video output unit 12 (a flag indicating a horizontal blanking period, a transfer completion flag of frame data), and an audio output unit. 13 (transfer completion flag of audio frame data) and the state of data transfer between them and the buffer memory 2, external memory 3 and FIFO memory 4 (number of data transfer, data request flag to FIFO memory 4) Holds state data indicating such as

More specifically, the following flags are included. Start code detection flag (hereinafter also referred to as flag 1) This flag is set when the stream input unit 1 detects a start code in an MPEG stream. -Horizontal blanking flag (flag 2) This flag indicates a horizontal blanking period, and is set by the video output unit 12. It is set at a period of about 60 microseconds. -Transfer completion flag of video frame data (flag 3) This flag is transmitted from the external memory 3 to the video output unit 12 by
D when the decoded image data for the frame is transferred
Set by the MAC 5a. • Audio frame data transfer completion flag (flag 4) This flag is set when the decoded audio data for one frame is transferred from the external memory 3 to the audio output unit 13.
Set by AC5a. Data transfer completion flag (flag 5) This flag indicates that when the compressed image data of the number of data specified by the IO processor 5 has been DMA-transferred from the stream input unit 1 to the buffer memory 2 by the DMAC 5a (when the terminal count has been reached). ). DMA request flag (flag 6) This flag indicates that there is data to be subjected to DMA transfer of the compressed image data or the compressed audio data in the buffer memory 2 to the external memory 3, and is set by the IO processor 5 ( Request from task 1 to task 2 described later). Data request flag for video FIFO (flag 7) This flag is a flag for requesting data transfer from the external memory 3 to the video FIFO in the FIFO memory 4.
This is set when the compressed video data of the video FIFO becomes equal to or less than a predetermined amount. This flag is set at a period of about 5 to 40 microseconds. Data request flag for audio FIFO (flag 8) This flag is a flag for requesting data transfer from the external memory 3 to the audio FIFO in the FIFO memory 4.
This is set when the compressed audio data of the audio FIFO becomes equal to or less than a predetermined amount. This flag is set at a period of about 15 to 60 microseconds. Decoder communication request flag (flag 9) This flag is a flag for requesting communication from the decoding processing unit 1002 to the input / output processing unit 1001. Host communication request flag (flag 10) This flag is transmitted from the host processor to the input / output processing unit 10
01 is a flag for requesting communication.

The above-mentioned flags are constantly monitored, not interrupted, by each task executed by the IO processor 5. The instruction memory 52 stores a plurality of task programs sharing a large number of data input / output (data transfer) controls. In this embodiment, six task programs of tasks 0 to 5 are stored. Task 0 (Host I / F task) This task is to perform communication with the host computer, that is, communication processing with the host computer via the host I / F unit 14 when the flag 10 is set. Task to do. For example, start, stop, fast-forward playback, reverse playback, and the like of the MPEG stream are received from the host processor, and a notification of the decoding status (error or the like) is given. This process uses the flag 10 as a trigger. Task 1 (parsing task) This task analyzes (parsing) the MPEG data input from the stream input unit 1 when a start code is detected by the stream input unit 1 (the flag 1).
Then, the program extracts individual elementary streams and transfers the extracted elementary streams to the buffer memory 2 by DMA transfer (the first half of the transfer path (1)). The types of elementary streams extracted here include compressed video data (also called video elementary streams), compressed audio data (also called audio elementary streams), and private data. When the elementary stream is stored in the buffer memory 2, the flag 6 is set. Task 2 (stream transfer / audio task) This task is a program that controls the following transfers (a) to (c).

(A) DMA transfer of each elementary stream from the buffer memory 2 to the external memory 3 (the latter half of the transfer path (1)). This transfer is based on the flags 1, 3
Is the trigger. (b) DMA transfer of the compressed audio data from the external memory 3 to the audio FIFO of the FIFO memory 4 according to the data size (remaining amount) of the compressed audio data held in the audio FIFO (in the transfer path (2)) Transfer to audio FIFO). This data transfer is performed when the data size of the compressed audio data held in the audio FIFO becomes smaller than a certain amount. This transfer is triggered by the flag 8 described above.

(C) From the external memory 3 to the buffer memory 2
DMA transfer of the decoded audio data from the buffer memory 2 to the audio output unit 13 (the transfer path described above).
(Four)). This transfer is triggered by the flag 2 described above. Task 3 (video supply task) This task is to perform DMA of the compressed video data from the external memory 3 to the video FIFO of the FIFO memory 4 according to the data size (remaining amount) of the compressed video data held in the video FIFO. Transfer (video FIFO in transfer path (2) above)
Transfer). This data transfer is performed when the data size of the compressed video data held in the video FIFO becomes smaller than a certain amount. This transfer is triggered by the flag 7 described above. Task 4 (video output task) This task processes the DMA transfer of the decoded video data from the external memory 3 to the buffer memory 2 and further from the buffer memory 2 to the video output unit 12 (the above transfer path (4)). It is a program. This transfer is triggered by the flag 2 described above. Task 5 (Decoder I / F task) This task is a program that processes commands from the decode processing unit 1002 to the IO processor 5.
Commands include "getAPTS", "getVPTS", "getSTC"
and so on. getVPTS (Video Presentation Time Stam
p) is a command by which the decode processing unit 1002 requests the IO processor 5 to acquire the VPTS added to the compressed video data. getAPTS (Audio Presentat
(Ion Time Stamp) is the A assigned to the compressed audio data by the decode processing unit 1002 to the IO processor 5.
This is a command for requesting acquisition of a PTS. getSTC (Syst
em Time Clock) is a command by which the decode processing unit 1002 requests the IO processor 5 to acquire an STC. Upon receiving these commands, the IO processor 5
STC, VPTS, APTS in decoding processing unit 1002
Notify each. STC, VPTS, APTS,
The decoding processing unit 1002 is used to synchronize the decoding of audio and video, and to adjust the progress of decoding in units of frames. This process uses the flag 9 as a trigger.

The instruction reading circuit 53 includes a plurality of program counters (hereinafter, abbreviated as PCs) each indicating an instruction fetch address.
4 is stored. Specifically, the instruction readout circuit 53 has PCs 0 to 5 corresponding to the tasks 0 to 5, and switches the PC at high speed by hardware when the designation of the PC by the task management unit 58 is changed. It is configured. With this configuration, the IO processor 5 saves the PC value of the current task in the memory at the time of the task switch and is released from the process of restoring the PC value of the next task from the memory.

The decoder 55 decodes the instruction read from the instruction memory 52 and stored in the instruction register 54,
The arithmetic execution unit 56 is controlled so as to execute the instruction.
In addition, the decoder 55 sets the entire IO processor 5 to the instruction reading stage of the instruction reading circuit 53,
And the execution stage of the operation execution unit 56 performs at least three stages of pipeline control.

The arithmetic execution unit 56 is provided with an ALU (Arithmetic L
ogical unit), a multiplier, a BS (Barrel Shifter), and the like, and execute the operation specified by the instruction under the control of the decoder 55. The general-purpose register set group 57 includes six register sets corresponding to task 0 to task 5 (one register set includes four 32-bit registers and four 16-bit registers). A register set having a total of 24 32-bit registers and 24 16-bit registers, and corresponding to the task being executed is used.
As a result, the IO processor 5 saves all the current register data in the memory at the time of the task switch and is released from the process of restoring the register data of the next task from the memory.

The task management section 58 performs task switching by switching the PC of the instruction reading circuit 53 and the register set of the general-purpose register set group 57 every predetermined number of instruction cycles. In this embodiment, the predetermined number is four. In addition, since the IO processor 5 processes one instruction in one instruction cycle, the task management unit 58 switches tasks every four instructions without causing the above-mentioned overhead. As a result, response delays for various asynchronous input / output requests are suppressed. In other words, the response delay to the input / output request is at most only 24 instruction cycles. <1.3.3.1.1 Instruction Read Circuit> FIG. 12 is a block diagram showing a detailed configuration example of the instruction read circuit 53.

In the figure, an instruction reading circuit 53 includes a task-specific PC storage unit 53a and a current IFAR (Instruction Fetch).
Address Register) 53b, incrementer 53c, next IFAR 53d, selector 53e, selector 53f, D
Equipped with ECAR (DECode Address Register) 53g,
At the time of task switching, the instruction read address is switched without overhead.

The task-specific PC storage unit 53a stores tasks 0 to
It has six address registers corresponding to 5 and holds a program count value for each task. Each program count value is a restart address of the corresponding task. At the time of task switching, the task management unit 58 and the decoder 55
, The program count value is read from the address register corresponding to the task to be executed next, and the program count value of the address register corresponding to the task currently being executed is updated to a new restart address. . At this time, the task to be executed next and the current task are respectively "nexttaskid (rd add
r) "signal (hereinafter also referred to as task ID)," taskid (wr a
ddr) "signal.

The program count values corresponding to tasks 0, 1, and 2 are shown in PC0, PC1, and PC2 of FIG. In the figure, (0-0) represents instruction 0 of task 0, and (1-4) represents instruction 4 of task 1. For example, PC0 is read when task 0 is resumed (instruction cycle t0), and is updated to the address of the instruction (0-4) when switching to the next task (instruction cycle t4).

Incrementor 53c, next IFAR 53
d, and the loop circuit composed of the selector 53e
This is a circuit for updating the instruction read address selected by 3e. The address output from the selector 53e is shown as IF1 in FIG. In the figure, for example, task 0
When switching from to the task 1, the selector 53e
In cycle t4, the instruction (1-0) address read from the task-specific PC storage unit 53a is selected, and in cycle t5 to t7, the incremented instruction address from the next IFAR 53d is selected.

The current IFAR 53b holds the selected output IF1 of the selector 53e with a delay of one cycle, and
2 is output as an instruction read address. In other words, it holds the instruction read address of the currently active task. The instruction read address of the current IFAR 53b is shown in IF2 of FIG. As shown in FIG.
Indicates an instruction address of a different task every four instruction cycles.

The DECAR 53g holds the address of the instruction held in the instruction register 54. That is, it indicates the instruction being decoded. The DEC in FIG.
It shows the address held in R53g. EX in FIG. 13 indicates an instruction address being executed. Selector 53
f selects a branch address when a branch instruction is executed or an interrupt occurs, and otherwise selects the address of the next IFAR 53d.

With the provision of such an instruction read circuit 53, the IO processor 5 performs four stages (IF1, IF2, DEC, EX) of pipeline processing as shown in FIG. The IF1 stage is a stage for selecting and updating a plurality of program count values.
The IF2 stage is a stage for reading an instruction. <1.3.3.1.2 Task Management Unit> FIG. 14 is a block diagram showing a detailed configuration of the task management unit 58. In the figure, the task management unit 58 is broadly divided into a slot manager that manages task switching timing and a scheduler that manages task order.

The slot manager includes a counter 58a,
It has a latch 58b, a comparator 58c, and a latch unit 58d, and outputs a task switching signal (chgtaskex) for instructing task switching to the instruction reading circuit 53 every four instruction cycles. Specifically, the latch 58b includes two flip-flops (FFs) that hold the lower two bits of the output of the counter 58a.
Circuit. The counter 58a outputs 3 bits obtained by incrementing the 2-bit output value of the latch 58b by +1 every clock indicating an instruction cycle. As a result, the counter 58a repeatedly outputs 1, 2, 3, and 4. The comparator 58c outputs a task switching signal (chgtaskex) to the instruction reading circuit 53 and the scheduler when the output value of the counter 58a matches the constant 4.

The scheduler has a task round management unit 5
8e, priority encoder 58f, latch 58g
Each time a task switching signal (chgtaskex) is output, the task id is updated, and the current task id and the task id to be executed next are output to the instruction reading circuit 53.
Specifically, the latch unit 58d and the latch 58g
Both hold the current task id in encoded form (3 bits). The value of the encoded format represents the task id.

When a task switching signal (chgtaskex) is input, the task round management unit 58e refers to the latch unit 58d and outputs a task id to be executed next in a decoded format (6 bits). . In the decoded format (6 bits), one bit corresponds to one task, and the bit position represents the task id. The priority encoder 58f converts the task id output from the task round management unit 58e from a decoded format to an encoded format. The latch unit 58
d, latch 58g, together with encoded task i
d is held one cycle later.

With this configuration, the task round management unit 5
8e receives a task switching signal (chgtaske) from the comparator 58c.
x) is output, the priority encoder 58
From f, change the id of the task to be executed next to "nexttaskid (rd
addr) "as a signal from the latch 58e to the current task id.
As a “taskid (wr addr)” signal. <1.4 Description of Operation> The operation of the video / audio processing apparatus 1000 according to the first embodiment configured as described above will be described.

In the input / output processing unit 1001, the MPEG stream input asynchronously from the stream input unit 1 is temporarily controlled by the input / output processor 5 via the buffer memory 2 and the memory controller 6.
And further via the memory controller 6 to F
It is held in the IFO memory 4. At this time, for the FIFO memory 4, the IO processor 5
(B) By executing task 3, compressed video data and compressed audio data are supplied according to the remaining amount. As a result, a fixed amount of compressed moving image data and compressed audio data are supplied to the FIFO memory 4 without excess and deficiency, so that the decoding processing unit 1002 can be dedicated to the decoding processing while being separated from asynchronous input / output. it can. The processing so far is performed by the input / output processing unit 1001 independently and in parallel with the decoding processing unit 1002.

On the other hand, in the decoding processing unit 1002,
The MPEG stream data held in the FIFO memory 4 is decoded by the processor 7, the code conversion unit 9, the pixel operation unit 10, and the pixel read / write unit 11 thereafter. FIFO
FIG. 15 is an explanatory diagram showing the decoding operation after the memory 4.
In the figure, the horizontal axis represents the time axis, and the header analysis of approximately one macroblock and the decoding of each block are shown. The vertical direction indicates that the decoding of each block is executed in a pipeline manner in each unit of the decoding processing unit 1002.

As shown in the figure, the processor 7 repeats the header analysis of the compressed video data and the decoding process on the compressed audio data in a time-division manner. That is, the processor 7 analyzes the header of one macroblock, notifies the code conversion unit 9, the pixel operation unit 10, and the pixel read / write unit 11 of the analysis result, and then starts decoding the macroblock for the code conversion unit 9. Instruct. Then processor 7
Performs decoding processing of the compressed audio data until an interrupt signal is notified from the code conversion unit 9. The decoded audio data is temporarily held in the internal memory 8 and further DMA-transferred to the external memory 3 by the memory controller 6.

The code converter 9 receives a macroblock decoding start instruction from the processor 7 and stores it in the buffer 200 for each block in the macroblock. At this time, the code conversion unit 9 changes the order of the write address to the buffer 200 according to the scan type of the block notified when the processor 7 analyzes the header. That is, the order of the write addresses is changed between the case of the zigzag scan and the case of the alternate scan.
Accordingly, the pixel operation unit 10 does not need to change the order of the read addresses, and can always read in the same order of the read addresses regardless of the scan type. The code conversion unit 9 repeats the above operation until the six blocks in the macroblock have been subjected to the VLD processing, and writes the data into the buffer 200. When the VLD of six blocks is completed, an interrupt is generated in the processor 7. This interrupt signal is a macroblock decode end signal End Of Macro Blo
ck (EOMB). The code converter 9 generates the EOMB by detecting the block end signal End Of Block (EOB) of the sixth block.

The pixel operation unit 10 performs IQ and IDCT on the block data stored in the buffer 200 in block units as shown in FIG. Store. The pixel read / write unit 11 performs, in parallel with the pixel operation unit 10, based on the block data in the buffer 201 and the motion vector notified by the header analysis by the processor 7, as shown in FIG.
As shown in FIG. 5, a rectangular area is cut out from a reference frame in the external memory 3 and block synthesis is performed. The result of block synthesis is stored in the external memory 3 via the FIFO memory 4.

The above is the operation when the block is not a skip macro block. In the case of a skip macro block, the code conversion section 9 and the pixel operation section 10 do not operate, and only the pixel read / write section 11 operates. If there is a skipped macroblock, the image is the same as the rectangular area in the reference frame, so that the image is copied by the pixel read / write unit 11 to the external memory 3 as a decoded image.

In this case, an interrupt signal from the code converter 9 to the processor 7 is generated as follows. That is, a signal indicating that the processor 7 has sent a control signal for starting a motion compensation operation to the pixel read / write unit 11,
The pixel read / write unit 11 calculates the logical product of the signal indicating that the motion compensation operation is possible and the signal indicating that the pixel is a skip macro block, and further calculates the logical product and the EOMB.
An interrupt signal is input to the processor 7 as a logical sum with the signal.

As described above, according to the video / audio processing apparatus of the first embodiment of the present invention, the MPEG stream input processing from the storage medium or the communication medium, the display image data to the display apparatus and the audio output apparatus, The input / output processing unit 1001 shares output processing of audio data and processing of supplying a stream to the decoding processing unit 1002, and the decoding processing unit 1002 shares decoding processing of compressed video data and compressed audio data. ing. As a result, the decoding processing unit 1002 is released from the processing that occurs asynchronously and can exclusively use the decoding processing.
As a result, since a series of processing of inputting, decoding, and outputting the MPEG stream is efficiently executed, full decoding of the MPEG stream (without dropped frames) can be realized without using a high-speed operation clock.

Further, it is desirable that the present video / audio processing apparatus be integrated into an LSI on one chip. In this case, 100MHz
The full decoding is possible with the following operation clock (actually 54 MHz). In this regard, the operating clock is 100
MHz High performance CP over 200MHz
U enables full decoding when the image size is small, but the manufacturing cost is high. On the other hand, the present video / audio processing apparatus is superior in terms of manufacturing cost and full decoding.

Further, the decoding processing section 1002 of the present video / audio processing apparatus has the following roles. In other words, the processor 7 is in charge of header analysis that requires a wide variety of condition judgments for both compressed video data and compressed audio data, and is also responsible for decoding audio compressed data. Since a large amount of routine calculation is required for block data of compressed video data, dedicated hardware (firmware) such as a code conversion unit 9, a pixel operation unit 10, and a pixel read / write unit 11 performs decoding processing. In charge of As shown in FIG. 15, the code conversion unit 9, the pixel operation unit 10, and the pixel read / write unit 11 are pipelined. In the pixel operation unit 10, IQ and IDCT can be processed in parallel. The pixel read / write unit 11 implements access to a reference frame in units of two blocks.
As a result, the efficiency of the compressed audio decoding process is improved, so that the hardware dedicated to video decoding can obtain high processing performance without using a high-speed clock. More specifically, a processing ability equal to or higher than that of the related art was obtained with a clock of about 50 to 60 MHz without using a high-speed clock exceeding 100 MHz. Therefore, it is not necessary to use a high-speed element, and the manufacturing cost can be reduced.

The basic unit of video decoding is a macro block unit in the processor 7, a block in the code conversion unit 9 and the pixel operation unit 10, and a pixel read / write unit 1.
Since two blocks are used in one, the capacity of the buffer buffer in video decoding can be minimized. <2 Second Embodiment> The video and audio processing apparatus according to the present embodiment is configured to perform a compression function (hereinafter, referred to as an encoding process) and a graphics function in addition to a decoding function of compressed stream data. ing. <2.1 Configuration of Video / Audio Processing Apparatus> FIG. 16 is a block diagram showing a configuration of a video / audio processing apparatus according to the second embodiment of the present invention.

The video / audio processing device 2000 includes a stream input / output unit 21, a buffer memory 22, a FIFO memory 24, an input / output processor 25, and a memory controller 2.
6, processor 27, internal memory 28, code conversion unit 2
9, pixel operation unit 30, pixel read / write unit 31, video output unit 12, audio output unit 13, buffer 200, buffer 2
01. The video / audio processing device 2000 has the following functions in addition to the functions of the video / audio processing device 1000 shown in FIG. That is, a compression function of video data and audio data and a graphics function of drawing polygon data are added.

Therefore, in the video / audio processing apparatus 2000, components having the same names as those in FIG.
Further, a function for performing a compression function and a graphics function is added. Hereinafter, description of the same points as in FIG. 4 will be omitted, and different points will be mainly described. The stream input / output unit 21
The difference is that it is bidirectional. That is, the MPEG memory is stored in the buffer memory 22 under the control of the input / output processor 25.
When the data is transferred, the transferred parallel data is converted into serial data and output to the outside as an MPEG data stream.

Buffer memory 22, FIFO memory 24
Is also bidirectional. I / O processor 25
Controls not only the data transfer on the routes (1) to (4) shown in the first embodiment, but also the transfer on the routes (5) to (8). (1) Stream input / output unit 21 → buffer memory 22 → memory controller 26 → external memory 3 (2) external memory 3 → memory controller 26 → FIFO memory 24 (3) external memory 3 → memory controller 26 → buffer memory 22
→ Video output unit 12 (4) External memory 3 → Memory controller 26 → Buffer memory 22
→ Audio output unit 13 (5) External memory 3 → Memory controller 26 → Internal memory 28 (6) External memory 3 → Memory controller 26 → Pixel read / write unit 31 (7) FIFO memory 24 → Memory controller 26 → External memory 3 (8 ) External memory 3 → memory controller 26 → buffer memory 22
→ The stream of the stream input / output unit 21 (5) (6) is the path of the original data when encoding processing of video data and audio data is performed, and (7) (8)
4 shows the path of an MPEG stream after compression.

First, the encoding process will be described.
It is assumed that data to be encoded is stored in the external memory 3. The video data in the external memory 3 is transferred to the pixel read / write unit 31 by controlling the memory controller 26 by the pixel read / write unit 31. The pixel read / write unit 31 performs a process of writing video data to the second buffer 201 and a difference image generation process. The difference image generation processing includes:
It consists of motion detection (calculation of a motion vector) for each block and generation of a difference image. Therefore, the pixel read / write unit 31
Has a motion detection circuit for detecting a motion vector by searching in a rectangular area similar to the encoding target block and a reference frame. Instead of the motion detection circuit, a motion estimation circuit for estimating a motion vector to be encoded using a motion vector of an already calculated block of an adjacent frame may be provided.

The pixel operation unit 25 receives the difference image data in block units, and performs DCT, IDCT, quantization processing (hereinafter, Q processing), and IQ. The video data thus quantized is stored in the buffer 200. The code conversion unit 29 receives the quantized data from the buffer 200 and performs variable length code processing (VLC). The variable-length coded data is stored in the first-in first-out memory 24, is stored in the external memory 3 through the memory controller 26, and the processor 27 adds header information for each macroblock.

The video data in the external memory 3 is transferred to the internal memory 28 via the memory controller 26. The processor 27 compresses the audio data in the internal memory 28 by a process of adding header information for each macroblock and a time division. As described above, the encoding process is performed in a path reverse to that of the first embodiment.

Next, the graphics processing will be described. The graphics processing is a three-dimensional image generation processing performed by a combination of rectangular figures called polygons. In this apparatus, processing for generating pixel data inside the polygon from pixel data at the vertex coordinates of the polygon is performed. First, the vertex data of the polygon is stored in the external memory 3.

The vertex data is stored in the internal memory 28 by the processor 27 controlling the memory controller 26. The processor 27 reads out the vertex data from the internal memory 28 and performs DDA (Digital Difference Analyze).
Is performed, and the data is written in the FIFO memory 24. The code conversion unit 29 performs a FIFO
The vertex data is read from the O memory 24 and the pixel operation unit 30
Transfer to

The pixel operation section 30 performs DDA processing and transmits the result to the pixel read / write section 31. The pixel read / write unit 31 performs Z buffer processing or α
The image data is written to the external memory 3 via the memory controller 26 by performing the blending process. <2.1.1 Pixel Operation Unit> FIG. 17 is a block diagram showing a configuration of the pixel operation unit 30.

In the figure, the same components as those of the pixel operation unit 10 shown in FIG. 7 are denoted by the same reference numerals, description thereof will be omitted, and the following description will focus on differences. The difference is that, as shown in the figure, the pixel operation unit 30 is different from the pixel operation unit 10 shown in FIG. That is, an instruction register 309 and a distribution unit 310 are added.

The reason why the execution units 501a to 501c have three surfaces is to improve the calculation performance. Specifically, in the graphics processing, color images RGB are independently processed in parallel. In the IQ and Q processing, the multiplier 5
02 is used to increase the speed. In the IDCT, time is reduced by using a plurality of multipliers 502 and a plurality of adders / subtracters 503. In the IDCT, there is an operation called a butterfly operation, and since there is a dependency between all the data on which the operation is based, a data line 103 for performing inter-unit communication of the execution units 501a to 501c is provided.

First instruction memory 506, second instruction memory 5
07 is DCT, Q processing, DD in addition to IDCT and IQ.
A microprogram for A is stored. FIG.
3 shows an example of the storage contents of the first instruction memory 506 and the second instruction memory 507. Compared to FIG. 8, a Q processing microprogram, a DCT microprogram, and a DDA microprogram are added.

Instruction pointer holding units 308a to 308c
Are provided corresponding to the execution units 501a to 501c, and have conversion tables for converting addresses input from the first program counter and outputting the converted addresses to the instruction register unit 309. The converted address is stored in the instruction register 30
9 means the register number. Further, the instruction pointer holding units 308a to 308c hold the later-described modify flags and output them to the instruction execution units 501a to 501c.

Regarding the conversion table, the instruction pointer holding units 308a, 308b, 308c, for example, when the input address is 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 The converted address is output as follows. Instruction pointer holding unit 308a: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 1
1,12 instruction pointer holding unit 308b: 2,1,4,3,6,5,8,7,10,9,1
2,11 instruction pointer holding unit 308c: 4,3,2,1,8,7,6,5,12,11,1
As shown in FIG. 23, the 0,9 instruction register unit 309 includes a plurality of registers for holding microinstructions, three selectors, and three output ports. The three selectors select the microinstruction of the register specified by the conversion address (register number) input from the instruction pointer units 308a, 308b, 308c. The three output ports are provided corresponding to the selectors, and each of the micro-instructions selected by the selectors is executed by the execution units 501 a to 501
01c. Three selectors and output ports are provided to supply different microinstructions to the three adder / subtracters 503 (or the three multipliers 502) at the same time. In the present embodiment, three output ports are connected to three adder / subtracters 503 and three multipliers
02 is selectively supplied.

For example, the instruction register section 309 includes registers R1 to R16 (register numbers 1 to 16).
The microprogram stored in the registers R1 to R16 represents a matrix operation process required in DCT and IDCT, and is stored so as to perform the same process in any of the three register numbers. That is, in the microprogram having the above three execution orders, the order of some microinstructions whose execution order is interchangeable is changed. This is because the execution units 501a to 501c execute the microprogram in parallel, so that the execution units 501a to 501c
This is to avoid resource interference such as contention of register (not shown) access between c. Also, the above matrix operation processing is performed by 8 ×
The contents include multiplication, transposition, and transfer of eight matrices.

Next, the microinstruction stored in each register of the instruction register section 309 is described as "op Ri, Rj, dest, (modify flag)" in the mnemonic format. However, the micro-instruction of the instruction register unit 309 is only “op and Ri, Rj and (modify flag)”. The “dest” part is specified from the instruction memories 506 and 507. Partial instruction pointer holding units 308a to 308 of "(modify flag)"
Specified from c.

Here, “op” is an operation code indicating a multiplication instruction, an addition / subtraction instruction, a transfer instruction, etc., and “R”
i, Rj "are operands. The multiplication instruction is an instruction to be executed by each multiplier 502 in the three execution units 501a to 501c, and the addition instruction and the transfer instruction are three execution units 501
Instructions executed by each of the multipliers 502 in a to c. "
“dest” indicates the storage destination of the operation result.
t ″ is not a register of the instruction register unit 309, but an instruction memory 506 (in the case of a multiplication instruction) or an instruction memory 507.
(In the case of an addition / subtraction instruction or a transfer instruction). This is to make the microprogram of the instruction register unit 309 common to the execution units 501a to 501c. If the transfer destination is specified by a register, execution units 501a to 501a
It is necessary to prepare an individual microprogram for each 1c, and the capacity of the microprogram is increased several times.

The "modify flag" is a flag indicating whether addition or subtraction is performed in an addition / subtraction instruction. This “modify flag” is stored in the instruction register 3
09, not from the register 09
8a to 8c. This is DCT, IDC
All elements in the constant matrix used for the matrix operation in T are "
Since the row (or column) of "1" and the row (or column) of which all elements are "-1" are included, by specifying the "modify flag" from the instruction pointers 308a-308c, the same micro It is possible to share programs.

When the three micro-instructions input from the instruction register unit 309 are addition / subtraction instructions, the distribution unit 310 outputs the "op and Ri, Rj" part and the "input from the instruction memory 506". The "dest" part and the "modify flag" input from the instruction pointer units 308a to 308c are distributed to the three adders / subtractors 503, and the micro instruction of the instruction memory 506 is simultaneously divided into three multipliers 502.
Distribute to Further, the distribution unit 310 includes the instruction register unit 3
When the three micro-instructions input from 09 are multiplication instructions, those “op and Ri, Rj” portions and the “dest” portion input from the instruction memory 506 are converted into three multipliers 502. , And the microinstructions in the instruction memory 507 are distributed to the three adders / subtracters 503. In other words, the distribution unit 310 controls the three adders / subtractors 503.
Are supplied from the instruction memory 507 to the instructions common to the three adders / subtractors 503, and the three adders / subtracters 50
Instruction register section 309
Are supplied to each of them. Similarly, the microinstruction supplied to the three multipliers 502 is a microinstruction supplied from the instruction memory 506 for an instruction common to the three multipliers 502, and an instruction for a multiplication instruction different in the three multipliers 502. The micro instruction from the register unit 309 is supplied to each.

According to such a configuration of the pixel operation unit 30, the storage capacity of the instruction memory 506 and the instruction memory 507 can be reduced. If it is assumed that the pixel operation unit 30 does not include the instruction pointer holding units 308a to 308c, the instruction register unit 309, and the distribution unit 310, the instruction memory 5
06 and the instruction memory 507 are all three execution units 50
To supply different microinstructions for 1a-c,
Three microinstructions must be stored in parallel.

FIG. 22 shows instruction pointer holding units 308a to 308a.
c, an example of contents stored in the instruction memory 506 and the instruction memory 507 when the instruction register unit 309 and the distribution unit 310 are not provided. In the figure, a 16-step microprogram is stored, and one microinstruction has a 16-bit length. In this case, since the instruction memory 506 and the instruction memory 507 record three micro-instructions in parallel, a total storage capacity of 1536 bits (16 steps × 16 bits × 3 × 2) is required.

On the other hand, the pixel operation unit 30 of this embodiment
FIG. 23 shows an example of the contents stored in the instruction pointer holding units 308a to 308c and the instruction register unit 309 in FIG. Also in this figure, a 16-step microprogram is stored, and one microinstruction has 16 bits. In the figure, the instruction pointer holding units 308a to 308c store 16 register numbers (4-bit length), respectively.
09 stores 16 microinstructions. In this case, the instruction pointer holding units 308a to 308c and the instruction register unit 309
Is 448 bits (16 steps × (12+
16)). As described above, in the pixel operation unit 30, the storage capacity of the microprogram can be significantly reduced. Actually, since “dest” and “modify flag” are separately issued, a recording capacity or a circuit corresponding to that is required. Also, instruction memories 506 and 50
7 designates "dest" in the microinstruction and issues multiplication instructions and addition / subtraction instructions common to the execution units 501a to 501c.
Has not been completely removed. If six output ports are provided in the instruction register unit 309, the instruction memory 506 and the instruction memory 507 can be deleted.

In FIG. 23, the instruction pointer holding unit 3
08a to 308c indicate that the value of the first program counter is 0
In the case of １５15, the conversion address (register number) is output, but it is not limited to this. For example, the conversion address may be output when the value of the first program counter is 32 to 47. In this case, the configuration may be such that an appropriate offset value is added to the value of the first program counter. As a result, an arbitrary address string indicated by the first program counter can be converted into a conversion address.

With the above configuration, in the present embodiment, not only the decoding processing of the compressed video data and the compressed audio data, but also
Encoding processing of video and audio data and graphics processing based on polygon data are possible. Further, the processing efficiency is improved by the parallel operation of the plurality of execution units. Moreover, the instruction register units 308a to 308
By changing the order of some of the micro-instructions in c, it is possible to avoid resource interference between a plurality of execution units, thereby further improving processing efficiency.

In the above embodiment, the configuration having three execution units is shown because it is advantageous in that each of RGB colors can be independently calculated. Further, the number of execution units may be any number as long as it is three or more. In the above embodiment, the video and audio processing devices 1000 and 2000
Are desirably implemented as one-chip LSIs. Further, the external memory 3 has been described as being external to the chip, but it may be configured to be built in one chip.

In the above embodiment, the stream input / output unit 1 (or the stream input / output unit 21) stores an MPEG stream (or video / audio data) with respect to the external memory. 3 may be stored. Further, in the above embodiment, the IO processor 5 performs the task switching every four instruction cycles, but may perform the task switching every plural instruction cycles other than the four instruction cycles. Also, the number of instruction cycles for task switching may be weighted in advance for each task and set to a different number of instruction cycles. The number of instruction cycles for each task may be weighted according to the priority and the urgency.

[0102]

The video / audio processing apparatus of the present invention is a video / audio processing apparatus for externally inputting and decoding a data stream including compressed audio data and compressed video data, and outputting the decoded data to an output device. Input / output processing means for performing input / output processing that occurs asynchronously due to an external factor; and decoding processing means for performing decoding processing mainly for decoding a data stream stored in a memory in parallel with the input / output processing. Video data and decoded audio data decoded by the decoding processing means are stored in a memory, and the input / output processing is performed by inputting the data stream asynchronously input from the outside and further storing the data stream in a memory Supplying the data stream stored in the memory to the decoding processing means; Read from the memory in accordance with the force device each output rate, is configured to perform and outputting them as input and output processing.

According to this configuration, in addition to the input / output processing means and the decoding processing means operating in parallel in a pipeline manner, the asynchronous processing and the decoding processing are shared between the input / output processing means and the decoding processing means. Therefore, the decoding processing means can be released from the processing that occurs asynchronously and can exclusively use the decoding processing. As a result, the video and audio processing apparatus efficiently executes a series of processing of stream data input, decode, and output, thereby enabling full decoding of stream data (without dropping frames) without using a high-speed operation clock. ing.

Further, the decoding processing means is a sequential processing mainly for condition determination on the data stream, and includes a header analysis of the compressed audio data and the compressed video data and a decoding of the compressed audio data. A sequential processing means for performing the processing, and a fixed form processing in parallel with the sequential processing. The routine processing may include a routine processing means for decoding the compressed video data excluding the header analysis of the compressed video data.

According to this configuration, it is possible to greatly improve the processing efficiency by eliminating the coexistence of the sequential processing having different processing characteristics and the routine processing suitable for the parallel processing in one unit. In particular, the processing efficiency of the routine processing means can be improved. This is because, in the present video / audio processing apparatus, since the standard processing means is released from the asynchronous processing and the sequential processing described above, it can exclusively perform various standard operations required for decoding the compressed video data. As a result, high processing performance can be obtained without using a high-speed operation clock.

Further, the input / output processing means includes an input means for inputting an asynchronous data stream from the outside, a video output means for outputting decoded video data to an external display device, and a video output means for decoding to an external audio output device. And a processor for executing the first to fourth tasks stored in the instruction memory while switching, the first task transferring a data stream from an input unit to the memory. The second task is a program that supplies a data stream from the memory to a decoding processing unit, and the third task is a program that outputs decoded video data from the memory to a video output unit. The fourth task is a program for outputting decoded audio data from the memory to an audio output unit. It may be configured to be the.

Here, the processor stores each task program using a program counter section having at least four program counters corresponding to the first to fourth tasks and an instruction address indicated by one program counter. An instruction fetch unit that fetches an instruction from the instruction memory; an instruction execution unit that executes the instruction fetched by the instruction fetch unit; And a task control unit for controlling the operation.

According to this configuration, in what range the input rate and the input cycle of the stream data determined by the external device and the output rate and the output cycle of the video data and the audio data determined by the external display device and the external audio output device are set. Even so, there is an effect that the response delay to the input / output request is extremely small. Further, the video and audio processing apparatus of the present invention includes: an input unit for inputting a data stream including compressed audio data and compressed video data; and a sequential processing mainly on condition determination for the data stream. A sequential processing unit for analyzing header information added to a predetermined block unit in the inside and decoding compressed audio data in the data stream; and a routine process mainly including a routine operation, using a result of the header analysis. Routine processing means for decoding the compressed video data in the data stream in units of predetermined blocks in parallel with the sequential processing, wherein the sequential processing unit performs the routine processing when the header analysis of the predetermined block is completed. To start decoding of the predetermined block, and when receiving a notification of the end of decoding of the predetermined block from the routine processing means, the next predetermined block is notified. It may be configured to start the header analysis.

According to this configuration, the sequential processing means is in charge of header analysis which requires a wide variety of condition judgments for compressed video data and compressed audio data, and is also responsible for decoding of audio compressed data. On the other hand, the routine processing means performs processing on the block data of the compressed video data.
Responsible for a large amount of routine calculations. Due to such role assignment, the sequential processing means performs overall audio decoding, which requires less computation than video decoding, header analysis of compressed video data, and control of routine processing means. Under the control, the routine processing unit performs a routine operation exclusively,
Lean and efficient processing can be realized. Therefore, the processing capability can be obtained without operating at a high frequency, and the manufacturing cost can be reduced. In addition, the sequential processing means sequentially performs overall audio decoding, header analysis of the compressed video data, and control of the standard processing means, so that it can be constituted by one processor.

Further, the routine processing means includes a data conversion means for performing variable length decoding of the compressed video data in the data stream in accordance with the instruction of the sequential processing means, and a predetermined operation for the video block obtained by the variable length decoding. Computing means for performing inverse quantization and inverse discrete cosine transform by applying
Synthesizing means for performing motion compensation processing by synthesizing the video block after inverse discrete cosine transform and the decoded block to restore video data, wherein the sequential processing means is subjected to variable-length decoding by data conversion means. Obtaining means for obtaining the obtained header information, analyzing means for analyzing the obtained header information, notifying means for notifying the routine processing means of a parameter obtained as an analysis result, and compression in the data stream inputted by the input means. The audio decoding means for decoding audio data, and upon receiving an interrupt signal notifying the completion of decoding of the predetermined block from the routine processing means, stops the operation of the audio decoding means and activates the acquisition means, and the notifying means Control means for instructing the data conversion means to start variable-length decoding of a video block when notified. It may be.

According to this configuration, the sequential processing means decodes the audio after performing header analysis for each predetermined block such as a macro block, and when the decoding of the predetermined block is completed by the standard processing means, the header analysis of the next block is performed. To start. Thus, the sequential processing means repeats the header analysis and the audio decoding in a time-division manner, so that it can be realized at low cost with one processor. Further, since the routine processing means does not need to perform a wide variety of condition determination processing, it can be made into dedicated hardware (or hardware and firmware) at low cost.

Here, the arithmetic means has a first buffer having a storage area corresponding to one block, and the data conversion means has a variable length decoding means for performing variable length decoding of the compressed video data in the data stream. First address table means for storing a first address sequence in which the addresses of the storage area of the first buffer are arranged in zigzag scan order;
A second address table for storing a second address sequence in which the addresses of the storage area of the first buffer are arranged in an alternate scan order; It may be configured to have a writing unit for writing the obtained block data to the first buffer.

According to this configuration, the writing means can write the block data in the storage area of the first buffer in accordance with both the zigzag scan and the alternate scan. Therefore, the arithmetic means does not need to change the order of the read addresses when reading the block data from the storage area of the first buffer, and can always read in the same order of the read addresses regardless of the scan type.

Further, the analyzing means calculates the quantization scale and the motion vector based on the header information, and the notifying means notifies the calculating means of the quantization scale and the motion vector to the synthesizing means. May be. According to this configuration, the calculation of the motion vector can be assigned to the sequential processing unit, and the combining unit can routinely perform the motion compensation process using the calculated motion vector. .

The arithmetic means includes first and second control storage units for storing microprograms, respectively.
A first program counter for designating a first read address, a second program counter for designating a second read address, and a second control storage by selecting one of the first and second read addresses in the control storage unit; A selector for outputting to the unit, a multiplier and an adder, and an execution unit for performing inverse quantization and inverse discrete cosine transform in block units by microprogram control by the first and second control storage units. It may be configured.

According to this configuration, the microprogram (firmware) does not need to perform a wide variety of condition determination processing, and only realizes routine processing. Therefore, the program size is small, the program is easy to create, and cost reduction is achieved. Are suitable. Moreover, the multiplier and the adder can be operated independently and in parallel using two program counters.

Further, when the second read address is selected by the selector, the execution section performs the processing using the multiplier and the processing using the adder independently and in parallel, and the first read address is determined by the selector. When selected, the processing using the multiplier and the processing using the adder may be performed in conjunction with each other. According to this configuration, the idle time of the multiplier and the adder can be reduced, and the processing efficiency can be improved.

Here, the calculating means further includes a first buffer for holding the video block from the data converting means, and a second buffer for holding the block subjected to the inverse discrete cosine transform by the execution unit. The first control storage unit stores a microprogram for performing an inverse quantization process and a microprogram for performing an inverse discrete cosine transform, and the second control storage unit includes a microprogram for performing an inverse discrete cosine transform and an inverse discrete cosine transform. And a microprogram for transferring the video block to the second buffer, wherein the execution means transfers the inverse discrete cosine transformed video block to the second buffer, and performs the inverse quantization of the next video block. Are performed in parallel, and a process of performing an inverse discrete cosine transform of the inversely quantized video block is performed in conjunction with the multiplier and the adder. It may be.

According to this configuration, since the inverse quantization process and the transfer process to the second buffer are executed in parallel, the processing efficiency can be improved. Further, the input means further inputs polygon data, the sequential processing means further analyzes the polygon data and calculates vertex coordinates and edge inclinations of the polygon, and the routine processing means further calculates the polygon data. The image data of the polygon may be generated in accordance with the vertex coordinates and the inclination.

According to this configuration, the sequential processing unit is in charge of analyzing polygon data, and the standard processing unit is in charge of standard image data generation processing. The video / audio processing apparatus can perform graphics processing for efficiently generating image data from polygon data. Here, the first,
The second control storage unit further stores a microprogram for performing scan conversion by the DDA algorithm, and the execution unit further performs scan conversion by microprogram control based on the vertex coordinates and the inclination calculated by the sequential processing unit. It may be configured to do so.

According to this configuration, generation of image data can be easily realized by a scan conversion microprogram in the first and second control storage units. Further, the synthesizing means generates a difference block representing a difference image from the video data to be further compressed, the second buffer holds the further generated difference image, and the first control storage unit further performs a discrete cosine transform. The second control storage unit further stores a microprogram for performing discrete cosine transform and a microprogram for transferring the video block subjected to discrete cosine transform to the first buffer. The means further performs discrete cosine transform and quantization on the difference block held in the second buffer and transfers the result to the first buffer, and the data conversion means further performs variable length coding on the block in the first buffer. And the sequential processing means further performs header information on the predetermined block which has been variable-length coded by the data conversion means. The may be configured to add.

According to this configuration, the routine processing unit is responsible for quantization and discrete cosine transformation as routine processing, and the sequential processing unit is responsible for processing that requires a condition determination (addition of header information). In this case, the present video / audio processing apparatus can efficiently execute the encoding process from image data to compressed video data without using a high-speed clock. Also,
The arithmetic means includes first and second control storage units each storing a microprogram; a first program counter for designating a first read address in the first control storage unit;
A second program counter that specifies a read address, a selector that selects one of the first read address and the second read address and outputs the selected read address to the second control storage unit, and a multiplier and an adder, respectively. 1. A plurality of execution units that execute inverse quantization and inverse discrete cosine transform in units of blocks by microprogram control by a second control storage unit, and each execution unit shares partial blocks obtained by dividing blocks It may be configured to perform the processing.

According to this configuration, since a plurality of execution units execute operation instructions in parallel, a large number of routine operations can be efficiently executed in parallel at the pixel level. Further, the arithmetic means is further provided corresponding to each execution unit, each conversion table is a plurality of address conversion table holding a conversion address partially changed the address order corresponding to a predetermined address sequence, An instruction register group consisting of a plurality of registers for storing individual microinstructions constituting a microprogram for realizing a predetermined operation in association with a translation address, and a first and second control storage unit and a plurality of execution units; A switching unit that switches a microinstruction output from the first control storage unit or the selector to each execution unit to a microinstruction in an instruction register and outputs the microinstruction to a plurality of execution units. (2) If the read address is an address in the predetermined address string, the address is converted according to each of the address conversion tables. It is converted to. The instruction register group may be configured to output a micro instruction corresponding to each translation address output from the translation table.

According to this configuration, while a plurality of execution units execute microprograms in parallel, resource interference such as access competition between execution units can be avoided, and processing can be performed more efficiently. Here, each of the conversion tables should be added or subtracted in accordance with a microinstruction output indicating addition and subtraction in the register while the first program counter outputs the first read address in the predetermined address string. Outputting a flag indicating a perception to the plurality of execution units, wherein each execution unit performs addition and subtraction according to the flag, and the flag is set according to a microinstruction of the second control storage unit. Is also good.

According to this configuration, since the conversion table specifies whether to perform addition or subtraction by a microinstruction, the same microprogram can be shared in two ways, further reducing the total capacity of the microprogram. Thus, the hardware scale can be reduced, and the cost can be reduced. Further, the second control storage unit further stores a micro instruction execution result storage destination along with the micro instruction output in the register while the first program counter outputs the first read address in the predetermined address sequence. May be output to the plurality of execution units, and each execution unit may store the execution result according to the storage location information.

According to this configuration, since the storage destination information can be specified separately from the microprogram in the instruction register group, the microprogram can be shared in different processing, for example, partial processing of matrix operation. As a result, the total capacity of the microprogram can be further reduced, and the hardware scale can be reduced, and the cost can be reduced.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of a decoding process by a video / audio decoder according to a first conventional technique.

FIG. 2 is an explanatory diagram of a decoding process by a two-chip decoder according to a second conventional technique.

FIG. 3 is a block diagram illustrating a schematic configuration of the image processing apparatus according to the first embodiment of the present invention.

FIG. 4 is a block diagram illustrating a configuration of an image processing apparatus according to the first embodiment of the present invention.

FIG. 5 is a diagram showing an MPEG stream in a hierarchical manner and showing operation timing of each section of the image processing apparatus.

FIG. 6 is a diagram showing an analysis of a macroblock header by a processor 7 and contents of control to other units.

FIG. 7 is a block diagram illustrating a configuration of a pixel operation unit 10.

FIG. 8 shows a first instruction memory 506 and a second instruction memory 50;
7 shows an example of the microprogram stored in FIG.

FIG. 9 is a diagram showing operation timings of the pixel operation unit 10;

FIG. 10 is a block diagram showing a detailed configuration of a pixel read / write unit 11;

FIG. 11 is a block diagram showing a configuration of an IO processor 5.

FIG. 12 is a block diagram showing a detailed configuration example of an instruction reading circuit 53.

FIG. 13 is a time chart showing the operation timing of the IO processor 5;

FIG. 14 is a block diagram illustrating a configuration of a task management unit.

FIG. 15 is an explanatory diagram showing a decoding operation after the FIFO memory 4;

FIG. 16 is a block diagram illustrating a configuration of an image processing apparatus according to a second embodiment of the present invention.

FIG. 17 is a block diagram illustrating a configuration of a pixel operation unit 30.

FIG. 18 shows a first instruction memory 506 and a second instruction memory 50
7 shows an example of the storage content of No. 7.

FIG. 19 is a block diagram showing a configuration of a code conversion unit 9;

FIG. 20 shows a block storage area for storing 8 × 8 spatial frequency data and a zigzag scan route.

FIG. 21 shows a block storage area for storing 8 × 8 spatial frequency data and a route of an alternate scan.

FIG. 22 shows an example of the contents stored in the instruction memory 506 and the instruction memory 507 when the instruction pointer holding units 308a to 308c, the instruction register unit 309, and the distribution unit 310 are not provided.

FIG. 23 shows an example of the contents stored in the instruction pointer holding units 308a to 308c and the instruction register unit 309.

[Explanation of symbols]

 Reference Signs List 1 stream input unit 2 buffer memory 3 external memory 4 FIFO memory 5 input / output processor 5a DMAC 6 memory controller 7 processor 8 internal memory 9 code conversion unit 10 pixel operation unit 12 video output unit 13 audio output unit 14 host I / F unit 1000 Video / audio processing device 1001 Input / output processing unit 1002 Decoding processing unit 1003 Sequential processing unit 1004 Standard processing unit

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (Reference) H03M 7/30 H04N 7/133 Z 5J064 H04N 7/30 G10L 9/18 M (72) Inventor Shuzo Kiyohara Kadoma, Osaka 1006 Kadoma, Ichidai-ji Matsushita Electric Industrial Co., Ltd. FD23 FD25 GA16 5C059 KK10 KK15 LA06 MA23 MB18 MC11 SS13 SS26 UA02 UA05 UA36 UA37 UA38 UA39 5D045 DA20 5J064 AA01 BA16 BC01 BC02 BC04 BC09 BC25 BD03

Claims (12)

[Claims] 1. A first and a second control storage unit each storing a microprogram, a first program counter for designating a first read address in the first control storage unit, and a second program counter for designating a second read address. A program counter, a selector for selecting one of the first read address and the second read address and outputting the selected read address to the second control storage unit, and an execution unit for executing an operation under microprogram control by the first and second control storage units An arithmetic device comprising: 2. The arithmetic unit according to claim 1, wherein the execution unit has a first arithmetic unit and a second arithmetic unit, and when the second read address is selected by the selector, the first arithmetic unit and the second arithmetic unit. An arithmetic device, wherein arithmetic operations are performed in parallel in two arithmetic units, and when the first read address is selected by the selector, the first arithmetic unit and the second arithmetic unit are operated in conjunction with each other. 3. The arithmetic unit according to claim 1, wherein the execution unit has a multiplier and an adder, and performs inverse quantization in block units by microprogram control by first and second control storage units. An arithmetic unit for performing an inverse discrete cosine transform. 4. The arithmetic unit according to claim 3, wherein the execution unit independently performs a process using the multiplier and a process using the adder when the second read address is selected by the selector. A processing using a multiplier and a processing using an adder are performed in conjunction with each other when the first read address is selected by the selector. 5. The arithmetic unit according to claim 4, further comprising: a first buffer for holding a video block from the data conversion means; and a second buffer for holding a block subjected to inverse discrete cosine transform by an execution unit. Wherein the first control storage unit stores a microprogram for performing inverse quantization and a microprogram for performing inverse discrete cosine transform, and the second control storage unit includes a microprogram for performing inverse discrete cosine transform. The inverse discrete cosine transformed video block is
A microprogram to be transferred to a buffer, wherein the execution means executes, in parallel, a process of transferring the inverse discrete cosine transformed video block to the second buffer and a process of inversely quantizing the next video block. An arithmetic unit for performing a process of performing an inverse discrete cosine transform on the inversely quantized video block in conjunction with a multiplier and an adder. 6. The arithmetic device according to claim 5, wherein the synthesizing unit further generates a difference block representing a difference image from the video data to be compressed, and wherein the second buffer further generates the difference block. The first control storage unit further stores a microprogram for performing discrete cosine conversion and a microprogram for performing quantization processing, and the second control storage unit further includes a microprogram for performing discrete cosine conversion. The discrete cosine transformed video block is
A microprogram to be transferred to a buffer, wherein the execution means further executes discrete cosine transform and quantization on the difference block held in the second buffer and transfers the result to the first buffer; The means further performs variable length coding on the block of the first buffer, and the sequential processing means further adds header information to the predetermined block which has been subjected to the variable length coding by the data converting means. An arithmetic unit characterized by the following. 7. The arithmetic device according to claim 3, further comprising: generating image data of the polygon according to the vertex coordinates of the polygon and the inclination of the edge. 8. The arithmetic device according to claim 7, wherein the first and second control storage units further store a microprogram for performing scan conversion by a DDA algorithm, and the execution unit stores the vertex coordinates. An arithmetic unit for performing scan conversion by microprogram control based on the inclination and the inclination. 9. A first and second control storage unit for respectively storing a microprogram, a first program counter for designating a first read address in the first control storage unit, and a second program for designating a second read address. A first counter for selecting one of the first read address and the second read address and outputting the selected address to the second control storage unit; a multiplier and an adder; And a plurality of execution units for performing inverse quantization and inverse discrete cosine transform in block units by microprogram control according to the above. Each execution unit shares and processes partial blocks obtained by dividing blocks. Arithmetic unit. 10. The arithmetic unit according to claim 9, wherein said arithmetic means is further provided corresponding to each execution unit, and each conversion table partially corresponds to a predetermined address sequence in an address order. A plurality of address conversion tables for storing a conversion address in which a plurality of addresses are exchanged; an instruction register group including a plurality of registers for storing individual microinstructions constituting a microprogram for realizing a predetermined operation in correspondence with the conversion address; A micro instruction provided between the second control storage unit and the plurality of execution units and output from the first control storage unit or the selector to each execution unit is switched to a micro instruction in an instruction register and output to the plurality of execution units. When the first read address or the second read address is an address in the predetermined address sequence, the address is Serial is converted into translated address by the address conversion table. The arithmetic unit according to claim 1, wherein the instruction register group outputs microinstructions corresponding to each conversion address output from a conversion table. 11. The arithmetic unit according to claim 10, wherein each of the conversion tables further includes an addition / subtraction operation in the register while the first program counter outputs a first read address in the predetermined address string. A flag indicating whether to add or subtract is output to the plurality of execution units in accordance with the output of the microinstruction indicating that each execution unit performs addition / subtraction according to the flag. An arithmetic unit, which is set according to a micro instruction in a storage unit. 12. The arithmetic unit according to claim 10, wherein the second control storage unit further stores the first program counter in the register while the first program counter outputs the first read address in the predetermined address string. With the micro instruction output of
An arithmetic unit which outputs information indicating a storage location of a microinstruction execution result to the plurality of execution units, wherein each execution unit stores the execution result according to the storage destination information.