JP3380236B2 - Video and audio processing device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to digital signal processing.
And belongs to the technical field of
Decompression of audio data, compression of video and audio data, graph
The present invention relates to an image processing apparatus that performs a pix process. [0002] In recent years, compression / expansion of digital moving image data has been performed.
Long technology has been established and LSI technology has improved
Decompression of compressed video and audio data
Longer decoder, encoder to compress video and audio data
And graphics processing that performs graphics processing.
Any of various video and audio processing apparatuses is regarded as important. As a first conventional technique, MPEG (Moving)
Picture Experts Group) standard compressed video and audio data
Video / audio decoder for expanding data (Japanese Patent Laid-Open No.
29). This video / audio decoder has one signal processor.
Video decoding and audio decoding using the processing unit
I do. FIG. 1 shows the decoding by the video / audio decoder.
FIG. 4 shows an explanatory diagram of the processing. The vertical axis in the figure is time, and the horizontal axis is calculated
It represents the quantity. When viewed along the vertical axis, video decoding
And audio decoding are performed alternately. This is a common
To decode both video and audio with hardware
is there. As shown in the figure, video decoding consists of sequential processing and block
Process. For sequential processing, data other than blocks
Code, ie header analysis of MPEG stream
This is a process that requires a wide range of condition judgments, and its operation
The amount is small. Block decoding is an MPEG stream
Is decoded and further dequantized in block units,
This is the process of performing inverse DCT (discrete cosine transform), and its operation
The amount is large. As shown in the figure, audio decoding is also diverse.
Sequential processing similar to the above, which requires
Decoding of the data itself. Audio data
The decoding process of the main unit requires higher accuracy than image data.
Must be processed and processed within a limited time
Therefore, it is necessary to perform high-speed processing with high accuracy,
Is big. [0005] As described above, the first prior art is one chip.
And can be implemented with as little hardware as one chip.
It achieves efficient audio-video decoding. Second prior art
As a technique, there is a two-chip decoder. One chip is
Video decoder, one other chip used as audio decoder
Can be FIG. 2 shows decoding by a two-chip decoder.
FIG. 4 shows an explanatory diagram of the processing. Both video decoder and audio decoder
Processing, which includes a number of conditions such as header analysis,
Block decoding processing, which mainly decodes the data
Do. Process video decoder and audio decoder independently
Therefore, the performance of each chip is higher than that of the first prior art.
May be lower. [0006] However, the above-mentioned prior art
According to the technology, there were the following problems. First conventional technique
According to the technique, the signal processing unit decodes both video and audio
High processing capacity is required.
In other words, it operates using a high-speed clock of 100 MHz or more.
Cost is high for semiconductors for consumer use
There is a problem. Processing without using high-speed clock
VLIW (Very Long Instruction)
Word) Processor etc. can not be considered.
However, the cost of the VLIW processor itself is high and
In addition, unless a processor that performs sequential processing is used separately,
There is a problem that the processing becomes inefficient. According to the second prior art, two processors are used.
However, there is a problem that the cost is high because of the use of the heat sink. Toes
Video and audio processors
Using low-cost general-purpose inexpensive processors
Can not. Because a video processor is
Do you need the ability to process image data in real time?
It is. The audio processor is also used for the video processor.
Although it does not require as much computational complexity as
Data requires higher accuracy than image data.
is there. Therefore, inexpensive or low-processing
Is required for both video and audio
Does not meet processing capacity. A digital (satellite) broadcasting tuner
(STB (Set Top Box)) or DVD (D
igital Versatile / Video Disc)
The video / audio processing device is used in the AV decoder
Received from the broadcast wave or disc
Input the MPEG stream read from the
Decode the EG stream and finally display,
Output the video signal and audio signal to the speaker, etc.
The amount of series of processing required by the system is enormous. Most
Recently, such a huge series of processes have been executed efficiently.
There is an increasing demand for video and audio processing devices. According to the present invention, compressed image and compressed audio data are
Stream data input, decode, and output
And perform high processing without operating at high frequency.
With the ability to reduce manufacturing costs.
It is an object to provide an image and sound processing device. Again
The other purpose of this is to decode compressed video data,
Data encoding and graphics processing at low cost
To provide a video and audio processing device. [0010] To solve the above-mentioned problems,
The video and audio processing apparatus of the present invention is capable of
A data stream including video data is input from outside,
Decode and output the decoded data to an output device
An input / output device that occurs asynchronously due to external factors
Input / output processing means for performing processing, and
To decode the data stream stored in memory.
Decoding means for performing main decoding processing.
The video data decoded by the decoding processing means.
Data and decoded audio data are stored in memory,
The input / output processing is performed on the data input asynchronously from the outside.
Input data stream and store it in memory
And decode the data stream stored in the memory.
External display device and audio output device.
Read from memory according to output rate
And output to them as input / output processing.
It is configured as follows. According to this configuration, the input / output processing means and the deco
The code processing means operates in parallel in a pipeline
In addition, asynchronous processing and decoding processing are performed by input / output processing means.
And decoding processing means.
The means is released from the processing that occurs asynchronously and
Be able to devote themselves to reasoning. As a result, this video and audio processing
The devices are called stream data input, decode, and output.
Stream processing is performed efficiently, so stream data
Full decoding (no dropped frames) of high-speed operation
This is possible without using a lock. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A video / audio processing apparatus according to the present invention will be described.
The embodiments will be described in the following sections. 1 First embodiment 1.1 Schematic configuration of video / audio processing device 1.1.1 Input / output processing unit 1.1.2 Decoding processing unit 1.1.2.1 Sequential processing unit 1.1.2.2 Standard processing unit 1.2 Configuration of video / audio processing device 1.2.1 Input Configuration of output processing unit 1.2.2 Decoding processing unit 1.2.2.1 Sequential processing unit 1.2.2.2 Routine processing unit 1.3 Detailed configuration of each unit 1.3.1 Processor 7 (sequential processing unit) 1.3.2 Routine processing unit 1.3.2.1 Code conversion unit 1.3.2.2 Pixel operation unit 1.3.2.3 Pixel read / write unit 1.3.3 Input / output processing unit 1.3.3.1 IO processor 1.3.3.1.1 Instruction readout circuit 1.3.3.1.2 Task management unit 1.4 Operation description 2 Second embodiment 2.1 Configuration of Video / Audio Processing Device 2.1.1 Pixel Calculation Unit <1. First Embodiment> Video / audio processing in this embodiment
The control device is a satellite broadcast receiver (STB: Set TopBox)
), DVD (Digital Versatile Disc) playback device,
Compressed video provided in DVD-RAM recording / playback devices, etc.
/ MP from audio or DVD as audio data
EG stream is input and decompressed (hereinafter simply decoded
To output video and audio signals to external output
Output to the device. <1.1 Schematic Configuration of Video / Audio Processing Apparatus> FIG.
Configuration of Video / Audio Processing Device in First Embodiment of FIG.
FIG. The video / audio processing apparatus 1000 performs input / output processing.
Unit 1001, decode processing unit 1002, memory controller
And input / output processing and decoding processing are separated.
It is configured to be performed in parallel. Also, external memory
3 is for MPEG stream and audio data after decoding
Temporary working memory, decoded video data
It is used as a frame memory for storing data. <1.1.1 Input / output processing unit> The input / output processing unit 1001
Occurs asynchronously with the internal operation of the image / audio processing device 1000.
Perform input / output processing. This input / output processing is performed by (a) external
Input an MPEG stream that is input asynchronously
Temporary storage in the external memory 3, (b) external memory
Decodes the MPEG stream stored in
002, (c) decoded video data
Data from the external memory 3 and the external table.
Display device and audio output device (not shown)
The content should be output together. <1.1.2 Decoding processing unit> Decoding processing unit 1002
Is independent and parallel to the operation of the input / output processing unit 1001,
MPEG stream supplied by the input / output processing unit 1001
Ream decoding, video data and audio after decoding
The data is stored in the external memory 3. MPEG stream
Decoding processing requires a large amount of computation and a wide variety of processing contents
Therefore, the decoding processing unit 1002
3. Equipped with a standard processing unit 1004, and a wide variety of condition judgments
Mainly for sequential processing and routine mass processing
And parallel processing separated from routine processing suitable for parallel operation
It is configured to be. Here, the sequential processing is MP
EG stream header analysis etc., header detection
And a number of conditional determinations such as header content determination. Again
Type processing is performed in units of blocks consisting of a predetermined number of pixels.
Calculation, so it can be used for pipeline-like parallel processing.
Suitable and completely different data (pixels)
For parallel processing such as vector operation to perform the same operation
Are suitable. <1.1.2.1 Sequential processing unit> The sequential processing unit 1003
Compressed audio data and pressure supplied from the force processing unit 1001
The header analysis of the compressed video data and the routine processing unit 1004
Control to be activated for each black block and compressed audio data
The code processing is performed as the above sequential processing. Header analysis
Is the macroblock header in the MPEG stream
And decoding of motion vectors. Block here
Represents an image composed of 8 * 8 pixels. Macro block
Is composed of four luminance blocks and two chrominance blocks.
You. The motion vector is the 8 * 8 pixel in the reference frame.
This is a vector pointing to a rectangular area, and the block
Indicates which rectangular area in the frame the difference was taken from
You. <1.1.2.2 Standard processing unit> The standard processing unit 1004
Activation of decoding for each macroblock from the processing unit 1003
In response to the instruction, the audio decoding process of the
At the same time, the macroblock decoding process is
Perform as a matter of course. This decoding process is for decoding variable-length codes.
(VLD: Variable Length code Decoding), inverse quantization
(IQ: Inverse Quantization), inverse discrete cosine transform (I
DCT: Inverse Discrete Cosine Transform), motion supplement
Reimbursement (MC: Motion Compensation) must be performed in the same order.
It is assumed. In the motion compensation, the routine processing unit 1004
External memory as a frame memory for the block after decoding
3 via the memory controller 6. <1.2 Configuration of Video / Audio Processing Device> FIG.
FIG. 2 is a block diagram showing a more detailed configuration of the processing device 1000.
You. <1.2.1 Configuration of input / output processing unit>
The processing unit 1001 includes a stream input unit 1, a buffer memory
2. Input / output processor 5 (hereinafter abbreviated as IO processor 5)
), DMAC (Direct Memory Access Controller)
5a, video output unit 12, audio output unit 13, host I / F
And a unit 14. The stream input unit 1 is connected to a serial
MPEG data stream input to
(Hereinafter referred to as MPEG data). That
At this time, the stream input unit 1
From GOP (Group Of Picture: contains one I picture,
MPEG data stream equivalent to about 0.5 seconds of video
The start code is detected by the IO processor.
Notify service 5. By this notification, the converted MPEG data
The buffer memory 2 is controlled by the IO processor 5.
Is forwarded to The buffer memory 2 includes a stream input unit 1
Buffer for temporarily storing MPEG data transferred from
Memory. MPE held in buffer memory 2
The G data is further stored under the control of the input / output processor 5.
Transferred to the external memory 3 via the memory controller 6
You. The external memory 3 is an SDRAM (Synchronous Dynami
c Random Access Memory) chip
Transferred from the memory 2 via the memory controller 6.
MPEG data temporarily stored. In addition, external
The memory 3 stores decoded video data (hereinafter referred to as frame data).
) And the decoded audio data. The input / output processor 5 has a stream input unit.
1, buffer memory 2, external memory 3 (memory control
), Data input between the FIFO memory 4
Control the output. That is, the routes shown in (1) to (4) below
Controls data transfer (DMA transfer). (1) Stream input unit 1 → buffer memory 2 → memory controller
6 → External memory 3 (2) External memory 3 → Memory controller 6 → FIFO memory 4 (3) External memory 3 → Memory controller 6 → Buffer memory 2 → V
Video output unit 12 (4) External memory 3 → memory controller 6 → buffer memory 2 → sound
Voice output unit 13 In these paths, the input / output processor 5
Video data and audio data in the
Control the transmission. (1) and (2) are MPEG data before decoding.
Data transfer path. Ingress and egress in the transfer path of (1) and (2)
The power processor 5 converts the compressed video data and the compressed audio data into
Separately. (3) and (4) are the images after decryption, respectively.
This is a transfer path for image and audio data. Video and audio after decoding
Data is stored on an external display device (not shown) and audio output device (
Outer) It is transferred according to each output rate. The DMAC 5a has a stream input unit 1,
The video output unit 12, the audio output unit 13, and the buffer memory 2
DMA transfer between the buffer memory 2 and the external memory 3
Between the external memory 3 and the FIFO memory 4
DMA transfer between I / O processors under the control of IO processor 5
I do. The video output unit 12 is connected to an external display device (CRT)
Etc.) (for example, the frequency of the horizontal synchronization signal Hsync).
A data request to the input / output processor 5 according to the
The data is input by the input / output processor 5 through the transfer path of (3)
The input video data is output to the display device. The audio output unit 13 is provided for an external audio output device.
Data request to input / output processor 5 according to output rate
To the transfer path of (4) above by the input / output processor 5.
Audio data input from the audio output device (D / A
Output to a combination of barter, audio amplifier, speaker, etc.)
I do. The host I / F unit 14 includes an external host processor,
For example, in the case of a DVD reproducing apparatus, the entire control is performed.
An interface for communicating with the processor.
You. In this communication, the host processor sends an MPEG stream.
Start, stop, fast-forward playback, reverse playback, etc.
Instructions are sent. <1.2.2 Decoding unit> Decoding unit 10 in FIG.
02 is a FIFO memory 4, a sequential processing unit 1003, a fixed form
Processing unit 1004, and the input / output processing unit 1001
Decoding of MPEG data supplied via the FO memory 4
Perform the loading process. Also, the sequential processing unit 1003
The system includes a processor 7 and an internal memory 8. Standard processing unit 1004
Is a code conversion unit 9, a pixel operation unit 10, a pixel read / write unit
11, a buffer 200 and a buffer 201 are provided. The FIFO memory 4 has two FIFOs (hereinafter referred to as FIFOs).
Lower video FIFO, audio FIFO)
Transferred from the external memory 3 under the control of the power processor 5.
Compressed video data and compressed audio data
Memorize it in a ceremony. <1.2.2.1 Sequential processing unit> The processor 7
Read compressed video data and compressed audio data of memory 4
Controls some of the compression video data.
Code processing and decoding of all compressed audio data
And do. Decoding of part of compressed video data
Analysis of header information in MPEG data and motion vector
It includes calculation and control of the compressed video decoding process. this is,
The processor 7 performs all decoding processing of the compressed video data,
This is because the sharing is performed with the standard processing unit 1004. Toes
The processor 7 is capable of performing a variety of condition determinations.
The next processing is shared, and the routine processing unit 1004
And perform various arithmetic processing. Audio decoding, on the other hand,
Since the amount of calculation is smaller than that of image decoding, the processor 7
I am in charge of everything. The function of the processor 7 will be specifically described with reference to FIG.
Will be described. FIG. 5 shows the MPEG stream hierarchically.
Both show the operation timing of each part of the video and audio processing device.
I have. In the figure, the horizontal axis is the time axis. The first level is M
5 shows the flow of a PEG stream. 1 second as in the second level
The MPEG stream in between comprises a plurality of frames (I, P,
B picture). As in the third layer, one frame is
Includes a picture header and multiple slices. The fourth level
One slice consists of a slice header and multiple macro blocks.
Including As in the fifth layer, one macroblock is
Includes a black block header and six blocks. Data structure of first to fifth layers shown in FIG.
Is a publicly known document such as ASCII
Formula latest MPEG textbooks ". Professional
As shown in the fifth and lower layers of FIG.
Analysis of header up to macroblock layer in G stream
The compressed audio data is decoded. At that time, the processor 7
According to the header analysis result for each macroblock.
Code conversion unit 9, pixel operation unit 10, and pixel read / write unit 11
To start decoding macroblocks,
Code conversion unit 9, pixel operation unit 10, and pixel read / write unit 11
While the macroblock is being decoded by
Read compressed audio data from FIFO memory 4 and decorate
To load. Code conversion unit 9, pixel operation unit 10, and pixel reading
The decoding of the macroblock is completed by the writing unit 11.
Then, the processor 7 sends a notice to that effect by an interrupt signal.
The decoding of the compressed audio data is interrupted and the next
Start analyzing the header of the black block. The internal memory 8 stores the work
Memory that temporarily holds the decoded audio data.
You. The held audio data is output by the input / output processor 5.
The data is transferred to the external memory 3 through the path (4). <1.2.2.2 Routine processing unit> The code conversion unit 9 is a FIFO
Variable length decoding of compressed video data read from memory 4
(VLD). As shown in FIG.
Is the header information and motion vector of the decoded data.
To the processor 7
To send the macro block (the luminance blocks Y0 to Y3 and the color difference
6 blocks consisting of blocks Cb and Cr)
(Solid line section in the figure) pixel calculation via buffer 200
Transfer to unit 10. Mac after decoding by code conversion unit 9
Block data is data representing spatial frequency components.
You. The buffer 200 is provided by the code conversion unit 9.
Spatial frequency for one block (for 8 × 8 pixels) to be written
Holds data representing numerical components. The pixel operation unit 10
Block transferred from the buffer conversion unit 9 via the buffer 200.
Inverse quantization (IQ) and inverse separation for lock data
The cosine transform (IDCT) is performed for each block. Pixel performance
If the processing result by the arithmetic unit 10 is a luminance block,
Data representing the luminance value of the
Color difference of the pixel or data representing the difference.
To the pixel read / write unit 11 via the buffer 201
Is done. The buffer 201 is composed of one block (8 × 8 picture).
) Of pixel data. Pixel read / write unit 11
Is a block unit of the processing result of the pixel operation unit 10.
Motion compensation. That is, P picture, B picture
, The decoded reference frame in the external memory 3
From the memory controller to the rectangular area indicated by the motion vector
6 through which the processing results of the pixel operation unit 10 are extracted.
Decompose to the original block image by combining with the lock
I do. The decoding result by the pixel read / write unit 11 is
The data is stored in the external memory 3 via the controller 6. Each content of the above-mentioned motion compensation, IQ and IDCT
Is a well-known technology, so a detailed description is omitted (see above).
Literature). <1.3 Detailed Configuration of Each Unit> Next, the video / audio processing device 10
The detailed configuration of each of the main components of 00 will be described. <1.3.1 Processor 7 (sequential processing unit)>
Analysis of macroblock header by processor 7 and other
FIG. 6 is a diagram showing control contents for a unit. First shown in the figure with abbreviations
Each data in the macro block header is
And so on, and the description is omitted here. As shown in the figure, the processor 7 performs code conversion.
Variable length decoded header by issuing command to section 9
Data is acquired sequentially, and the code conversion unit is
9, the pixel operation unit 10 and the pixel read / write unit 11
Set the data required to decode the block. Concrete
Specifically, first, the processor 7 sends the MBA to the code conversion unit 9.
To get I (Macro BlockAddress Increment)
A command is issued (S101), and the code conversion unit 9
Obtain MBAI. The macro based on this MBAI
If the block data is a skip macroblock (now
The macroblock to be decoded is the same as the previous one
If so), the macro block data is omitted
Proceeds to S117, and if not a skip macroblock
The header analysis is continued (S102, 103). Next, the processor 7 sets the MBT (Macro Bl
issue a command to acquire the
The MBT is acquired from the code conversion unit 9. From this MBT
Scan type is zigzag scan or alternate
The pixel operation unit 10 determines whether the scan
00 is designated (S104). further,
The processor 7 performs the STWC from the already obtained header data.
(Spartial Temporal Weight Code) exists
Is determined (S105), and if there is, a command is issued.
And acquire it (S106). Similarly, the processor 7 sets the FrMT (Fr
ame Motion Type), FiMT (Field Motion Type),
DT (DCT type), QSC (Quantizer Scale Code),
MV (Motion Vector), CBP (Coded Block Patter
n) is acquired (S107-116). At that time,
The sensor 7 converts the analysis results of FrMT, FiMT, and DT into pixels.
Notify the read / write unit 11 and calculate the QSC analysis result as a pixel operation
To the code conversion unit 9.
Notice. This is needed for IQ, IDCT and motion compensation
Is the information, the code conversion unit 9, the pixel operation unit 10, the pixel reading
This is set in the writing unit 11. In the two-processor configuration, a wide variety
Each processor performs the above sequential processing that requires condition judgment.
The configuration was redundant since it was performed individually. Then professional
The processor 7 sends a macroblock data to the code converter 9.
A code start instruction is issued (S117). This allows
The mode conversion unit 9 performs processing for each block in the macroblock.
To start VLD and send the result of VLD through buffer 200
And outputs the result to the pixel operation unit 10. Processor 7
Calculates a motion vector based on the MV data (S1
18), the calculation result is notified to the pixel read / write unit 11
(S119). In the above processing, the motion vector
Is the motion vector data (MV) acquisition (S113).
Then, the motion vector is calculated (S118), and the motion vector is calculated.
Is set in the pixel read / write unit 11 (S119).
A series of processing is required. In this regard, the processor 7
Immediately after obtaining the vector data (MV) (S113)
Calculation and setting of motion vector (S118, 119)
Issue a decode start instruction to the routine processing unit 1004
And then calculate and set the motion vector
I have. As a result, the motion vector calculation and the
And setting processing, and decoding processing to the routine processing unit 1004
Are processed in parallel. That is, the routine processing unit 1
The decoding start timing of 004 is advanced. As described above, one macroblock
Since the header analysis of the compressed video data is completed,
The server 7 obtains the compressed audio data from the FIFO memory 4
Then, the audio decoding process is started (S120). Audio de
Code processing is performed by the code conversion unit 9 to decode macroblocks.
Continue until an interrupt signal indicating code completion is input
It is. This interrupt signal causes the processor 7 to
The above header analysis is started for the block. <1.3.2 Routine processing unit> Next, the routine processing unit 1004
The code conversion unit 9 converts the six blocks in the macro block into
Pixel operation unit 10 and pixel read / write unit 11 are connected in parallel (pipe
(Depending on the line)
ing. Here, the pixel operation unit 10, the pixel read / write unit
11, in order of the code conversion unit 9, their structure will be described in more detail.
explain. <1.3.2.1 Code Conversion Unit 9> FIG.
9 is a block diagram showing the configuration of FIG. The code converter 9 shown in FIG.
1, counter 902, incrementer 903, selector
904, scan table 905, scan table 9
06, flip-flop (hereinafter abbreviated as FF) 907,
And the result of variable length decoding (VLD)
Zigzag scan or alternate in blocks
Write to buffer 200 to arrange in scan order
Is configured. The VLD unit 901 receives the data from the FIFO memory 4
Variable length decoding (VLD) of the read compressed video data
Of the decoded data, header information and motion vector
Information (the broken line section in FIG. 5)
Transfer the macro block (luminance blocks Y0 to Y3 and color
(6 blocks consisting of difference blocks Cb and Cr)
(Solid line section in FIG. 5) is divided into blocks (64 spatial frequencies).
The data is output to the buffer 200 in units of (data). A counter 902, an incrementer 903,
The circuit portion including the selector 904 is connected to the VLD unit 901
From 0 to 63 in synchronization with the output of these spatial frequency data.
Is counted repeatedly. Scan table 905
Indicates the address of the block storage area of the buffer 200.
This table is stored in the order of zag scan.
The output values (0 to 63) of the counter 902 are sequentially input, and
Next, the address is output. FIG. 20 shows the buffer 200
Storage of 8 × 8 spatial frequency data of
The area and the route of zigzag scanning are shown. Scantay
Bull 905 sequentially stores pixel addresses in the route shown in FIG.
Output. The scan table 906 is stored in the buffer 20
0 address of the block storage area
Table stored in the order of
Output values (0 to 63) are sequentially input, and the
Output FIG. 21 shows the 8 × 8
A block storage area for storing spatial frequency data;
This shows the route of the Tanate scan. Scan table 90
5 sequentially outputs pixel addresses in the route shown in FIG.
You. The FF907 is a scan type (zigzag)
Scan or alternate scan).
Carry. This flag is set by the processor 7
You. The selector 908 selects a switch according to the flag of the FF 907.
From the can table 905 and the scan table 906
Select output address and write to buffer 200
Only output as address. <1.3.2.2 Pixel Operation Unit> FIG.
It is a block diagram showing composition. As shown in the figure, the pixel operation unit 10 includes a multiplier 5
02 and an adder / subtractor 503, and an
A program counter (hereinafter abbreviated as a first PC) 504;
Second program counter (hereinafter abbreviated as second PC) 50
5, a first instruction memory 506, and a second instruction memory 507
And a selector 508, and a part of IQ and IDCT.
Are configured to be executed in parallel with overlapping
ing. . The execution unit 501 includes a first instruction memory 506,
Micro instructions sequentially output from the second instruction memory 507
Access and operation of buffers 200 and 201 according to
Execute First instruction memory 506, second instruction memory 5
07 is a block (frequency) held in the buffer 200.
Component) to realize IQ and IDCT.
This is a control memory for storing a black program. FIG.
Stored in the first instruction memory 506 and the second instruction memory 507
Here is an example of a microprogram that has been created. In the figure, the first instruction memory 506 stores I
DCT1A micro program and IQ micro program
And read out the address by the first PC 504.
Address is specified. IQ microprogram buffer
200, and a multiplication operation.
Therefore, the adder / subtractor 503 is not used. Second instruction memory 507
Is the IDCT1B microprogram and IDCT2
And a second program through the selector 508.
Read address is set by 1PC 504 or 2nd PC 505
It is specified. Here, IDCT1 performs multiplication and addition / subtraction.
IDCT means the first half of IDCT processing.
1A microprogram and IDCT1B microprogram
And the RAM are read out at the same time.
Performed using the body. In addition, IDCT2 performs addition and subtraction.
Processing of the latter half of the main IDCT and buffer 201
Means the writing process of the ID of the second instruction memory 507.
By reading the CT2 microprogram
This is executed using the adder / subtractor 503. The IQ is multiplied by the multiplier 502, and the IDCT2 is
Since they are processed by the adder / subtractor 503, they are processed in parallel.
It is possible to work. FIG. 9 shows the relationship between the I
Q shows operation timing charts of IDCT1 and IDCT2.
You. In FIG. 9, the code conversion unit 9
When the data of the luminance block Y0 is written (at timing t
0), the fact is transmitted to the pixel operation unit 10 by the control signal 102.
Know. The pixel operation unit 10 analyzes the header of the processor 7
Using the QS (Quantizer Scale) value set at the time,
First instruction memory according to address designation of first PC 504
By reading 506 IQ microprograms,
To perform IQ on the data in the buffer 200. This and
Selector 508 selects the first PC 504 (in the
(Ming t1). Further, the pixel operation section 10 includes a first PC 50
IDCT1A and IDCT1 according to the address designation
Buffer by reading B microprogram
IDCT1 is performed on 200 data. At this time,
Since the selector 508 selects the first PC 504, the first
The first and second instruction memories 506 and 507 have the first
An address from PC 504 is specified (at timing t
2). Next, the pixel operation unit 10 executes the QS (Quan
address of the first PC 504 using the tizer scale) value.
The IQ microprocessor of the first instruction memory 506 according to the specification
By reading the program, the block of the buffer 200 is read.
The IQ of the data of Y1 is performed, and at the same time, the second PC
505 of the second instruction memory 507
By reading the IDCT2 microprogram
Process the latter half of the IDCT process for block Y0
You. At this time, the selector 508 selects the second PC 505.
You. The first PC 504 and the second PC 505 are independently addressed.
(Timing t3). After this, the pixel operation unit 10 similarly operates as a block
Processing is continued in units (after timing t4). <1.3.2.3 Pixel Read / Write Unit> FIG.
FIG. 3 is a block diagram illustrating a detailed configuration of a unit 11. See the same figure
As described above, the pixel read / write unit 11 includes buffers 71 to 74 (hereinafter referred to as buffers 71 to 74).
Below, referred to as buffers A to D).
, Combining section 76, selectors 77 and 78, read / write control
The control unit 79 is included. The read / write control unit 79 controls the buffer 201
Buffer data for block data input through
~ D to perform motion compensation, and the final decoded image is
The data is transferred to the external memory 3 in lock units. More specifically
Is the motion vector set during the header analysis of the processor 7.
2 frames from the reference frame in the external memory 3 according to the
The memory controller reads a rectangular area corresponding to the lock.
It controls the trawler 6. As a result, buffer A or buffer A
A rectangular area of 2 blocks indicated by the motion vector in file B
The area data is stored. Then, the picture type (I
Or P picture or B picture)
Is performed by the synthesizing unit 76. More buff
Block data input through the
By combining (adding) with the rectangular area after pel interpolation
Calculates the pixel value of the block and stores it in buffer B.
I do. Thus, the final decryption block stored in buffer B is obtained.
The lock is stored in the external memory 3 via the memory controller 6.
Will be transferred. <1.3.3 Input / output processing unit> The input / output processing unit 1001
Perform a large number of data input / output (data transfer) as described above
Multiple tasks to share various data transfers
Switching without overhead, and data input / output request
Is configured not to cause a response delay to
You. The overhead here is at the time of task switching
Save and restore of the context that occurs. In other words
The output processor 5 is provided with an instruction address of the program counter.
Data and register data to memory (stack area)
Eliminate the overhead of returning and returning
It is configured as follows. Here, the detailed configuration
Will be described. <1.3.3.1 IO Processor> FIG.
FIG. 4 is a block diagram illustrating a configuration of a sa. In FIG.
The O processor 5 includes a state monitoring register 51, an instruction memory
52, instruction reading circuit 53, instruction register 54, decoder
55, operation execution unit 56, general-purpose register set group 57,
A disk management unit 58 is provided.
In order to respond to the
It is configured to execute while switching tasks for each cycle)
Have been. The state monitoring register 51 is provided in the register CR1.
~ CR3, and the IO processor 5
Various status data (flags, etc.) for monitoring the status
Hold. For example, the state monitoring register 51
Status of the system input unit 1 (star in MPEG stream)
Code detection flag), the state of the video output unit 12 (horizontal
Flag indicating blanking period, transfer of frame data
Completion flag), the state of the audio output unit 13 (audio frame
Data transfer completion flag) and the buffer memory
2, data between the external memory 3 and the FIFO memory 4
Data transfer status (number of data transfers, data to FIFO memory 4)
(Data request flag). More specifically, the following flags are included. Start code detection flag (hereinafter also referred to as flag 1)
Set when a start code in the trim is detected
Is done. -Horizontal blanking flag (flag 2) This flag indicates a horizontal blanking period.
And is set by the video output unit 12. About 60 microphones
It is set in seconds. -Transfer completion flag of video frame data (flag 3) This flag is transmitted from the external memory 3 to the video output
D when the decoded image data for the frame is transferred
Set by the MAC 5a. • Audio frame data transfer completion flag (flag 4) This flag is transmitted from the external memory 3 to the audio output
DM when the decoded voice data for the frame is transferred
Set by AC5a. Data transfer completion flag (flag 5) This flag is transmitted from the stream input unit 1 to the buffer memory.
2. Compression of the number of data specified by the IO processor 5
When image data is DMA-transferred by the DMAC 5a
(When the terminal count is reached). DMA request flag (flag 6) This flag indicates whether the compressed image data in the buffer memory 2 or
Data to be DMA-transferred compressed audio data to external memory 3
Is a flag indicating that there is a data
(Required from Task 1 to Task 2 to be described later)
Request). A data request flag to the video FIFO (flag 7) This flag is transmitted from the external memory 3 to the FIFO memory 4
A flag requesting data transfer to the video FIFO,
When the amount of compressed video data in the video FIFO falls below a predetermined amount
Is set. This flag takes approximately 5 to 40 microseconds
Period. A data request flag to the audio FIFO (flag 8) This flag is transmitted from the external memory 3 to the
A flag requesting data transfer to the audio FIFO,
If the audio FIFO compressed audio data falls below a certain amount
Is set. This flag is about 15-60 micro
It is set in seconds. Decoder communication request flag (flag 9) This flag is input / output processing from the decoding processing unit 1002.
This is a flag for requesting the unit 1001 to communicate. Host communication request flag (flag 10) This flag is transmitted from the host processor to the input / output processing unit 10
01 is a flag for requesting communication. The above-mentioned flags are transmitted by the IO processor 5.
Tasks are executed instead of interrupts.
Monitored. The instruction memory 52 has a large number of data inputs / outputs.
Task programs that share power (data transfer) control
Memorize the system. In the present embodiment, six tasks 0 to 5 are set.
Memorize the disk program. Task 0 (host I / F task) This task is executed when the flag 10 is set.
Communication with the computer, that is, via the host I / F unit 14
For performing communication processing with the host computer
It is. For example, an MPEG stream from the host processor
Start, stop, fast-forward playback, reverse playback, etc.
Acceptance and notification of decoding status (error etc.)
Will be In this processing, the flag 10 is used as a trigger.
You.・ Task 1 (Purging task) This task is started by the stream input unit 1
Is detected (the above flag 1), the stream input unit
Parsing MPEG data input from 1 (parsing)
To extract and extract individual elementary streams
DMA transfer of the elementary stream
(The first half of the transmission path (1)) to the buffer memory 2.
Program. Elementalis extracted here
The type of trim is compressed video data (video element
Stream), compressed audio data (audio
Oelementary stream), private
There is data. Buffer elementary stream
When stored in the memory 2, the flag 6 is set.
You. Task 2 (stream transfer / audio task) This task is a professional task that controls the following transfer (a) to (c).
Gram. (A) From the buffer memory 2 to the external memory 3
DMA transfer of each elementary stream
The latter half of the transmission route (1)). This transfer is based on the flags 1, 3
Is the trigger. (b) Compressed audio data held in audio FIFO
From the external memory 3 according to the data size (remaining amount)
Compressed audio data into the audio FIFO in the FIFO memory 4
DMA transfer of data (audio transfer in the above transfer path (2))
Transfer to off FIFO). This data transfer is
Data size of compressed audio data held in FIFO
This is done when the amount is less than a certain amount. This roll
Transmission is triggered by the flag 8 described above. (C) From the external memory 3 to the buffer memory 2
To the audio output unit 13 from the buffer memory 2
DMA transfer of audio data after the transfer
(Four)). This transfer is triggered by the flag 2 described above. -Task 3 (Video supply task) This task is to perform the compressed video data held in the video FIFO.
External memory 3 depending on the data size (remaining amount) of data
Video data to the video FIFO in the FIFO memory 4
DMA transfer (video FIFO in transfer path (2) above)
Is a program that processes This data transfer
Transmission of the compressed video data held in the video FIFO
This is performed when the data size becomes smaller than a certain amount.
You. This transfer is triggered by the flag 7 described above. Task 4 (video output task) This task is performed from the external memory 3 to the buffer memory 2.
After decoding from the buffer memory 2 to the video output unit 12,
Processes DMA transfer of video data (the above transfer path (4))
Program. This transfer triggers flag 2 above.
Gar. Task 5 (decoder I / F task) This task is executed by the decode processor 1002
This is a program for processing a command directed to the server 5.
Commands include "getAPTS", "getVPTS", "getSTC"
and so on. getVPTS (Video Presentation Time Stam
p) indicates that the decode processing unit 1002
Of VPTS assigned to compressed video data
Command. getAPTS (Audio Presentat
ion Time Stamp), the decode processing unit 1002
A assigned to the compressed audio data for the processor 5
This is a command for requesting acquisition of a PTS. getSTC (Syst
em Time Clock), the decode processing unit 1002
A command to request the processor 5 to acquire the STC
is there. Upon receiving these commands, the IO processor 5
STC, VPTS, APTS in decoding processing unit 1002
Notify each. STC, VPTS, APTS,
The decoding processing unit 1002 decodes audio and video.
Code, or adjust the decoding progress in frame units.
Used to adjust. In this process, the flag 9 is set.
Trigger. The instruction read circuit 53 has an instruction fetch address
Program counters (hereinafter abbreviated as PC)
Using the PC specified by the task management unit 58
Instruction from the instruction memory 52 to read the instruction
4 is stored. Specifically, the instruction reading circuit 53
Task management with PCs 0-5 corresponding to tasks 0-5
When the designation of the PC by the unit 58 is changed, the hardware
Is configured to switch PCs faster
You. With this configuration, the IO processor 5 can perform task switching.
Saves the PC value of the current task to memory when
Released from the process of returning the PC value of the next task from memory
Have been. The decoder 55 reads from the instruction memory 52.
Decodes the instruction issued and stored in the instruction register 54,
The arithmetic execution unit 56 is controlled so as to execute the instruction.
In addition, the decoder 55 controls the IO processor 5 as a whole.
Instruction reading stage of instruction reading circuit 53, decoder 55
Of the decoding stage and the execution stage of the arithmetic execution unit 56
At least three stages of pipeline control are performed. The arithmetic execution unit 56 is provided with an ALU (Arithmetic L
ogical unit), multiplier, BS (Barrel Shifter)
Specified by the instruction according to the control of the decoder 55.
Perform the operation. The general-purpose register set group 57 includes a task
Six register sets (one register set)
The star set consists of four 32-bit registers and four 16-bit registers.
Register). A total of 24 32 bits
Register and 24 16-bit registers.
The register set corresponding to the task in line is used.
This allows the IO processor 5 to perform task switching.
Save all current register data to memory
From the process of restoring the register data of the next task from
Have been. The task management unit 58 stores a predetermined number of instruction cycles.
The PC of the instruction reading circuit 53 and the general-purpose register
By switching the register set of the
Switch the disk. In this embodiment, the predetermined number is four.
You. Also, the IO processor 5 can execute one instruction in one instruction cycle.
Since the pipeline processing is performed, the task management unit 58
Task every 4 instructions without any overhead
Will switch. This will cause each asynchronous
Response delay is suppressed for various input / output requests. I mean
Response delay to I / O request is only 24 lives at most
Only a command cycle occurs. <1.3.3.1.1 Instruction Read Circuit> FIG. 12 shows an instruction read circuit.
FIG. 53 is a block diagram illustrating a detailed configuration example of a configuration 53. In the figure, the instruction reading circuit 53
Separate PC storage unit 53a, current IFAR (Instruction Fetch
Address Register) 53b, incrementer 53c, next
IFAR 53d, selector 53e, selector 53f, D
Equipped with ECAR (DECode Address Register) 53g,
Instruction reading without overhead when switching tasks
It is configured to switch addresses. The task-specific PC storage unit 53a stores tasks 0 to
5 address registers for each task
Holds the program count value. Each program cow
The event value is a restart address of the corresponding task. Tas
When switching between tasks, the task management unit 58 and the decoder 55
Under the control of the address corresponding to the task to be performed next.
The program count value is read from the
Address register corresponding to the task being executed
The program count value is updated to the new restart address.
You. At this time, the task to be executed next and the current task are
The task management unit 58 executes “nexttaskid (rd add
r) "signal (hereinafter also referred to as task ID)," taskid (wr a
ddr) "signal. The program card corresponding to tasks 0, 1, and 2
The count values are shown in PC0, PC1, and PC2 in FIG. Smell
(0-0) indicates instruction 0 of task 0, and (1-4) indicates
Indicates instruction 4 of disk 1. For example, PC0
Read at the time of restart (instruction cycle t0), the next timer
To the address of instruction (0-4) when switching to
New (instruction cycle t4). Incrementor 53c, next IFAR 53
d, and the loop circuit composed of the selector 53e
Update the instruction read address selected by 3e
Circuit. The address output from the selector 53e is
This is shown as IF1 in FIG. In the figure, for example, task 0
When switching from to the task 1, the selector 53e
In cycle t4, read from task-specific PC storage unit 53a
Select the fetched instruction (1-0) address and cycle
From t5 to t7, increment from the next IFAR 53d
Selected instruction address. The current IFAR 53b is selected by the selector 53e.
The selection output IF1 is held one cycle later, and the instruction memory 5
2 is output as an instruction read address. Paraphrase
The instruction read address of the currently active task
Hold. Instruction read address of current IFAR 53b
Is shown as IF2 in FIG. As shown in FIG.
Specifies the instruction address of a different task every four instruction cycles
are doing. The DECAR 53g is stored in the instruction register 54.
Holds the address of the held instruction. That is,
Refers to an instruction in the code. The DEC in FIG.
Indicates the address held in R53g. Also, in FIG.
EX indicates an instruction address being executed. Selector 53
f indicates the branch address when a branch instruction is executed or an interrupt occurs.
Select, otherwise select next IFAR53d address
I do. Provision of such an instruction reading circuit 53
As a result, the IO processor 5 has four stages as shown in FIG.
(IF1, IF2, DEC, EX) pipeline processing
It is carried out. Of these, the IF1 stage has multiple programs.
This is a stage for selecting and updating a ram count value.
The IF2 stage is a stage for reading an instruction. <1.3.3.1.2 Task Management Unit> FIG. 14 shows the task management unit.
FIG. 58 is a block diagram showing a detailed configuration of the embodiment 58. Smell
The task manager 58 manages the task switching timing.
The slot manager to manage and the order of tasks
It is roughly divided into a scheduler. The slot manager includes a counter 58a,
The latch 58b, the comparator 58c, and the latch unit 58d
Task that instructs task switching every four instruction cycles
Output a switching signal (chgtaskex) to the instruction reading circuit 53.
You. Specifically, the latch 58b outputs the output of the counter 58a.
Two FFs (Flip Flop) holding the lower 2 bits of the force
Circuit. The counter 58a has a clock indicating an instruction cycle.
Increment the 2-bit output value of latch 58b by +1 for each lock
Output the incremented 3 bits. As a result,
Tab 58a repeatedly outputs 1, 2, 3, and 4
Become. The output of the counter 58a is a constant.
When the value matches 4, the task switching signal (chgtaskex) is commanded.
It outputs to the instruction reading circuit 53 and the scheduler. The scheduler has a task round management unit 5
8e, priority encoder 58f, latch 58g
The task switching signal (chgtaskex) is output
Update the task id, and execute the current task
The task id to be executed is output to the instruction reading circuit 53.
Specifically, the latch unit 58d and the latch 58g
In both cases, the current task id is encoded (3 video
). The encoded form has its value
Represents task id. The task round management unit 58 e
When the replacement signal (chgtaskex) is input, the latch unit
The task id to be executed next is referred to
Output in coded format (6 bits). Decoded
Format (6 bits), one bit corresponds to one task
And the bit position represents the task id. Priority
The encoder 58f outputs from the task round management unit 58e.
The task id to be input is encoded from the decoded
To the loaded format. The latch unit 58
d, latch 58g, together with encoded task i
d is held one cycle later. With this configuration, the task round management unit 5
8e receives a task switching signal (chgtaske) from the comparator 58c.
x) is output, the priority encoder 58
From f, change the id of the task to be executed next to "nexttaskid (rd
addr) "as the signal from the latch 58e to the current task id.
As a “taskid (wr addr)” signal. <1.4 Description of operation> First embodiment configured as described above
Video and audio processing apparatus 1000 in the form
The operation will be described. In the input / output processing unit 1001, the stream
MPEG stream asynchronously input from the system input unit 1
Is a buffer memo under the control of the input / output processor 5.
External memory 3 via memory controller 6
And further via the memory controller 6 to F
It is held in the IFO memory 4. At this time, FIFO memory
4, the IO processor 5 executes the task 2
(B) According to the remaining amount by executing task 3
To supply compressed moving image data and compressed audio data. this
A certain amount of compression can be stored in the FIFO memory 4 without excess or shortage.
Since video data and compressed audio data are supplied,
Processing unit 1002 is separated from asynchronous input / output.
Thus, it is possible to exclusively use the decoding process. So far
The processing is performed by the input / output processing unit 1001 for decoding.
The processing is performed in parallel with the processing unit 1002 independently. On the other hand, in the decoding processing unit 1002,
MPEG stream data held in the FIFO memory 4
Hereafter, the processor 7, the code conversion unit 9, the pixel operation unit
10, decoded by the pixel read / write unit 11. FIFO
FIG. 15 is an explanatory diagram showing the decoding operation after the memory 4.
In the figure, the horizontal axis is the time axis and approximately one macroblock
Header analysis and decoding for each block
Is shown. In the vertical direction, each of the decode processing units 1002
Decoding of each block in the
It shows how it is performed. As shown in the figure, the processor 7
Header analysis of video data and data for compressed audio data
The code processing is repeated in a time sharing manner. That is, the processor
The server 7 analyzes the header of one macroblock and analyzes the header.
Code conversion unit 9, pixel operation unit 10, pixel read / write
After the notification to the unit 11, the macro conversion
Instructs to start decoding the lock. Then processor 7
Until the interrupt signal from the code conversion unit 9 is notified.
Then, the decoding process of the compressed audio data is performed. After decoding
Is temporarily stored in the internal memory 8 and further stored in the memory.
DMA transfer to the external memory 3 by the memory controller 6
It is. Also, the code conversion unit 9
Receives the macroblock decoding start instruction from the
Store in the buffer 200 for each block in the block
You. At this time, the code conversion unit 9
Depending on the block scan type reported during analysis
Change the order of write addresses to the buffer 200
You. In other words, the zigzag scan and the alternate scan
The order of the write addresses is changed in the case of the scan.
Accordingly, the pixel operation unit 10 determines the order of the read addresses.
Does not need to be changed, regardless of the scan type.
Can be read in the order of the read addresses
You. The code conversion unit 9 converts the six blocks in the macroblock.
Repeat the above operation until the VLD process is completed
Write to buffer 200. After 6 blocks of VLD
Then, an interrupt is generated in the processor 7. This interrupt signal
Is the macro block decoding end signal End Of Macro Blo
ck (EOMB). The code conversion unit 9 is used for the sixth block.
To detect the block end signal End Of Block (EOB)
Is generating more EOMB. The pixel operation section 10 is parallel to the code conversion section 9.
And stored in the buffer 200 as shown in FIG.
Apply IQ and IDCT to block data in block units.
Then, the processing result is stored in the buffer 201. Pixel reading
The writing unit 11 includes a buffer in parallel with the pixel operation unit 10.
201 block data and a header by the processor 7
1 based on the motion vector notified by the analysis.
As shown in FIG. 5, a rectangle from the reference frame of the external memory 3
Region extraction and block synthesis are performed. Block
The result is stored in the external memory 3 via the FIFO memory 4.
Will be delivered. The above is not a skip macroblock
Operation in case of skip macro block
Indicates that the code conversion unit 9 and the pixel operation unit 10 do not operate,
Only the read / write unit 11 operates. Skip macro block
If there is a block, the same image as the rectangular area in the reference frame
The image is decoded by the pixel read / write unit 11
The image is copied to the external memory 3 as an image. In this case, the code conversion unit 9
The interrupt signal to 7 is generated as follows. Sand
That is, the processor 7 performs motion compensation on the pixel read / write unit 11.
A signal indicating that a control signal for starting the compensation operation has been sent,
That the pixel read / write unit 11 can perform a motion compensation operation.
Signal indicating that it is a skipped macroblock
Take the logical product of the signal and the logical product and the above EOMB
An interrupt signal is input to the processor 7 as a logical sum with the signal
Is done. As described above, the first embodiment of the present invention
According to the video / audio processing device of the form, the storage medium or the communication medium
MPEG stream input processing, display device and sound
Output processing of display image data and audio data to voice output device
And supplies a stream to the decoding processing unit 1002.
The input / output processing unit 1001 shares the processing with the compressed video data.
Decoding unit for decoding data and compressed audio data
1002 is configured to share. This
The decoding processing unit 1002 performs processing that occurs asynchronously.
And can be dedicated to the decoding process.
As a result, MPEG stream input, decode, output
The series of processes described above are executed efficiently,
Full decoding of MPEG stream without lock
(Without dropped frames). Further, the present video / audio processing apparatus is integrated into one chip.
It is desirable to use an LSI. In this case, 100MHz
The following operation clock (actually 54 MHz)
Decoding is possible. In this regard, the operating clock is 100
MHz High performance CP over 200MHz
U can perform the above full decoding if the image size is small
However, the manufacturing cost is high. this
On the other hand, this video and audio processing device is
Excellent in decoding. Further, the decoding processing of the video / audio processing apparatus
The management unit 1002 shares roles as follows. Toes
The processor 7 also performs compressed audio processing on the compressed video data.
A variety of conditional decisions are required for data.
Responsible for analyzing data and decoding compressed audio data
Also in charge. For block data of compressed video data
Requires a large amount of routine computation,
Conversion unit 9, pixel operation unit 10, and pixel read / write unit 11.
Hardware (firmware) for decoding
In charge of As shown in FIG.
The prime operation unit 10 and the pixel read / write unit 11 are pipelined
Have been. The pixel operation unit 10 has an IQ and an IDCT
Column processing is enabled. The pixel read / write unit 11 has two blocks.
Access to reference frames in lock units is realized.
With these, the efficiency of the compressed audio decoding process has been improved.
Hardware dedicated to video decoding is expensive.
High processing power can be obtained without using a fast clock
it can. Specifically, a high-speed clock exceeding 100 MHz
With a clock of about 50 to 60 MHz without using
The processing ability of the degree or more was obtained. Therefore, high-speed devices
It is not necessary to use it, and the manufacturing cost can be reduced. The basic unit of video decoding is
The macro conversion unit 9 and the code conversion unit 9
And the pixel read / write unit 1 in the pixel operation unit 10
Because 1 has 2 blocks,
Buffer buffer capacity can be minimized
It becomes. <2 Second embodiment> Video / audio processing apparatus of the present embodiment
In addition to the ability to decode compressed stream data,
In addition, the compression function (hereinafter referred to as encoding) and graphics
It is configured to perform a fix function. <2.1 Configuration of Video / Audio Processing Apparatus> FIG.
2 shows a configuration of a video and audio processing device according to a second embodiment.
It is a block diagram. This video / audio processing apparatus 2000 has
Input / output unit 21, buffer memory 22, FIFO memo
24, input / output processor 25, memory controller 2
6, processor 27, internal memory 28, code conversion unit 2
9, pixel operation unit 30, pixel read / write unit 31, video output
Unit 12, audio output unit 13, buffer 200, buffer 2
01. The video / audio processing device 2000 is shown in FIG.
In addition to the functions of the video and audio processing apparatus 1000 shown in FIG.
Features have been added. That is, video data and audio data
Data compression function and graphics to draw polygon data.
Functions are added. For this reason, the video / audio processing device 2000
Therefore, components having the same names as those in FIG.
In addition, functions that perform compression and graphics functions
Has been added. The description of the same points as those in FIG.
The following description focuses on the following points. The stream input / output unit 21
The difference is that it is bidirectional. In other words, the input / output process
From the buffer memory 22 under the control of the
When data is transferred, the transferred parallel data is
Convert to real data, and MPEG data stream
And output to the outside. Buffer memory 22, FIFO memory 24
Is also bidirectional. I / O processor 25
Is the route data shown in (1) to (4) shown in the first embodiment.
In addition to controlling the data transfer, the transfer of the route of (5) to (8)
Also control. (1) Stream input / output unit 21 → buffer memory 22 → memory control
26 → external memory 3 (2) external memory 3 → memory controller 26 → FIFO memory 24 (3) external memory 3 → memory controller 26 → buffer memory 22
→ Video output unit 12 (4) External memory 3 → Memory controller 26 → Buffer memory 22
→ Audio output unit 13 (5) External memory 3 → Memory controller 26 → Internal memory 28 (6) External memory 3 → Memory controller 26 → Pixel read / write unit 31 (7) FIFO memory 24 → Memory controller 26 → External memory 3 (8 ) External memory 3 → memory controller 26 → buffer memory 22
→ The stream of the stream input / output unit 21
This is the path of the original data when processing is performed, and (7) and (8) are
4 shows the path of an MPEG stream after compression. First, the encoding process will be described.
The data to be encoded is stored in the external memory 3.
Shall be. The video data of the external memory 3 is
The pixel read / write unit 31 controls the controller 26.
Is transferred to the pixel read / write unit 31. Pixel read / write unit
A process 31 writes video data to the second buffer 201.
And difference image generation processing. The difference image generation processing includes:
Motion detection (calculation of motion vector) and difference in block units
Image generation. Therefore, the pixel read / write unit 31
Is a rectangular area similar to the encoding target block and the reference frame.
Motion to detect a motion vector by searching within the
A detection circuit inside. Note that the motion detection circuit
Instead of the already calculated block of the adjacent frame
Look at the motion vector to be encoded using the motion vector
A stacking motion estimation circuit may be provided. The pixel operation unit 25 calculates the difference image for each block.
Receiving image data, DCT, IDCT, quantization processing
(Hereinafter, Q processing) and IQ are performed. Is quantized in this way
The video data is stored in the buffer 200. Code change
The conversion unit 29 receives the quantized data from the buffer 200.
Variable length code processing (VLC). Variable length coded
The stored data is stored in the first-in first-out memory 24 and
Stored in the external memory 3 through the re-controller 26
At the same time, the processor 27
Header information is added. The video data in the external memory 3 is a memo.
Transferred to the internal memory 28 via the re-controller 26
You. The processor 27 outputs header information for each macroblock.
The audio data in the internal memory 28 is
Data compression processing. As described above, the encoding process
Will be processed in the reverse path to the first embodiment.
You. Next, the graphics processing will be described.
You. Graphics processing is a rectangular type called polygon
In 3D image generation processing performed by combining figures
is there. In this device, the image at the vertex coordinates of the polygon is
Processing to generate pixel data inside polygon from raw data
I do. First, the polygon vertex data is stored in the external memory 3.
Is stored. The vertex data is stored in the memory
By controlling the controller 26, the
Is stored. The processor 27 calculates the vertex from the internal memory 28.
Read data and DDA (Digital Difference Analyze)
Is performed and the data is written to the FIFO memory 24. Co
The conversion unit 29 receives the FIF according to the instruction from the pixel operation unit 30.
The vertex data is read from the O memory 24 and the pixel operation unit 30
Transfer to The pixel operation unit 30 performs a DDA process on the pixels.
It transmits to the read / write unit 31. The pixel read / write unit 31
According to the instruction of the processor 27, the Z buffer processing or α
The memory controller 26 performs the blending process.
The image data is written to the external memory 3 via the external memory 3. <2.1.1 Pixel Operation Unit> FIG.
It is a block diagram showing composition. FIG. 13 is the same as the pixel operation unit 10 shown in FIG.
The same components are assigned the same numbers, and the description is omitted.
The different points will be mainly described. The difference is as shown in the figure.
The pixel operation unit 30 is different from the pixel operation unit 10 shown in FIG.
That the execution unit has three planes (501a to 501c)
And an instruction pointer holding unit 308 and an instruction register 309.
That is, the distribution unit 310 is added. The execution units 501a to 501c have three surfaces.
The reason for this is to improve the calculation performance. Specifically
Is used to convert color images RGB in graphics processing.
Perform independent parallel calculations. In the IQ and Q processing, the multiplier 5
02 is used to increase the speed. IDCT smell
Use multiple multipliers 502 and adder / subtractors 503
By doing so, time is reduced. In IDCT
There is an operation called butterfly operation, which is
Since there is a dependency between all the source data,
Data line for communicating between units 501a to 501c
103 is provided. First instruction memory 506, second instruction memory 5
07 is DCT, Q processing, DD in addition to IDCT and IQ.
A microprogram for A is stored. FIG.
In the first instruction memory 506 and the second instruction memory 507,
An example of memory contents is shown. Q processing micropro
Gram, DCT microprogram and DDA microphone
B programs have been added. Instruction pointer holding units 308a to 308c
Are provided corresponding to the execution units 501a to 501c.
Address input from the first program counter respectively
Conversion table that converts and outputs to the instruction register unit 309
Have The converted address is stored in the instruction register 30
9 means the register number. In addition, instruction pointer storage
The holding units 308a to 308c are respectively
Hold the output flag and output it to the instruction execution units 501a to 501c.
Power. Instruction pointer is held for the conversion table
The units 308a, 308b, 308c are, for example, input address
If the source is 1,2,3,4,5,6,7,8,9,10,11,12
The following converted address is output. Instruction pointer holding unit 308a: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 1
1,12 instruction pointer holding unit 308b: 2,1,4,3,6,5,8,7,10,9,1
2,11 instruction pointer holding unit 308c: 4,3,2,1,8,7,6,5,12,11,1
0,9 The instruction register unit 309 includes a microphone as shown in FIG.
B. Multiple registers holding instructions. 3 selectors and 3 registers.
Output port. The three selectors are instruction poi
Input from the counters 308a, 308b, 308c.
Of the register specified by the replacement address (register number).
Select the micro instruction. The three output ports are selectors
Are provided corresponding to the
The micro instruction is executed by the execution units 501a to 501 through the distribution unit 310.
01c. Three selectors and output ports
The only difference is that three adder / subtracters 503 (or three
Supply different multipliers simultaneously to the multiplier 502)
That's why. In this embodiment, the three output ports are
310, three adder / subtracters 503 and three multipliers 5
02 is selectively supplied. For example, the instruction register unit 309 is a register
R1 to R16 (register numbers 1 to 16) are provided.
Microprogram stored in registers R1 to R16
RAM is a matrix operation processing required in DCT and IDCT.
In any of the above three register numbers.
Are stored so as to perform the same processing. In other words, above
The micro program having three execution orders is the execution order
Some micro-instructions where
You. This is because the execution units 501a to 501c
Execution program 501a-501
Resource interference such as register (not shown) access conflict between c
This is to avoid it. Further, the matrix operation processing is performed by 8 ×
The contents include multiplication, transposition, and transfer of eight matrices. Next, each register of the instruction register unit 309
Are stored in mnemonic format.
"Op Ri, Rj, dest, (modifier
G). However, the instruction register
The micro instruction is “op, Ri, Rj and (modify
Lag) ”only. "Dest" is an instruction
It is specified from the memories 506 and 507. "(Modifa
Instruction flag holding sections 308a to 308a)
8c. Here, "op" is a multiplication instruction and an addition / subtraction instruction.
Instruction code, transfer code, etc., "R
i, Rj "are operands.
Instruction executed by each multiplier 502 in row units 501a to 501c
And the addition instruction and the transfer instruction include three execution units 501
Instructions executed by each of the multipliers 502 in a to c. "
“dest” indicates the storage destination of the operation result.
t ″ is not a register of the instruction register unit 309, but an instruction
Memory 506 (for multiplication instructions) or instruction memory 507
(In the case of an addition / subtraction instruction or a transfer instruction). this
Executes the microprogram of the instruction register unit 309
This is to make them common to the units 501a to 501c. If
If the destination is specified by a register, the execution units 501a to 501a
1c It is necessary to prepare an individual microprogram for each
It is necessary to increase the capacity of the microprogram several times.
And The "modify flag" is
Is a flag indicating whether the operation is addition or subtraction.
You. This “modify flag” is stored in the instruction register 3
09, not from the register 09
8a to 8c. This is DCT, IDC
All elements in the constant matrix used for the matrix operation in T are "
1 ”row (or column) and all elements are“ −1 ”row (or column)
Are included in the instruction pointers 308a to 308c.
Instruction flag by specifying the
It is possible to share the same microprogram of unit 309
I'm working. The distribution unit 310 is provided with the instruction register unit 309
If the three micro instructions input from the
In this case, those "op and Ri, Rj" parts and the instruction
The “dest” part input from the memory 506 and the
"Modifier" input from the command pointer units 308a to 308c.
Is distributed to three adder / subtracters 503, and at the same time
The micro instruction of the instruction memory 506 is divided into three multipliers 502.
Distribute to Further, the distribution unit 310 includes the instruction register unit 3
09 are the multiplication instructions
In this case, those "op and Ri, Rj"
"Dest" part input from the instruction memory 506 and
Is distributed to the three multipliers 502 and the instruction memory 507
The micro instruction is distributed to three adder / subtracters 503. say
In other words, the distribution unit 310 allows the three adder / subtracters 503
Is supplied to three adder / subtracters 503
One common instruction is stored in the instruction memory 507 from the instruction memory 507.
Micro instructions are supplied to each of the three adders / subtracters 50.
Instruction register section 309
Are supplied to each. same
, The microinstructions supplied to the three multipliers 502
Are instructions for instructions common to the three multipliers 502
Micro-instructions are supplied from memory 506 and three multiplications
Instruction register section for multiplication instructions different in
Microinstructions from 309 are provided for each. According to such a configuration of the pixel operation unit 30,
If the storage capacity of the instruction memory 506 and the instruction memory 507 is
Can be reduced. If the pixel operation unit 30 receives the instruction
Inter holding units 308a to 308c, instruction register unit 309,
Assuming that the distribution unit 310 is not provided, the instruction memory 5
06 and the instruction memory 507 are all three execution units 50
To supply different micro-instructions for 1a-c,
Three microinstructions must be stored in parallel. FIG. 22 shows instruction pointer holding units 308a to 308a.
c, does not include the instruction register unit 309 and the distribution unit 310
The instruction memory 506 and the instruction memory 507
Here is an example of the content. In the figure, a 16-step micro
The program is stored and one microinstruction is 16 bits
G. In this case, the instruction memory 506 and the instruction
Mori 507 can record three microinstructions in parallel.
From the above, a total of 1536 bits (16 steps × 16 bits)
G × 3 × 2). On the other hand, the pixel operation unit 30 of this embodiment
Instruction pointer holding units 308a to 308c and an instruction register
FIG. 23 shows an example of the storage contents of the star section 309. In the figure
Also, a 16-step microprogram is stored
The micro instruction has 16 bits. In the figure,
The instruction pointer holding units 308a to 308c respectively store 16 records.
The register number (4 bits) is stored in the instruction register 3
09 stores 16 microinstructions. In this case, the life
Instruction pointer holding units 308a-c and instruction register unit 309
Is 448 bits (16 steps × (12+
16)). As described above, the pixel operation unit 30
Program memory capacity can be significantly reduced.
You. Actually, "dest" and "modify flag"
Since it is issued separately, the recording capacity for that
Or a circuit is required. Also, instruction memories 506 and 50
7 designates “dest” in the microinstruction, and
Issues multiplication and addition / subtraction instructions common to row units 501a to 501c.
Instruction memories 506 and 507
Has not been completely removed. If command
If six output ports are provided in the register section 309, the instruction
Memory 506 and instruction memory 507 can be deleted
become. In FIG. 23, the instruction pointer holding unit 3
08a to 308c indicate that the value of the first program counter is 0
Outputs the conversion address (register number) in the case of ~ 15
But not limited to this. For example, the first program
Outputs the conversion address when the counter value is 32 to 47
You may make it. In this case, the first program counter
Configuration to add an appropriate offset value to the
No. As a result, any arbitrary value indicated by the first program counter
The address string can be converted into a conversion address. With the above configuration, in this embodiment, the compression image
In addition to decoding image data and compressed audio data,
Encoding of video and audio data and polygon decoding
Data-based graphics processing.
You. In addition, parallel processing of multiple execution units improves processing efficiency.
It is above. Moreover, the instruction register units 308a to 308
In c, the order of some microinstructions has been changed
Resource interference among multiple execution units.
As a result, the processing efficiency is further improved. In the above embodiment, three execution units are provided.
The configuration that shows that each of the RGB colors is
This is because it can be operated independently. Further execution unit
May be any number as long as it is three or more. In addition,
In the embodiment, the video and audio processing devices 1000 and 2000
Are desirably implemented as one-chip LSIs. Sa
Furthermore, the external memory 3 is described as being outside the chip.
However, it may be configured to be built in one chip. In the above embodiment, the external memory
Stream input / output unit 1 (or stream input / output unit
21) is an MPEG stream (or video / audio data
Data), but the host processor directly
It may be configured to be stored in the memory 3. In addition,
In the embodiment, the IO processor 5 has four instruction cycles.
Task switching is performed every time, but other than 4 instruction cycles
May be performed for each of a plurality of instruction cycles. Also, task off
The number of replacement instruction cycles is weighted in advance for each task.
The number of instruction cycles may be different. Also priority
Weights the number of instruction cycles for each task according to degree and urgency
May be performed. According to the video / audio processing apparatus of the present invention, compressed audio
Data stream containing data and compressed video data
Input and decode from the
Video and audio processing device that outputs to the
I / O processing means that performs I / O processing that occurs asynchronously
And the data stored in the memory in parallel with the input / output processing.
Performs decoding processing mainly for data stream decoding.
Decoding processing means.
More decoded video data and decoded audio data
Data is stored in the memory, and the input / output processing is performed externally.
Inputting said data stream which is input asynchronously;
The data stored in the memory and the data stored in the memory.
Data stream to the decoding processing means,
To the output rate of each display device and audio output device.
To read from memory and output to them.
It is configured to perform as input / output processing. According to this configuration, the input / output processing means and the deco
The code processing means operates in parallel in a pipeline
In addition, asynchronous processing and decoding processing are performed by input / output processing means.
And decoding processing means.
The means is released from the processing that occurs asynchronously and
Be able to devote themselves to reasoning. As a result, this video and audio processing
The devices are called stream data input, decode, and output.
Stream processing is performed efficiently, so stream data
Full decoding (no dropped frames) of high-speed operation
This is possible without using a lock. Further, the decoding processing means includes a data stream
It is a sequential process mainly for condition judgment
Header analysis of compressed audio data and compressed video data
And decoding of the compressed audio data.
Serial processing means, and in parallel with the sequential processing,
I do. Routine processing excludes header analysis of compressed video data.
Standard processing means for decoding compressed video data
The configuration may be as follows. According to this configuration, sequential processing with different processing characteristics
Processing and routine processing suitable for parallel processing in one unit
Greater processing efficiency by eliminating coexistence
Can be improved. In particular, the processing of routine processing means
Efficiency can be improved. Because this video and audio processing
In the processing device, the routine processing means performs the above asynchronous processing and
Since it was released from sequential processing, compressed video data
Can be used exclusively for routine various operations required for code
This is because that. As a result, do not use a high-speed operation clock.
High processing power can be obtained. Further, the input / output processing means may be provided externally.
An input means for inputting an asynchronous data stream and an external
Video output for outputting decoded video data to a display device
Output means and audio data decoded by an external audio output device.
Voice output means for outputting data,
Process to execute while switching the first to fourth tasks
A first task from the input unit to the memory
A program for transferring a data stream to the
The second task is to transfer data from the memory to the decode processing means.
A program for supplying a stream, wherein the third task
The video data is decoded from the memory to the video output unit.
The fourth task is a program that outputs data
From the memory to the audio output unit.
The program may be configured to output. Here, the processor is configured to execute
At least four program cows corresponding to the fourth task
Program counter unit with a
Each task program is executed using the instruction address indicated by the program counter.
Instruction fetching an instruction from the instruction memory
Command section and instructions to execute the instruction fetched by the instruction fetch section.
The instruction execution unit is instructed every time a predetermined number of instruction cycles elapse.
Program counter for instruction fetch section
With a task control unit that controls
Good. According to this configuration, it is determined by the external device.
Input rate and input cycle of stream data, external display
Video data and audio determined by the device and external audio output device
What is the output rate and output cycle of each data
Response delay for I / O requests
It has the effect of being small. In addition, the video / audio processing of the present invention
The device includes data including compressed audio data and compressed video data.
Input means for inputting a data stream, and a data stream
Is a sequential process that mainly focuses on
Header added for each block in the stream
Analysis of information and compression audio data in the data stream
A sequential processing means for performing decoding and a
Data processing using the result of header analysis.
The compressed video data in the stream is
Standard processing means for decoding in predetermined block units.
The sequential processing means analyzes the header of the predetermined block.
When the processing of the predetermined block is completed,
Instruct the start of the code,
When a decoding end notification is received, the
The configuration may be such that the header analysis is started. According to this configuration, the sequential processing means performs the compression
Wide variety of image data and compressed audio data
Responsible for header analysis that requires judging conditions
Also responsible for decoding compressed audio data. On the other hand,
Processing means for the block data of the compressed video data,
Responsible for a large amount of routine calculations. Such division of roles
And the sequential processing means perform in comparison with video decoding.
Audio decoding, which is less complex, and compressed video data
The analysis of the data and the control of the routine processing means are performed. Of that control
Below, the routine processing means performs routine calculations exclusively,
Lean and efficient processing can be realized. Therefore high lap
Processing power can be obtained without operating at wavenumber,
The manufacturing cost can be reduced. In addition,
The column shows the general audio decoding and header decoding of the compressed video data.
Analysis and control of routine processing means are performed sequentially, so that
It can be composed of a heat sink. The routine processing means may be a sequential processing means.
The compressed video data in the data stream is
Data conversion means for variable-length decoding, and variable-length decoding
To perform a predetermined operation on the video block
Calculating means for performing more inverse quantization and inverse discrete cosine transform;
Video block after inverse discrete cosine transform and decoded block
To perform motion compensation by combining
Synthesizing means for restoring the data, wherein the sequential processing means
Obtains header information that has been variable-length decoded by data conversion means
Acquisition means to perform, and analysis to analyze the acquired header information
Standard processing of means and parameters obtained as analysis results
Notification means for notifying the means, and input by the input means
Audio decoding to decode compressed audio data in the data stream
Decoding means and decoding of a predetermined block from the routine processing means.
When receiving an interrupt signal notifying that the
The operation of the step is stopped and the acquisition means is started, and the
When the informing means gives the notification, the information is displayed on the data converting means.
Control means for instructing the start of variable-length decoding of an image block.
You may comprise so that it may have. According to this configuration, a location such as a macro block
The sequential processing means performed header analysis on a fixed block basis.
After that, perform audio decoding and use the
When the decoding of the block is completed, the header analysis of the next block
To start. In this way, the sequential processing means uses the time division header
One processor to repeat analysis and audio decoding
At low cost. Also, routine processing
The means does not need to perform a wide variety of condition judgment processing
Low cost, dedicated hardware (or hardware
Firmware). Here, the calculating means further comprises one block.
A first buffer having a storage area corresponding to the
The data conversion means includes a compressed video data in the data stream.
Length decoding means for variable length decoding data, and a first buffer
The addresses of the storage areas are arranged in zigzag scan order.
First address table means for storing one address string;
The address of the storage area of the first buffer is set to an alternate
Second address for storing a second address sequence arranged in the order of the channels
Table means, one of the first address string and the second address string
According to the variable length decoding means obtained by the variable length decoding means
For writing block data to be written to the first buffer
It may be configured to have a step. According to this configuration, the writing means is provided with a zigzag
Gscan and alternate scan
Write the block data to the storage area of the first buffer
Can be taken. Therefore, the calculating means stores the data in the first buffer.
When reading block data from the storage area,
You do not need to change the order of the dresses, scan type
Regardless of the read address, read in the same order
be able to. [0114] Further, the analyzing means may be configured to execute the processing based on the header information.
Calculate the quantization scale and the motion vector based on the
The notifying means uses the quantization scale as the calculating means,
May be configured to notify the combining means. This structure
According to Sung, the calculation of the motion vector is in charge of the sequential processing means
And the combining means calculates the calculated motion vector.
The motion compensation processing can be routinely performed by using. . Further, each of the arithmetic means is provided with a microcontroller.
A first and second control storage unit for storing a program;
A first program for designating a first read address in the control storage unit
And a second program for specifying a second read address.
RAM counter, first read address and second read address
Selector for selecting one of the above and outputting the selected one to the second control storage unit
And a multiplier and an adder, and the first and second control storage units
Inverse amount per block by microprogram control
And an execution unit for performing the inverse transformation and the inverse discrete cosine transform.
It is good also as a result. According to this configuration, the microprogram
(Firmware) performs a wide variety of condition judgment processing
Since there is no need to implement routine processing,
Small ram size, easy to make, low cost
Suitable for. Moreover, two program counters are used.
To operate the multiplier and the adder independently and in parallel.
Can be. Further, the execution unit is provided with a selector by a selector.
2 When a read address is selected, processing using a multiplier
And processing using the adder are performed independently and in parallel, and the selector
When the first read address is selected by the
Process and the process using the adder
You may comprise. According to this configuration, the multiplier and the adder
Can reduce the play time and improve the processing efficiency
You. Here, the calculating means further comprises:
First buffer holding video block from conversion means
And the block that has been subjected to inverse discrete cosine transform by the execution unit
And the first control storage unit is configured to
Microprogram for quantization processing and inverse discrete cosine transform
And the second control storage.
The microprogram for inverse discrete cosine transform and the inverse
Transfer the cosine transformed video block to the second buffer
And the execution means,
Transfer the discrete cosine transformed video block to the second buffer
And the process of dequantizing the next video block
Execute in parallel and inversely quantize the inversely quantized video block
The cosine transform is performed by linking the multiplier and the adder.
It may be configured to execute. According to this configuration, the inverse quantization processing and the second
Process is performed in parallel with the transfer process to the
Can be up. Further, the input means further comprises:
Inputting polygon data, the sequential processing means further
Analyze polygon data to determine polygon vertex coordinates and edges
And the routine processing means further calculates the inclination of
The vertex coordinates and the inclination, and therefore the image data of the polygon
Data may be generated. According to this configuration, the sequential processing means is a polygon.
Data analysis, and the standard processing means uses standard images.
Responsible for data generation processing. This video and audio processing device
Graph that efficiently generates image data from polygon data
Ix processing. Here, the first,
The second control storage unit is further configured to run by the DDA algorithm.
Storing a microprogram for performing the conversion,
Are the vertex coordinates calculated by the sequential processing means and the inclination.
Scan conversion by microprogram control based on time
May be configured to be performed. According to this configuration, the image data is generated before.
The scan conversion microprogram is stored in the first and second control storage units.
Can be easily realized. In addition, the synthetic hand
The stage represents a difference representing a difference image from the video data to be further compressed.
A second block, wherein said second buffer is further generated.
The first control storage unit further stores the obtained difference image.
Microprogram for string conversion and microphone for quantization
And the second control storage unit is further discrete
Microprogram for cosine transformation and discrete cosine transformation
Microprogram for transferring video block to first buffer
And the execution means further stores in the second buffer
Discrete cosine transform and quantization on retained difference blocks
To transfer the data to the first buffer,
Is also variable length coded for the block in the first buffer
And the sequential processing means further includes a data conversion means.
Header information for a given block that has been
The information may be added. According to this configuration, the routine processing means is of a routine type.
In charge of quantization and discrete cosine transform
Means is responsible for processing that requires condition judgment (addition of header information).
Hit. In this case, the video and audio processing device
To convert compressed image data from image data without using
Code processing can be performed efficiently. Also,
The arithmetic means stores a microprogram, respectively.
The first and second control storage units and the first control storage unit
A first program counter for specifying an output address;
A second program counter for designating a read address;
Select one of the first read address and the second read address
A selector for outputting to the second control storage unit, a multiplier and an adder
And a microphone by the first and second control storage units.
Block-based inverse quantization and inverse separation by program control
And a plurality of execution units for performing a cosine transform.
Section shares and processes the divided partial blocks
May be configured. According to this configuration, a plurality of execution units are connected in parallel.
Executes operation instructions, so a large amount of routine operations can be performed at the pixel level.
It can be executed efficiently by parallelizing with a bell. Ma
Further, the arithmetic means is further provided corresponding to each execution unit.
Each conversion table is associated with a predetermined address string.
Maintains translated addresses that have their address order rearranged
Implement multiple address translation tables and predetermined operations
Change the individual microinstructions that make up the microprogram
Consists of multiple registers that are stored in correspondence with the replacement address
An instruction register group, the first and second control storage units,
Provided between the first control storage unit and the selector.
Micro instruction output to each execution unit from the instruction register
Switch to micro instructions and output to multiple execution units.
A first read address or a second read address.
The address is an address in the predetermined address sequence
In this case, the address is determined according to each address conversion table.
Is converted to a conversion address. Instruction register group
Corresponds to each translation address output from the translation table
May be configured to output a microinstruction. According to this configuration, a plurality of execution units are connected in parallel.
Access between execution units during execution of microprogram
Avoids resource interference such as
Can be Here, each of the conversion tables further includes
And the first program counter in the predetermined address string
During output of the first read address, adjustment in the register
Should be added or subtracted with the output of the microinstruction indicating the operation
Outputting a flag indicating whether to perform the processing to the plurality of execution units;
Each execution unit performs addition and subtraction according to the flag,
The flag is in accordance with a microinstruction in the second control storage unit.
It may be configured to be set. According to this configuration, processing is performed by a microinstruction.
The conversion table specifies whether to perform arithmetic or subtraction
So the same microprogram can be shared in two ways
And further reduce the overall capacity of the microprogram
Hardware scale, and thus low cost
Can be realized. Further, the second control storage unit further stores
And the first program counter in the predetermined address string
While outputting the first read address, the
With the output of the micro instruction, the storage destination of the micro instruction execution result
Is output to the plurality of execution units, and each of the execution units
Is configured to store the execution result according to the storage location information
May be. According to this configuration, the storage destination information is stored in the instruction record.
Can be specified separately from the microprogram in the
The microprogram is processed differently, for example, matrix operation.
Can be shared in the partial processing of. The result
As a result, further reduce the total capacity of the microprogram
Hardware scale, and thus low cost
Can be realized.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an explanatory diagram of a decoding process by a video / audio decoder according to a first conventional technique. FIG. 2 is an explanatory diagram of a decoding process by a two-chip decoder according to a second conventional technique. FIG. 3 is a block diagram illustrating a schematic configuration of the image processing apparatus according to the first embodiment of the present invention. FIG. 4 is a block diagram illustrating a configuration of an image processing apparatus according to the first embodiment of the present invention. FIG. 5 is a diagram showing an MPEG stream in a hierarchical manner and showing operation timing of each section of the image processing apparatus. FIG. 6 is a diagram showing an analysis of a macroblock header by a processor 7 and contents of control to other units. FIG. 7 is a block diagram illustrating a configuration of a pixel operation unit 10. FIG. 8 shows a first instruction memory 506 and a second instruction memory 50;
7 shows an example of the microprogram stored in FIG. FIG. 9 is a diagram showing operation timings of the pixel operation unit 10; FIG. 10 is a block diagram showing a detailed configuration of a pixel read / write unit 11; FIG. 11 is a block diagram showing a configuration of an IO processor 5. FIG. 12 is a block diagram showing a detailed configuration example of an instruction reading circuit 53. FIG. 13 is a time chart showing the operation timing of the IO processor 5; FIG. 14 is a block diagram illustrating a configuration of a task management unit. FIG. 15 is an explanatory diagram showing a decoding operation after the FIFO memory 4; FIG. 16 is a block diagram illustrating a configuration of an image processing apparatus according to a second embodiment of the present invention. FIG. 17 is a block diagram illustrating a configuration of a pixel operation unit 30. FIG. 18 shows a first instruction memory 506 and a second instruction memory 50
7 shows an example of the storage content of No. 7. FIG. 19 is a block diagram showing a configuration of a code conversion unit 9; FIG. 20 shows a block storage area for storing 8 × 8 spatial frequency data and a zigzag scan route. FIG. 21 shows a block storage area for storing 8 × 8 spatial frequency data and a route of an alternate scan. FIG. 22 shows an example of the contents stored in the instruction memory 506 and the instruction memory 507 when the instruction pointer holding units 308a to 308, the instruction register unit 309, and the distribution unit 310 are not provided. FIG. 23 shows an example of the contents stored in the instruction pointer holding units 308a to 308c and the instruction register unit 309. [Description of Signs] 1 Stream input unit 2 Buffer memory 3 External memory 4 FIFO memory 5 Input / output processor 5a DMAC 6 Memory controller 7 Processor 8 Internal memory 9 Code conversion unit 10 Pixel operation unit 12 Video output unit 13 Audio output unit 14 Host I / F unit 1000 Video / audio processing device 1001 Input / output processing unit 1002 Decoding processing unit 1003 Sequential processing unit 1004 Standard processing unit

────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Kozo Kimura 1006 Kazuma Kadoma, Kadoma, Osaka Prefecture Matsushita Electric Industrial Co., Ltd. (56) References JP-A-8-1111642 (JP, A) 326615 (JP, A) Japanese Patent Laid-Open No. 9-37249 (JP, A) Realization of real-time decryption that can support HDTV, Ken Ohto, Toshinori Otaka, Takayasu Sakurai, Nikkei Electronics, Nikkei BP, March 14, 1994 March 14, 1994, p. 93-100 (58) Fields surveyed (Int. Cl. ⁷ , DB name) H04N ^7/ 24-7/68

Claims (1)

(57) [Claims] 1. An input means for inputting a data stream including compressed audio data and compressed video data, and a block in the data stream, which is a fixed form processing mainly including a fixed form calculation. Routine processing means for performing variable length decoding of the header information added to each unit and the compressed video data in the data stream, and decoding of the variable length decoded compressed video data in block units; Analysis processing of header information subjected to variable length decoding processing by the standard processing means,
Serial processing means for performing time-division decoding processing of compressed audio data in the data stream, wherein the routine processing means performs variable-length decoding processing of header information and compressed video data, and performs variable-length decoding of compressed video data blocks. After the long decoding processing is completed, the sequential processing means is instructed to start the analysis processing of the header information of the next block, and the sequential processing means has the variable length encoded header information to be analyzed. Sometimes, the fixed form processing means is instructed to perform variable length decoding processing of the header information, and when the variable length decoded header information is obtained, it is analyzed, and after the acquisition of the header information of the block is completed, the fixed form processing means is Then, a variable length decoding process of the compressed video data of the block is started, and the compressed video data after the variable length decoding is decoded using the analyzed header information. Video and audio processing apparatus characterized by instructing the.