JPH08298666A - Image processor - Google Patents

Abstract

PURPOSE: To reduce the capacity of an interlacing conversion memory in an MPEG decoder for reproducing moving image data. CONSTITUTION: This image processor is provided with a 1st frame memory(FM0) 11 and a 2nd frame memory(FM1) 12 each of which consists of 2N slots and a 3rd frame memory(FM2) 13 consisting of (N+4) slots. Each slot has capacity for storing 8 lines of a picture. The capacity of each of the FM0 11 and FM1 12 for storing a reference frame of compensating movement is one frame and the capacity of the FM2 13 for the interlacing conversion of a B picture is (a half frame + 32 lines). The picture processor is also provided with a slot management memory(SM) 14 consisting of (2N+6) words each of which stores one slot number of the FM2 13. In the case of writing data in the FM2 13 by a decoding part 15 the contents of the SM14 are updated by a control part 17 so that an output part 16 can read out data from the FM2 13 in the correct order of slots.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus preferably used for decoding image information.

[0002]

2. Description of the Related Art As an international standard for compression and expansion of moving image data, it is generally called MPEG (Moving Picture Image C) under the name of the working group of ISO / IEC.
The international standard known as the odingExperts Group) is known. An MPEG decoder for reproducing moving image data is a variable length decoder (VL).
D), an inverse quantizer (Inverse Quantizer: IQ),
Inverse Discrete Cosine Tran
sformer: IDCT) and motion compensator
The data processing unit constituted by ator: MC) is a main component. MPEG decoders also require multiple frame memories for motion compensation and interlace conversion.

MPEG is characterized by the use of two frames, a temporally previous image and a later image, for motion compensation. On the other hand, if motion compensation is used for all images, there are problems such as error propagation and special reproduction, so I (Intra-coded) pictures and P (Predictive-cod)
ed) picture and B (Bidirectionally predictive-cod)
ed) picture has been introduced. An I picture, a coding type I picture, does not reference any other picture. P picture, that is, coding type P
The picture of is only motion-compensated from the temporally previous frame. A B picture, that is, a picture of coding type B, is bidirectionally motion-compensated from a temporally previous frame and a temporally subsequent frame. B-pictures are not used as reference frames when decoding other frames.

The manner of prediction of each coding type will be described. It is assumed that the input picture bit stream is given to the MPEG decoder in the order of I0, P3, B1, and B2. P3 is motion compensated from I0, B
1 is motion compensated from I0 and P3, B2 is I0 and P3
3 is motion compensated. Display is I0, B1, B2, P
It is done in the order of 3. As described above, in the MPEG decoder, since the decoding order and the display order do not match, it is necessary to change the order in the MPEG decoder. Further, since data of two image frames of I0 and P3 is required for decoding B1 and B2, a frame memory for two image frames is required for motion compensation. Therefore, the MPEG decoder requires two frames for reference of motion compensation.

Next, the order of decoding in pixel units of MPEG and the order of pixel units in outputting an image will be described. In a television or the like, pixels are output in an order from upper left to lower right by skipping one line, such as outputting only even lines first and then outputting only odd lines. The portion of only the even lines is called the top field, and the portion of the odd lines only is called the bottom field. It can be said that the interlaced output is such that the top field is first output from the upper left to the lower right, and then the bottom field is output from the upper left to the lower right.

Image data is two-dimensional, and it is considered that data at spatially close positions have a high correlation, but in the case of interlaced output, for example, considering one line with a top field, the one line above is the bottom field. Will belong to. That is, the pixels on one line are very close in space, but are distant in time.
Therefore, if the movement is vigorous, the correlation with two lines closer in time may be higher than that with one line. In consideration of such a case, in MPEG, the decoding order in pixel units is roughly divided into two types, that is, a frame structure and a field structure.

In MPEG, 16 × 16 pixels are used as a basic unit called one macroblock for decoding. Macroblocks are decoded in order from left to right, but here, for example, the rightmost pixel of one line at the top of the image is included in the rightmost macroblock of the screen. . On the other hand, when the decoding of the rightmost macro block is completed, the result is that 16 lines of data have been decoded.
Therefore, in MPEG, decoding of 16 lines of data ends almost at the same time.

In the case of the frame structure, the data of one frame of an image constitutes a macroblock of 16 pixels in the vertical direction and 16 pixels in the horizontal direction and is decoded for each macroblock. Therefore, the top field and the bottom field are decoded almost at the same time. Therefore, the order of image output does not match at all, so order conversion is required.

In the case of the field structure, the image frame is divided into a top field and a bottom field, each field constitutes a macroblock of 16 pixels in the vertical direction and 16 pixels in the horizontal direction, and each macroblock is decoded. . In this case, one macroblock has only the top field or only the bottom field, and the bottom field data is decoded after all the top field data is decoded. In this case, the order is almost the same as the image output, but since the decoding order is performed in macroblock units, it does not completely match the image output order.

Image output is performed in the order of top field and bottom field. Focusing on the time when the last 8 lines of the top field are output, this final 8
Before the line output is started, the last 16
Decoding of macroblocks for lines must be completed. This is because the value of the rightmost 16 pixels in the last 8 lines is fixed only after decoding the last macroblock of the image. Therefore, at the above time point, decoding must be completed for both the bottom field and the top field. On the other hand, after this point, the top field 8 lines and the bottom field must be sequentially output, but this data has already been decoded. Therefore, unless the 8 lines of the top field and all the data of the bottom field are stored in the frame memory, the data disappears before the data is output and the image cannot be output. In other words, a frame memory having a capacity of about half a frame for storing all the data of the bottom field and the data of 8 lines of the top field is required.

In summary, two frames of memory are required for motion compensation, and about half a frame of memory is required for interlace conversion. That is, a minimum of about 2.5 frames of memory is required.

Shunichi Ishiwata et al. "MPEG2 Decoder LSI"
, "Efficient Memory Allocation", Proc. Of the 1994 IEICE Spring Conference, C-659,199
In March 4th, an example of an MPEG decoder using a memory for 1.5 frames for interlace conversion is described.

[0013]

The above-mentioned conventional MPEG decoder has a problem that the cost of the MPEG decoder becomes high because it requires a memory for 1.5 frames for interlace conversion. As described above, there is room for improvement in view of the fact that about half a frame of memory is sufficient for interlace conversion in principle.

An object of the present invention is to reduce the capacity of the frame memory included in the MPEG decoder.

[0015]

In order to achieve the above object, the present invention focuses on the following points. That is, the B picture is no longer used after the image output is completed, and it is possible to predict when the area of the macro block currently being decoded will be output by analyzing the additional information portion of the input picture. It focuses on and. Specifically, divide the data memory into multiple slots,
It was decided to control reading and writing of the data memory by using the slot number stored in the slot management memory.

According to the present invention, the contents of the slot management memory are updated when the data memory is written so that the data memory is read in the correct slot order. Therefore, if this data memory is used as an interlace conversion memory, interlace conversion of B pictures can be realized with a memory for about half a frame.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION A specific example of an MPEG decoder for realizing real-time reproduction processing of moving image data will be described below with reference to the drawings.

FIG. 1 shows the structure of the MPEG decoder of the present invention. The MPEG decoders 10 shown in FIG.
A first frame memory (FM0) 11 for storing image data for frames and a second frame memory (F0)
M1) 12 and a third frame memory (FM2) 13 for interlace conversion of B pictures. These three frame memories 11, 12, 13 are
Each is divided into a plurality of slots. MPEG
The decoder 10 includes a third frame memory (FM2) 13
Slot management memory (SM) 14 for storing a plurality of slot numbers of the image data, and image data for decoding the input picture IN are stored in three frame memories 11, 12,
Decoding unit 15 for writing to any of 13 and 3
An output unit 16 for reading data from any one of the frame memories 11, 12, and 13 to supply an output picture OUT, and a slot management memory (SM) 14 to refer to write slot numbers WS1 and WS2 and read slot numbers. Control unit 1 for supplying RS1 and RS2
7 and 7 are further provided. In FIG. 1, 21 is an address bus and 22 is a data bus.

FIG. 2 shows the first frame memory (FM0).
11 shows an internal configuration of 11. The first frame memory (FM0) 11 is composed of 2N slots numbered from 0 to 2N-1. Where N is MP
The amount depends on the image size to be decoded by the EG decoder 10. For example, NTSC (National Televis
In the image of (ion System Committee), N is 30. Each slot has a capacity for storing 8 lines of image. That is, the first frame memory (F
M0) 11 is data for 480 lines, that is, NT
Data for one frame of the SC image can be stored.

FIG. 3 shows the second frame memory (FM1).
12 shows the internal configuration of the twelve. The second frame memory (FM1) 12 is also the first frame memory (FM0) 1
Similar to 1, it consists of 2N slots numbered from 0 to 2N-1. Each slot is
It has the capacity to store 8 lines of image.

FIG. 4 shows a third frame memory (FM2).
13 shows an internal configuration of 13. The third frame memory (FM2) 13 has N numbered from 0 to N + 3.
It consists of +4 slots. Each slot has a capacity for storing 8 lines of image.

FIG. 5 shows a slot management memory (SM) 14
Shows the internal configuration of the. Slot management memory (SM)
14 is composed of 2N + 6 storage locations (words) from addresses 0 to 2N + 5. Each word stores one slot number of the third frame memory (FM2) 13.

FIG. 6 shows the internal structure of the decoding section 15. The decoding unit 15 includes a VLD 31 and an IQ 32.
The data processing unit 30 including the IDCT 33 and the MC 34 is a main component. Decoding unit 15
Is a data writing unit 35 and a memory selection register 36.
And a row counter 37.

The decoding unit 15 decodes the bit stream of the input picture IN, and outputs the resulting data D1 as the first frame memory (FM0) 11, the second frame memory (FM1) 12 or the third frame memory. (FM2) 13 plays a role of writing. Also,
As a result of the analysis of the input picture IN, the mode signal MODE,
The type signal TYPE and the write row number WROW are output. When performing decoding, the first frame memory (FM0) 11 or the second frame memory (FM1) 1
The data written in 2 is referred to as motion compensation prediction image data D2. A1 is an address transmitted to the address bus 21. Third frame memory (F
When writing to M2) 13, the write slot numbers WS1 and WS2 given from the control unit 17 are used. The operation timing of the decoding unit 15 is the output unit 16
Controlled by an interrupt signal INT from

The input picture IN is divided into a portion of additional information such as a coding type, a frame structure and a field structure, and a portion of pixel data. The additional information portion is input to the VLD 31 before the pixel data portion. Input picture IN to VLD31
, The type signal TY indicating the coding type (“I”, “P” or “B”) of the image frame is given.
The PE and the mode signal MODE indicating the frame structure (“FR”) or the field structure (“FD”) are output from the VLD 31. The type signal TYPE and the mode signal MODE do not change until the decoding of one frame of the image is completed and the decoding of the additional information of the next image is started. Next, the process for the pixel data portion starts, but the VLD 31 waits for the reception of the interrupt signal INT every 16 lines of image data. This operation is stopped because the output unit 16 synchronizes with a portion for reading data, and the specific timing will be described later. The 16 lines correspond to 16 which is the number of vertical pixels included in one macroblock. When the decoding of the image of one frame is completed, the VLD 31 starts decoding the next image again.

The row counter 37 indicates, in units of 16 lines, which part of the image is currently being decoded, and is rewritten by the VLD 31. The row counter 37 is incremented by 1 each time the VLD 31 finishes decoding the image 16 lines. Also, V
The row counter 37 is set to 0 when the LD 31 is decoding the additional information. The content of the row counter 37 is output to the control unit 17 as the write row number WROW, and is also used for determining the slot when the data writing unit 35 writes the pixel data in the memory.

The final decoded image MCP created by the MC 34 is processed by the data writing unit 35 by the first frame memory (FM0) 11, the second frame memory (FM1) 12 or the third frame memory (FM2). 1
Written in 3. The data writing unit 35 has write slot numbers WS1 and WS2 and write row number WRO.
The address A1 is generated by referring to W, the type signal TYPE, the mode signal MODE, and the value WQ of the memory selection register 36, and the data D1 is sent to the data bus 22. The value WQ of the memory selection register 36 is stored in the data writing unit 35.
It affects the selection of the write frame memory in the above step and the selection of the prediction frame memory in the MC 34.

FIG. 7 shows the internal structure of the output section 16. The output unit 16 includes a data reading unit 41, a memory selection register 42 for holding a value RQ, a row counter 43 for supplying a read row number RROW,
It is composed of a border color register 44 for holding the border color signal BCL. The data reading unit 41 includes a first frame memory (FM0) 11 and a second frame memory (FM0) 11.
The decoded data D3 written in the frame memory (FM1) 12 or the third frame memory (FM2) 13 of
To output the output picture OUT. A2 is
The address is sent to the address bus 21. Also,
The data reading unit 41 outputs an interrupt signal INT for controlling the operation timing of other blocks. The position where the data reading unit 41 reads the data is determined by the type signal TYPE from the decoding unit 15 and the read slot numbers RS1 and RS2 from the control unit 17.

The output unit 16 includes three frame memories 1
It has a role of outputting data read from any one of 1, 12, and 13 to the outside at a correct timing. The output picture OUT has a vertical blanking interval and an active period for actually outputting an image. Top field data is output during the active period following the first vertical blanking interval, and bottom field data is output during the next active period. The output unit 16 starts the vertical blanking interval or starts outputting data for 16 lines.
The interrupt signal INT is output. This interrupt signal IN
T includes information such as whether or not the time when the interrupt signal INT is issued is the start of the vertical blanking interval, and which part of one frame of the image is currently output.

FIG. 8 shows the internal structure of the controller 17. The control units 17 each have a slot management memory (SM) 1
Write pointer (M
1) 51 and a read pointer (M2) 53, a first slot number designating section 52 for designating a slot in which the decoding section 15 writes data, and a second slot number designating section for designating a slot in which the output section 16 reads data. Slot number designating section 54 and a memory control section 55 for controlling the slot management memory (SM) 14.
61 is an address bus, 62 is a data bus, A is an address supplied to the slot management memory (SM) 14, and D is data (slot number) on the data bus 62. Slot management memory (SM) 14 and memory control unit 55
The write pointer (M1) 51, the read pointer (M2) 53, and the first and second slot number designation units 52 and 54 perform memory management at the time of B-picture interlace conversion.

The first slot number designating section 52 fetches two slot numbers from the address of the slot management memory (SM) 14 pointed to by the write pointer (M1) 51, and writes them into the decoding section 15 as write slot numbers WS1 and W.
Supply as S2. The timing of this output is based on the interrupt signal INT from the output unit 16 and the type signal TYPE from the decoding unit 15.

The memory control section 55 uses the write pointer (M1) 51 as an address to control the slot management memory (S
M) 14 two slot numbers are taken out, and those slot numbers are written in another address of the slot management memory (SM) 14. The two slot numbers retrieved are
It matches the write slot numbers WS1 and WS2.
The different address is calculated from the write pointer (M1) 51, the mode signal MODE, and the write row number WROW. The timing of this operation is based on the interrupt signal INT from the output unit 16 and the type signal TYPE from the decoding unit 15. After the writing to the slot management memory (SM) 14 is completed and the first slot number designating unit 52 designates the writing slot numbers WS1 and WS2 to the decoding unit 15, the memory control unit 55
Increases the write pointer (M1) 51 by 2.

The second slot number designation unit 54 takes out two slot numbers from the address of the slot management memory (SM) 14 pointed to by the read pointer (M2) 53, and outputs the read slot numbers RS1 and RS2 to the output unit 16.
Supply as. After that, the second slot number designation unit 5
4 increments the read pointer (M2) 53 by 2. The timing of this operation is the interrupt signal INT from the output unit 16 and the type signal TY from the decoding unit 15.
Based on PE.

Next, selection of a write frame memory and selection of a reference frame memory for motion compensation will be described. These memory selections are determined when the additional information of one frame of the image is decoded, and the same memory is used until the decoding of the one frame of the image is completed. The value WQ of the memory selection register 36 can take "0" or "1", and its initial value is "0".
The operation is switched as follows according to the type signal TYPE from the VLD 31.

First, the case where TYPE = "I" will be described. If the WQ = “0”, the data writing unit 35 determines the first
The frame memory (FM0) 11 is selected as the write frame memory, and the second frame memory (FM1) 12 is selected as the write frame memory if WQ = “1”. When the writing of the I picture is completed, the value WQ of the memory selection register 36 is updated before the next picture is decoded. That is, if WQ = "0", it is updated to "1", and if WQ = "1", it is updated to "0". No motion compensation is performed.

The case where TYPE = "P" will be described. The data writing unit 35 selects the first frame memory (FM0) 11 as the write frame memory when WQ = "0" and the second frame memory (FM1) 12 when WQ = "1". Furthermore, the data writing unit 3
5 is the second frame memory (FM) if WQ = “0”.
1) 12 and if WQ = “1”, select the first frame memory (FM0) 11 for forward reference.
When the writing of the P picture is completed, the value WQ of the memory selection register 36 is read before the next picture is decoded.
Will be updated. That is, if WQ = "0", then "1"
If WQ = "1", it is updated to "0".

The case where TYPE = “B” will be described. Writing is always performed in the third frame memory (FM2) 13. At this time, the data writing unit 35 uses the WQ
= “0”, the first frame memory (FM0) 11
For forward reference, the second frame memory (FM1)
Respectively for the back reference. Also, WQ =
If it is "1", the second frame memory (FM1) is selected for forward reference and the first frame memory (FM0) 11 is selected for backward reference. In this case, the value WQ of the memory selection register 36 is not updated.

FIG. 9 shows a schematic operation of the decoding section 15. In FIG. 9, the bit stream of the input picture IN is decoded by the decoding unit 1 in the order of I0, P1, P4, B2, and B3.
It is supposed to be given to 5. In the expressions I0 and P4, the first I, P, and B represent the coding type, and the following numbers represent the display order.

First, when the picture I0 is input, TY
Since PE = “I” and WQ = “0”, the picture I0 is written in the first frame memory (FM0) 11. When the decoding of the picture I0 is completed, WQ is updated to "1". When the picture P1 is input, T
Since YPE = “P” and WQ = “1”, the motion-compensated picture P1 is the second frame memory (FM1).
12 is written. At this time, the MC 34 refers forward of the picture I0 already written in the first frame memory (FM0) 11. When the decoding of the picture P1 is completed, WQ is updated to "0". When the picture P4 is input, since TYPE = “P” and WQ = “0”, the motion-compensated picture P4 is written in the first frame memory (FM0) 11. At this time, MC3
4 forward-references the picture P1 already written in the second frame memory (FM1) 12. Picture P
When the decoding of 4 is completed, WQ is updated to "1". When the picture B2 is input, since TYPE = “B”, the motion-compensated picture B2 is written in the third frame memory (FM2) 13. At this time, WQ
= “1”, the MC 34 has the picture P1 already written in the second frame memory (FM1) 12.
Forward reference, and the first frame memory (FM0) 1
The picture P4 already written in 1 is back referenced. WQ is not updated. When the picture B3 is input, since TYPE = “B”, the motion-compensated picture B3 is written in the third frame memory (FM2) 13. At this time, since WQ = “1”, MC34
Refers forward to the picture P1 already written in the second frame memory (FM1) 12 and backward refers to the picture P4 already written to the first frame memory (FM0) 11. WQ is not updated.

As described above, for the I picture and P picture, the first frame memory (FM0) 11 and
The second frame memory (FM1) 12 is used alternately. For B pictures, the third frame memory (FM2) 13 is always written, and the first frame memory (FM0) 11 and the second frame memory (FM1) 12 are used as reference frames for motion compensation. To do.

Next, description will be made on which slot in the selected frame memory is to be written. The data writing unit 35 determines two write slots for every 16 lines and writes a total of 16 lines of data in the two slots of the write frame memory selected by the method already described. The write slot has a type signal T
YPE, mode signal MODE, write row number W
It is determined by the ROW and the write slot numbers WS1 and WS2 given from the first slot number designating section 52. The procedure is as follows.

First, the case where TYPE = “I” or “P” will be described. If MODE = “FR (frame structure)”, the decoded 16 lines are divided into two fields, top field 8 lines and bottom field 8 lines, and the top field 8 lines go to a slot having the same number as the write row number WROW. , The bottom field 8 lines are respectively written in the slots having the numbers of WROW + N. If MODE = “FD (field structure)”, the decoded 16 lines are the upper 8 lines and the lower 8 lines.
It is divided into two lines, the top 8 lines are WROW x 2
To the slot numbered, the lower 8 lines are WROW x 2
It is written in each slot having a number of +1. According to the above procedure, in both the frame structure and the field structure, only the top field data is written in the area having the slot number smaller than N, and only the bottom field data is written in the area having the slot number N or more. Moreover, in the same field, the order of increasing the slot number matches the order of writing the decoded image data, and also the order of interlaced output. Therefore, if such a writing method is performed, interlace output can be realized in the correct order by simply reading the image output in the order of increasing slot numbers. This reading will be described later in detail.

The case where TYPE = “B” will be described. MO
If DE = “FR (frame structure)”, the decoded 16 lines are divided into two, top field 8 lines and bottom field 8 lines, and the top field 8
The line is written to the slot having the write slot number WS1 and the bottom field 8 line is written to the slot having the write slot number WS2. MOD
If E = “FD (field structure)”, the decoded 16 lines are divided into two, upper 8 lines and lower 8 lines, the upper 8 lines are written to the slot having the write slot number WS1, and the lower 8 lines are written. It is written in each slot having the slot number WS2. Each slot of the third frame memory (FM2) 13 does not store top field data and bottom field data at the same time. In this case, since the slot number is given from the first slot number designating section 52, the operation of the data writing section 35 is simple.

Next, selection of the read frame memory will be described. The read frame memory is determined by the type signal TYPE at the start of image output of the top field and the value RQ of the memory selection register 42. The value RQ of the memory selection register 42 is “0”, “1” or “2”
Can be taken, and its initial value is "2". Also,
The border color register 44 is used to specify the initial color.

First, the case where TYPE = "I" or "P" is described. If RQ = “2”, the data reading unit 4
1 outputs the border color signal BCL set in the border color register 44 as the output picture OUT, and updates RQ to “0” when the output of one frame is completed. If RQ = “0”, the data reading unit 41 reads data from the first frame memory (FM0) 11, outputs the data, and updates RQ to “1” when the output of one frame is completed. If RQ = “1”, the data reading unit 41 reads data from the second frame memory (FM1) 12, outputs the data, and updates RQ to “0” when the output of one frame is completed.

When TYPE = "B", data is always read from the third frame memory (FM2) 13,
Output. The value RQ of the memory selection register 42 is not updated.

FIG. 10 shows the general operation of the output section 16. In FIG. 10, one column corresponds to a half frame period. Further, it is assumed that the decoding of the picture I0 is started when the bottom field output of the previous picture is started.

As described above, for writing, for the I picture and the P picture, the first frame memory (FM0) 11 and the second frame memory (FM1) 12 are used.
And are used alternately. As for the read, the first
The frame memory (FM0) 11 and the second frame memory (FM1) 12 are alternately used although their timings are different. Then, as shown in FIG. 10, the first frame memory (FM0) 11 and the second frame memory (F0
M1) 12, the third frame memory (FM2) 13, and the third frame memory (FM2) 13 are output in this order, that is, I0, P1, B2, B3. Here, it is natural that B2 and B3 are output before P4 is output, considering that P4 is used for backward reference when decoding B2 and B3.
It can be seen that the encoded images can be output in the correct order.

In FIG. 10, image output is normally performed while the write frame memory and the read frame memory are different. In the bottom field of the picture I0, that is, in the output period of the bottom 0, writing and reading of the first frame memory (FM0) 11 are performed at the same time. However, as described later, the first frame memory (FM0) 11 The top field data of the picture P4 is not written to the first frame memory (FM0) 11 before the bottom field data of the picture I0 is read. In the top field of the picture B2, that is, the output period of the top 2, the bottom field of the picture B2, that is, the output period of the bottom 2, and the top field of the picture B3, that is, the output period of the top 3, the writing and reading of the third frame memory (FM2) 13 are performed. And are being performed at the same time, but there will be no problem as described later.

Next, a description will be given of which slot in the selected frame memory is to be used for reading.
When starting the output of 16 lines, the data reading unit 41 determines two read slot numbers, one for the first half of 8 lines and the other for the latter half of 8 lines. For this determination, the row counter 43 is incremented by 1 every 16 lines.
Are using. The value of the row counter 43, that is, the read row number RROW, indicates which part of one frame the image is currently being output. R when outputting the first 16 lines of the top field
ROW = “0”, and is incremented by 1 every time 16 lines are output.

First, the case where TYPE = "I" or "P" is described. The first 8 lines use slots with the number RROW × 2, and the next 8 lines use slots with the number RROW × 2 + 1. As described at the time of data writing, regarding the I picture and the P picture, the top field and the bottom field are separately stored regardless of the frame structure and the field structure. Therefore, if the reading is performed in the above-described procedure, the top field and the bottom field are correctly output.

When TYPE = "B", the read slot numbers RS1 and RS2 given from the second slot number designating unit 54 are used.

As described above, the I-picture and the P-picture can be read from the slots in a fixed order, and the B-picture can be read from the slot designated by the second slot number designating unit 54. The operation of the data reading unit 41 is very simple.

Next, write slot numbers WS1 and WS
2 and read slot numbers RS1 and RS2 will be described. The operation of each of the first and second slot number designating sections 52 and 54 is roughly:
It only reads the slot number from the slot management memory (SM) 14 and outputs it. The memory control unit 55 only extracts the slot number from the slot management memory (SM) 14 and stores the data at another address. The address calculation when storing in another address is very simple, and this storage causes the second slot number designating unit 54 to read the slot numbers R in the correct order.
It becomes possible to specify S1 and RS2.

First N of slot management memory (SM) 14
The contents of the +4 word are initialized to slot numbers 0 to N + 3. The write pointer (M1) 51 has an initial value 0, and the read pointer (M2) 53 has an initial value X. Here, X is the third frame memory (FM
2) The number of slots is 13, that is, N + 4.

The first slot number designating section 52 reads two slot numbers from the slot management memory (SM) 14 and supplies them as write slot numbers WS1 and WS2. The read addresses supplied to the slot management memory (SM) 14 are M1 and M1 + 1.
The second slot number designation unit 54 reads two slot numbers from the slot management memory (SM) 14 and supplies them as read slot numbers RS1 and RS2. The read addresses supplied to the slot management memory (SM) 14 are M2 and M2 + 1. The first and second slot number designation units 52 and 54 set the B picture to 1
It operates every 6 lines of decoding, and does not operate at all when decoding I or P pictures.

The memory control unit 55 has the data writing unit 3
When the 5 performs writing, the write pointer (M1) is used in the same manner as the first slot number designating unit 52 reads.
Two slot numbers WS1 and WS2 are read out from the slot management memory (SM) 14 by using 51. At this time, the read addresses supplied to the slot management memory (SM) 14 are M1 and M1 + 1. Then, the memory controller 55 writes the write row number WROW and the mode signal MO.
The DE is used to determine which of the two slots currently being written should be output from the beginning of the screen, and the two slot numbers WS1, WS
2 are stored in different positions in the slot management memory (SM) 14 respectively. Specifically, if MODE = “FR (frame structure)”, the value of WS1 is at the address M1 + X-WROW of the slot management memory (SM) 14, and the value of WS2 is at the address of the slot management memory (SM) 14. The data is stored in the positions of M1 + X-WROW + N. If MODE = “FD (field structure)”, W
The value of S1 is stored in the position of address M1 + X of the slot management memory (SM) 14, and the value of WS2 is stored in the position of address M1 + X + 1 of the slot management memory (SM) 14, respectively.

11 and 12 show the process of updating the slot management memory (SM) 14. Here, for simplicity, N = 6. The number of slots per frame is 12. The slot management memory (SM) 14 is 18
It has a capacity to store each slot number in the positions of addresses 0 to 17. The picture B2 has a frame structure and the picture B3 has a field structure. Period 0 is an initial state, and the first 10 words of the slot management memory (SM) 14 are initialized to 0-9.

The period 1 is a decoding start point of the first 16 lines of the picture B2. At this time, writing is
Of the 10 slots of the third frame memory (FM2) 13, 2 slots corresponding to the address indicated by the write pointer (M1) 51, that is, slots 0 and 1 are used. The memory control unit 55 also stores the used slot numbers 0 and 1 at the positions of the different addresses 10 and 16 indicated by the arrows. Thereafter, the decoding proceeds in the periods 2, 3, and 4. Image output of the B2 top field is started in period 5, and reading is performed from the two slots, that is, slots 0 and 2, corresponding to the address indicated by the read pointer (M2) 53 at this point.
By the way, the slot number of the address pointed to by the read pointer (M2) 53 is indefinite in the period 0, but the slot number is already stored in the address pointed by the read pointer (M2) 53 by the memory control unit 55 in the period 5. There is. B2 top field data is stored in both slots 0 and 2, with slot 0 being the first 8 lines and slot 2 being the next 8 lines. That is, in the period 5, the data for 16 lines of the B2 top field can be correctly read. As described above, by the operation of the memory control unit 55 at the time of writing, the slot management memory (S
M) 14 has been rewritten.

Next, pay attention to the period 6. When the writing up to the period 5 is sequentially followed, it means that writing has been performed to the slots 0 to 9, that is, all the slots of the third frame memory (FM2) 13. Therefore, period 6
Therefore, the slot used once will be reused. The address indicated by the write pointer (M1) 51 in the period 6 is the read pointer (M2) 5 in the period 5 immediately before that.
3 is the address pointed to. In this way, the address pointed to by the write pointer (M1) 51 is the address pointed to by the read pointer (M2) 53 immediately before that, that is, the slot for which reading has already been completed, so that it can always be used. Therefore, FIG.
Both reading and writing in the third frame memory (F
There is no problem even if there is a period in which M2) 13 is used (output period of top 2, bottom 2 and top 3).

Here, for example, focusing on slot 0,
The data in slot 0 is read in period 5 and reused in period 6 immediately thereafter. Thus, there is no period when slot 0 is unused. The other slots are almost always used except in the initial state, which proves to be a very efficient memory usage method. In fact, according to the present invention, the interlace conversion of B picture can be realized by the frame memory (FM2) 13 of only half frame + 4 slots, and the effect of reducing the memory capacity is great.

13 to 16 are diagrams showing the detailed operation of the MPEG decoder 10 of FIG. In FIGS. 13 to 16, the vertical blanking interval VB is used so that the start of image decoding is not affected. Note that FIG. 15 and FIG.
Then, the corresponding period numbers in FIGS. 11 and 12 are shown in parentheses.

As described above, the VLD 31 temporarily stops its operation before decoding the image data of 16 lines and enters a state of waiting for the reception of the interrupt signal INT. The operation stop of the VLD 31 is necessary to prevent the data writing unit 35 from writing the data before the output unit 16 needs the data. It also has the meaning of preventing the output unit 16 from erroneously reading data that has not been decoded yet. VL
The timing for restarting D31 is the type signal TYPE
, Write row number WROW, and interrupt signal INT
Depends on and.

First, the case where TYPE = "I" or "P" will be described. When attempting to decode the first 16 lines of an image frame, write row number W
ROW is 0. When WROW = 0, the VLD 31 is restarted at the moment when the output of the next 16 lines is started after waiting for the completion of the output of the 16 lines of the bottom field. In other cases, write row number WROW
Is other than 0. When WROW ≠ 0, the VLD 31 is restarted at the moment when the output unit 16 starts outputting 16 lines or at the moment when the vertical blanking interval VB starts.

When TYPE = "B", at the start of the vertical blanking interval VB, at the start of display of 16 lines of the top field (excluding the last 16 lines), or at the start of 16 lines of the bottom field (final 16 lines). The VLD 31 is restarted at the start of display (except for the line). Also, at the same timing as the decoding of the B picture is restarted, the first
The slot number designating section 52 and the memory control section 55 are operated.

As shown in FIGS. 13 and 14, when the picture I0 and the picture P1 are decoded and written, writing is performed in an area that is not used at all. Decoding of the next picture P4 starts in the period 18 after the completion of output of 16 lines of the bottom field of the picture I0 by the above control. In the period 18, the first frame memory (FM0) 1
Although writing to 1 and reading are performed simultaneously, there is no problem because the write slots (slots 0 and 6) and the read slots (slots 8 and 9) do not match. The data of slot 6 is the period 1 immediately before it.
It has been read in 7. Considering that the bottom field data of the picture I0 cannot be read correctly if the decoding start timing of the picture P4 is in the period 17, the effect of delaying the decoding start timing of the picture P4 can be seen. Similarly, in the period 19, there is no competition between the writing of the picture P4 and the reading of the picture I0 in the first frame memory (FM0) 11.

Next, moving to FIGS. 15 and 16, the decoding of pictures B2 and B3 starts. In this case, the decoding start timing of the picture B2 and the picture B3 coincides with the start of the vertical blanking interval VB. This timing is different from the case of I picture and P picture. Then, the control using the slot management memory (SM) 14 described above is performed. As a result, FIG.
And writing and reading are performed in the slots as shown in FIG. Slot 0 read in period 31
Is written in the period 30 immediately before that. In the case of a B picture, data is output without being overwritten because data is basically written in that slot after one or two periods of the period read from a certain slot.
The number of slots used for B pictures is
Ten slots, less than 12 for one frame, are sufficient.

In the above example, one frame has 12 slots and 10 slots are used for B pictures. Generally, if 1 frame is 2N slots, by using N + 4 slots for B pictures, exactly the same is done. It is controllable. Of course, it is also possible to use N + 5 or more slots for B pictures. Also, here, one slot corresponds to eight lines of the image, but management in a larger unit is also possible.

As described above, the MPEG decoder 10 of FIG. 1 uses a memory for about 2.5 frames, and the operation of each circuit block is very simple.

[0070]

As described above, according to the present invention, the data memory is divided into a plurality of slots, and the reading / writing of the data memory is controlled by using the slot number stored in the slot management memory. Therefore, the interlace conversion of the B picture can be realized with the memory for about half a frame, and the capacity of the frame memory of the MPEG decoder can be reduced.

[Brief description of drawings]

FIG. 1 is a block diagram showing a specific example of an MPEG decoder according to the present invention.

FIG. 2 is a conceptual diagram showing an internal configuration of a first frame memory in FIG.

FIG. 3 is a conceptual diagram showing an internal configuration of a second frame memory in FIG.

FIG. 4 is a conceptual diagram showing an internal configuration of a third frame memory in FIG.

5 is a conceptual diagram showing an internal configuration of a slot management memory in FIG.

FIG. 6 is a block diagram showing an internal configuration of a decoding section in FIG.

7 is a block diagram showing an internal configuration of an output unit in FIG.

8 is a block diagram showing an internal configuration of a control unit in FIG.

9 is a diagram showing a schematic operation of a decoding unit in FIG.

FIG. 10 is a diagram showing a schematic operation of an output unit in FIG.

11 is a diagram showing an updating process of the slot management memory in FIG. 1. FIG.

12 is a view following FIG. 11. FIG.

13 is a diagram showing a detailed operation of the MPEG decoder of FIG.

FIG. 14 is a diagram subsequent to FIG. 13;

FIG. 15 is a diagram subsequent to FIG. 14;

FIG. 16 is a diagram following FIG. 15;

[Explanation of symbols]

 10 MPEG Decoder 11 First Frame Memory (FM0) 12 Second Frame Memory (FM1) 13 Third Frame Memory (FM2) 14 Slot Management Memory (SM) 15 Decoding Unit 16 Output Unit 21 Address Bus 22 Data Bus 30 Data processing unit 31 Variable length decoder (VLD) 32 Inverse quantizer (IQ) 33 Inverse discrete cosine transformer (IDCT) 34 Motion compensator (MC) 35 Data writing unit 36 Memory selection register 37 Row counter 41 data Read unit 42 Memory selection register 43 Row counter 44 Border color register 51 Write pointer (M1) 52 First slot number designation unit 53 Read pointer (M2) 54 Second slot number designation unit 55 Memory control unit 61 Address bus 62 Data bar IN Input picture INT Interrupt signal MODE mode signal OUT Output picture RROW Read row number RS1, RS2 Read slot number TYPE type signal WROW Write row number WS1, WS2 Write slot number

 ─────────────────────────────────────────────────── ─── Continued Front Page (72) Inventor William Brent Wilson Singapore Country 1441 Jalan Singa 17

Claims (1)

[Claims] 1. A data memory having N + 4 slots, a slot management memory for storing 2N + 6 slot numbers including the first N + 4 slot numbers initialized from 0 to N + 3, and an initial value. A write pointer having 0, a read pointer having an initial value N + 4, a data writing unit for writing data to designated two slots of N + 4 slots of the data memory, and A data reading unit for reading data from each of two designated slots out of N + 4 slots, a value of the write pointer, and a value obtained by adding 1 to the value of the write pointer as the addresses. Two slot numbers are read from the slot management memory, and the two read slot numbers are read. A first slot number designating unit for supplying a slot number to the data writing unit, and predicting a reading order of the slots of the data memory to calculate the two slot numbers supplied to the data writing unit. A memory control unit for respectively storing at different positions in the slot management memory and increasing the write pointer by 2, a value of the read pointer, and a value obtained by adding 1 to the value of the read pointer. A second slot number designation for reading two slot numbers from the slot management memory as addresses, supplying the read two slot numbers to the data reading unit, and incrementing the read pointer by 2. And an image processing device.