KR19980018884A - An image processor (Image Processor) - Google Patents

Abstract

There are provided five fields each consisting of N slots and a frame memory including three additional slots. Each slot has a capacity to store eight lines of an image. Any four of the five fields store reference frames for motion compensation. The remaining one field and three additional slots are provided for the interlace conversion of the B picture. The control unit includes a slot management memory, a pointer for a write slot, and a pointer for a read slot, and when writing to the frame memory by the bit stream analyzing unit such that the image output unit reads the frame memory in the correct slot order, Is updated.

Description

Image processing apparatus

The present invention relates to an image processing apparatus suitably used for decode processing of image information.

As an international standard for compressing and expanding moving picture data, an international standard called Moving Picture Coding Experts Group (MPEG) is generally known by taking the name of a working group of ISO / IEC. An MPEG decode for reproducing moving picture data includes a variable length decoder (VLD), an inverse quantizer (IQ), an inverse discrete cosine transformer (IDCT), a motion compensator : MC) as a main component. The MPEG decoder also requires a plurality of frames of memory for motion compensation or interlaced conversion.

MPEG is a feature that two frames of a general picture and a second-half picture are temporally used for reference of motion compensation. On the other hand, an I (Intra-coded) picture, a P (Predictive-coded) picture and a B (Bidirectionally predictive-coded) picture are introduced because of the problem of error propagation or special reproduction if motion compensation is used for all pictures . An I picture, that is, a coded I picture does not refer to a completely different picture. A P picture, i.e., a coded P picture, temporally performs motion compensation from the previous frame. The B picture, that is, the coded B picture, temporally performs bidirectional motion compensation from the first half frame and the second half frame. The B picture is not used as a reference frame at the time of decoding another frame.

The prediction state of each coding type will be described. It is assumed that the bit stream of the input picture is given to the MPEG decoder at the I0, P3, B1, and B2 sides. P3 is motion compensated from I0, B1 is motion compensated from I0 and P3, and B2 is motion compensated from IO and P3. The indication consists of I0, B1, B2, P3 in order. As described above, in the MPEG decoder, since the order of decoding and the order of display do not coincide, it is necessary to change the order of the MPEG decoders. Since the data of I0 and P3 image 2 frames are required for decoding of B1 and B2, a frame memory for two frames of image is required for reference of motion compensation. Therefore, the MPEG decoder requires two frames for reference of motion compensation.

Next, the order of decoding in pixel units for MPEG and the order in pixel units for image output will be described. In the case of a television or the like, only an even number line is output first, and then a single line is jumped as if only an odd number line is output, so that pixels are output in order from the upper left to the lower right. Only the even-numbered line is called the best field and only the odd-numbered line is called the lowest field. The interlaced output is first to output the top field in the order from left to right and then to the bottom field in order from top left to bottom right.

The image data is two-dimensional, and the data at spatially close positions are thought to be related to each other. However, in the case of an interlaced output, for example, when one line of the best field is considered, the one line image belongs to the lowest field. That is, the pixels on one line are very close in space, but are spaced apart in time. Therefore, in the case where the movement is severe, there may be a case where the correlation on the two lines closer in time than the one line is high. Assuming such a case, in MPEG, there are two types of order, namely, a frame structure and a field structure, roughly divided into the order of decoding in pixel units.

In MPEG, 16 x 16 pixels are decoded as a basic unit called one macroblock. The macroblocks are decoded in order from left to right, but here, for example, the first right pixel of one line at the top of the picture is included in the first right macroblock of the picture. On the other hand, when the decoding of the first right macroblock ends, the data of the 16 lines are decoded as a result. Therefore, in MPEG, data of about 16 lines are almost simultaneously decoded.

In the case of the frame structure, the data of one frame of the image constitutes a macroblock of 16 pixels in the vertical direction and 16 pixels in the horizontal direction, and is decoded for each macroblock. Thus, the best field and the lowest field are decoded almost simultaneously. Therefore, the order of image output is not completely coincident with the order of image output.

In the case of the field structure, an image frame is divided into a best field and a minimum field, and macroblocks of 16 pixels in the vertical direction and 16 pixels in the horizontal direction are formed in each field, and they are decoded for each macroblock. In this case, one macroblock contains only the best field or the lowest field, and the data of the lowest field is decoded after all the data of the best field is decoded. In this case, although the order is substantially the same as that of the image output, the order of decoding is not completely coincident with the order of image output because it is performed in units of macroblocks.

The image output is performed in the order of the highest field and the lowest field. When attention is paid to the point in time at which the output of the last 8 lines of the best field is started, the decoding of the last 16 macroblocks of the image frame must be completed before the output of the last 8 lines is started. This is because the rightmost 16 pixels of the last 8 lines start decoding the last macroblock of the pixel to determine the value. Therefore, at this time, both the best field and the lowest field must be decoded. On the other hand, after this point in time, both the best field 8 line and the lowest field should be output in order, but the data has already been decoded. Therefore, if both the best field 8 line and the data of the lowest field are stored in the frame memory, the data is destroyed before the data is output, and image output is not performed. As a result, a frame memory having a capacity of about half a frame for storing all the data of the minimum field and the data amount of eight lines of the best field is required.

To summarize the above, a memory for two frames is required to perform motion compensation, and a memory for about half a frame is required to perform the interlace conversion of the B picture. As a result, at least a total of about 2.5 frames of memory is required.

"Development of MPEG 2 Decoder LSI - Effective Memory Allocation", Lecture Meeting of the Institute of Electronics, Information and Communication Engineers, 1994, C-659, March 1994, An example of an MPEG decoder using a memory of 1/2 < th > In addition, Akihiko Takahata et al., &Quot; DRAM Interface in MPEG 2 Video Decoder LSI ", C-586, Lecture Papers of the Conference of the Institute of Electronics, Information and Communication Engineers, 1995, Lt; RTI ID = 0.0 > of MPEG < / RTI >

Since both of the conventional MPEG decoders require 1.0 to 1.5 frames of memory for interface conversion of B pictures, there is a problem that the price of the MPEG decoder is expensive. There is room for improvement in consideration of the fact that a memory of about 1/2 frame is sufficient for interlace conversion as described above.

An object of the present invention is to reduce the capacity of a frame memory of an MPEG decoder and efficiently utilize the frame memory.

1 is a block diagram showing a specific example of an MPEG decoder according to the present invention;

2 is a conceptual diagram showing the internal structure of the frame memory shown in Fig. 1;

3 is a conceptual diagram showing an internal configuration of one memory block in Fig.

4 is a conceptual diagram showing an internal configuration of an additional memory block in Fig.

5 is a block diagram showing an internal configuration of a bit stream analyzing unit shown in FIG. 1;

6 is a block diagram showing the internal configuration of the image output unit in Fig.

7 is a diagram showing a schematic operation of the image output unit in Fig.

8 is a block diagram showing an internal configuration of the control unit in Fig.

9 is a diagram showing a schematic operation example of the MPEG decoder of FIG.

FIG. 10 is a diagram showing a detailed operation example of the MPEG decoder of FIG. 1; FIG.

Fig. 11 is a view subsequent to Fig. 10; Fig.

Fig. 12 is a view subsequent to Fig. 11; Fig.

Fig. 13 is a view subsequent to Fig. 12; Fig.

Fig. 14 is a view subsequent to Fig. 13; Fig.

Fig. 15 is a view subsequent to Fig. 14; Fig.

16 is a diagram showing an example of a process of updating the slot management memory in Fig. 8;

Fig. 17 is a view subsequent to Fig. 16; Fig.

Fig. 18 is a view corresponding to Fig. 16 when the additional block in Fig. 2 is composed of two additional slots. Fig.

Fig. 19 is a view corresponding to Fig. 16 when the additional block in Fig. 2 is composed of one additional slot; Fig.

Description of the Related Art [0002]

10: MPEG decoder 11: Frame memory

12: bitstream analyzing unit 13: image output unit

14: control unit 15: data bus

16: address bus 17: register bus

31: header analyzing unit 32: variable length decoder (VLD)

33: Inverse quantizer (IQ) 34: Inverse discrete cosine transformer (IDCT)

35: Motion compensator (MC) 36: Internal register

52: Work memory RSLP: Pointer for read slot

SCM: Slot management memory WSLP: Pointer for write slot

In order to achieve the above object, the present invention has focused on the following points. In other words, it has been pointed out that the B picture is not already used when the picture output ends and that the area of the current macroblock being outputted is predicted by the analysis of the part (header) of the additional information of the input picture will be.

Specifically, according to the present invention, in an image processing apparatus including a frame memory, each of the frame memories includes five blocks each having a capacity of 1/2 frame of image, and 1 block having a capacity smaller than that of each of the five blocks Four blocks selected from the five blocks are used for storing I and / or P pictures for reference of motion compensation, and the remaining one block and the additional block are used for storing the B picture The data memory for the B picture is reconstructed into one block in the four blocks and the additional block. As a result, it is possible to realize not only the interlace conversion of the B picture but also the motion compensation of all types of pictures as a memory of about 2.5 frames. In addition, since the data memory is not used for the B picture when three or more I and / or P pictures are consecutive, one block constituting a part of the data memory is used for storing the I and / or P pictures There is an advantage to be able to.

Each of the five blocks and the additional blocks are each divided into a plurality of slots each having a constant capacity. The image processing apparatus according to the present invention further includes a slot management memory for storing the slot number and a controller for controlling the reading of the data memory using the slot number stored in the slot management memory The controller writes the slot number used at the time of writing the data memory into the slot management memory for reading the data memory. This allows the contents of the slot management memory to be updated at the time of writing of the data memory such that the data memory is read in the correct slot order. Therefore, by writing the slot immediately after reading any slot in the data memory, the interlace conversion of the B picture can be realized in the memory of about 1/2 frame.

The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

Hereinafter, specific examples of an MPEG decoder for real time reproduction processing of moving image data will be described with reference to the drawings.

Fig. 1 shows a configuration example of an MPEG decoder according to the present invention. The MPEG decoder 10 of FIG. 1 includes a frame memory 11 for storing image data of about 2.5 frames, a bit stream analyzing section 12 for analyzing an input bit stream IN, And a control unit 14 for supplying a start signal INIT to the bit stream analyzing unit 12 in response to the interrupt signal INTR Consists of. In Fig. 1, 15 is a data bus, 16 is an address bus, and 17 is a register bus. The bitstream analyzing unit 12 has a function of decoding the macroblock in addition to the function of interpreting the header portion in the input bitstream IN and writing the result into the frame memory 11. [ These two functions are activated by the start signal INIT from the control unit 14. [ The frame memory 11 is used for storage of an image for motion compensation and for interlace conversion. The image output unit 13 reads the reconstructed image data from the frame memory 11 and supplies the image output signal OUT in the interlaced order. The decoded moving image can be seen by connecting the image output signal OUT to the display. The image output section 13 transmits the interrupt signal INTR to the control section 14 in accordance with the timing of the output. The control unit 14 can read the internal registers of the bit stream analyzing unit 12 and the image output unit 13 via the register bus 17 or set values in the respective internal registers.

Fig. 2 shows a schematic configuration of the frame memory 11, and Fig. 3 and Fig. 4 show the detailed configuration thereof. 2, the frame memory 11 is composed of five blocks 20, 21, 22, 23, and 24 to which field numbers 0 to 4 are assigned, and one additional block 25 have. One block 20 is composed of N slots assigned slot numbers 0 to N-1 as shown in FIG. Here, N is an amount depending on the picture size for which the MPEG decoder 10 decodes, and N is 30 for example in an image of NTSC (National Television System Committee). Each slot has a storage capacity for eight lines of an image. As a result, one block 20 can store 240 lines of data, i.e., NTSC image 1 field (1/2 frame) data. The other four blocks 21, 22, 23, and 24 are also each composed of N slots. The additional block 25 is composed of three additional slots assigned slot numbers from N to N + 2 as shown in FIG.

The frame memory 11 is managed in units of slots. One slot (8 lines) serving as the management unit is 1/2 of the vertical size (16 lines) of the macroblock and is the most suitable unit for coping with both the frame structure and the field structure. Specifically, one slot is designated by a field number and a slot number. For example, the first slot in block 20 is designated field number 0, slot number 0. The first slot of the additional block 25 is designated as field number 0 and slot number N. [ The same slot of the additional block 25 may be designated as the field number 1 and the slot number N. [

Four of the five blocks 20, 21, 22, 23, and 24 are used for storing an image for reference of motion compensation, and the remaining one block and the additional block 25 are used for interlace conversion of the B picture do. In the following description, the number of slots used for the B picture, that is, N + 3 is denoted by Sn. For example, when the best field of the I picture is in a slot from 0 to N-1 of the field number 0, and the lowest field of the I picture is assigned to a slot from the number 0 to N-1 of the field number 1, The best field is stored in slots 0 to N-1 of field number 2, and the lowest field of the P picture is stored in slots of number 0 to N-1 of field number 3, respectively. In this case, the first 8 lines of the best field are stored in the slot 0 of the field where the best field is stored, and the next 8 lines are stored in this order in the slot 1. The B picture is stored, for example, in a slot from No. 0 to Sn-1 in field number 4. At this time, three additional slots assigned slot numbers from N to N + 2 are used. However, the slot in which the best field is stored varies depending on the structure of the image, which is a frame structure or a field structure. It is assumed that default image data is already written in each field of the frame memory 11 in the initial state. For example, data corresponding to black is written. A default image is output until the decoded result image is output.

Fig. 5 shows the internal structure of the bit stream analyzing unit 12 in Fig. The bitstream analyzing unit 12 includes a header analyzing unit 31, a VLD (Variable Length Decoder) 32, an IQ (Inverse Quantizer) 33, an IDCT (Inverse Discrete Cosine Transform) (Motion compensator) 35, and an internal register 36. The internal register 36 has ten registers,

Coding type register CTYR

Structure Register STRR

Best Forward Reference Field Register T_FRFR

Lowest forward reference field register B_FRFR

Best Forward Reference Field Register T_BRFR

Lowest Used Rear Reference Field Register B_BRFR

Best write field register T_WFDR

Lowest write field register B_WFDR

The first write slot register WSR1

The second write slot register WSR2

. The coding type register CTYR is a register for storing the coding type of the input image. The structure register STRR is a register indicating whether the structure of the input image is a frame structure or a field structure. The four reference field registers T_FRFR, B_FRFR, T_BRFR, and B_BRFR are registers for designating a field number indicating a storage position of a motion compensation reference image in the frame memory 11. The two write field registers T_WFDR and B_WFDR are registers for specifying the number of the write field in the frame memory 11. [ The first and second write slot registers WSR1 and WSR2 are registers for specifying the write slot number in the frame memory 11. [

The header analyzing unit 31 or the VLD 32 is activated by the start signal INIT from the control unit 14. [ The header analyzing unit 31 decodes the picture header information in the input bit stream IN when started and stores the coding type of the picture in the coding type register CTYR in the internal register 36 and the coding type of the picture in the structure register STRR Respectively. When the VLD 32 is activated, the IQ 33, the IDCT 34 and the MC 35 are sequentially activated, and the decoded result of the image data for 16 lines is output to the data bus 15, To the frame memory 11 via the bus. The 16 lines corresponds to 45 macroblocks in the case of 720 pixels in the horizontal direction. Among them, the MC 35 performs motion compensation to create final image data, and also writes the reconstructed data into the frame memory 11. [ The VLD 32 sends a motion vector MV to the MC 35 for motion compensation. The slot of the frame memory 11 is specified by the address on the address bus 16. [

The MC 35 only has a part related to memory management among the bit stream analyzing unit 12. The MC 35 has to inform where in the frame memory 11 the predictive image is stored for motion compensation. Therefore, the best forward reference field register T_FRFR, the lowest forward forward reference field register B_FRFR, the best rearward reference field register T_BRFR, and the lowest forward reference field register B_BRFR are used. Here, for example, the best field portion of the data for the forward reference is stored in the slot N-1 from the field slot 0 of the number designated by the best forward reference field register (T_FRFR).

In order to determine the slot into which the reconstructed image data is to be written, the most significant write field register (T-WFDR), the lowest write field register (B-WFDR), the first write slot register WSR1, (WSR2) and a structure register (STRR). More specifically, the writing position is determined as follows. In the case of the frame structure, the top eight writing lines are written in the slots designated by the field number of the best write field register T_WFDR and the slot number of the first write slot register WSR1. In the case of the field structure, for the best field (first half frame), the upper 8 lines are allocated to the slot designated by the field number of the best write field register (T_WFDR) and the slot number of the first write slot register (WSR1) The lower eight lines are written into the slot designated by the field number of the write field register T_WFDR and the slot number of the second write slot register WSR2. For the lowest field (next half frame), the upper eight lines are written in the slot designated by the field number of the lowest use write field register B_WFDR and the row number of the first write slot register WSR1, The lower eight lines are written into the slot designated by the field number of the field register B_WFDR and the slot number of the second write slot register WSR2. As described above, the best field and the lowest field are not mixed in one slot. The register related to the memory management in the internal register 36 is set by the control unit 14 via the register bus 17. [

6 is an internal configuration diagram of the image output unit 13 in Fig. The image output section 13 includes a timing generating section 41, a reading section 42, and an internal register 43. The internal register 43 has four registers,

Read Field Register RFDR

Read slot register RSLR

Output low register ORWR

Output parity register OPYR

. The read field register (RFDR) is a register for specifying the number of the read field in the frame memory (11). The read slot register RSLR is a register for specifying the number of the read slot in the frame memory 11. [ The output low register (ORWR) is a register that indicates which one of the fields in the current output line. The output parity register (OPYR) is a register that indicates which of the current top field and the bottom field is being output.

7 is a schematic operation diagram of the image output section 13. Fig. One cycle of output consists of a vertical retrace interval (VB), a best field output period, a vertical retrace interval and a minimum field output period. For example, in the case of NTSC, the output is 30 cycles per second. The timing generating section 41 generates a timing signal TMNG for the image output section 13 to operate at a constant cycle and also transmits an interrupt signal INTR to the control section 14 every eight lines of the screen output. The interrupt signal INTR occurs once at the start of the output of each field as shown in Fig. 7A, and then every 8 lines after the output of the interrupt signal INTR. The reading section 42 reads the designated slot in the scanning order in synchronization with the timing signal TMNG generated by the timing generating section 41 and supplies the image output signal OUT. The read slot is designated as a combination of a read field register (RFDR) and a read slot register (RSLR) in the internal register (43). The register for setting the read slot sets the control unit 14 via the register bus 17 when an interrupt occurs. The timing generator 41 changes the output low register ORWR and the output parity register OPYR in the internal register 43 as follows. That is, the value of the output parity register OPYR is set to 0 immediately before the interruption of the output start of each field as shown in FIG. 7B, and becomes 0 if it is in the state of outputting the next best field. If it is in the output state, it becomes 1. The value of the output low register (ORWR) becomes 0 immediately before the interruption of the output start as shown in FIG. 7 (c), and increases by one immediately before the interruption of the end of the 8 lines.

8 is a block diagram showing an internal configuration of the control unit 14 in Fig. The control unit 14 includes a controller 51, a work memory 52, and a program memory 53. The work memory 52 includes a slot management memory (SCM)

Best forward reference field work T_FRFW

Lowest forward reference field work B_FRFW

Best Practice Rear Reference Field Work T_BRFW

Lowest Rear Reference Field Work B_BRFW

Decode row number work DRNW

Decoded work DTYW

Output type work OTYW

Decode Structure Work DSTW

Best output field work T_OFDW

Lowest output field work B_OFDW

Pointer WSLP for write slot

Pointer RSLP for read slot

. The four reference field works T_FRFW, B_FRFW, T_BRFW and B_BRFW correspond respectively to the four reference field registers T_FRFR, B_FRFR, T_BRFR and B_BRFR in the bitstream analyzing section 12, And designates a field number indicating a storage position of an image for reference of motion compensation. The decode row number work (DRNW) is a work for decreasing a row number indicating which portion of one frame is decoded. For example, during the decoding of the first 16 lines, 0 is set, while the next 16 lines are decoded 1, the row number is increased. The decoded work DTYW is a work that indicates the coding type of an image currently being decoded. The output type work OTYW is a work that indicates whether or not the coding type of the image currently being output is B, and is used for identification because the coding type B is different from the memory management method. The decoded structure work DSTW is a work which indicates whether the structure of the image currently being decoded is a frame structure or a field structure and affects the setting of the write slot and the access to the slot management memory SCM. The two output field works T_OFDW and B_OFDW are works that represent the field numbers of the output image. The write slot pointer WSLP and the read slot pointer RSLP are used for memory management in the case of the coding type B as described later and always hold the address indicating the word in the slot management memory SCM .

The initial values incrementing from 0 to Sn-1 are sequentially stored in each word from address 0 to Sn-1 of the slot management memory (SCM) in the work memory 52. The initial values of these values are values sequentially designating the number of write slots used when decoding the picture of the coding type B coming first. The ten works in the work memory 52,

T_FRFW = 0

B_FRFW = 1

T_BRFW = 2

B_BRFW = 3

DTYW = I

OTYW = IP

T_OFDW = 0

B_OFDW = 1

WSLP = 0

RSLP = Sn

.

The controller 51 sequentially executes the command INST written in the program memory 53. [ First, the controller 51 executes a routine. Thereby, the setting of the internal register 36 in the bitstream analyzing unit 12 is performed. When the interrupt signal INTR is transmitted from the image output unit 13, the controller 51 shifts to the interrupt processing routine. Thereby, the setting of the internal register 43 in the image output unit 13 is performed.

The control unit 14 controls the bit stream analyzing unit 12 by executing a normal routine. In detail, the following steps 1.1 to 1.14 are executed.

Step 1.1: The header analyzing unit 31 is activated. In response to this, the header analyzing unit 31 sets the values in the coding type register CTYR and the structure register STRR in the internal register 36, respectively.

Step 1.2: Write the value of the coding type register (CTYR) to the decode type work (DTYW).

Step 1.3: Write the value of the structure register (STRR) to the decode structure (DSTW).

Step 1.4: If the decoded work DTYW is I or P and the output type work OTYW is IP, the first and second field numbers for designating the write field are determined as follows. That is, first, the number of a field not included in any of the four reference field works T_FRFW, B_FRFW, T_BRFW, and B_BRFW is selected. For example, if these works have values of 0, 1, 2, and 3, select 4, and if they contain values of 2, 3, 4, and 0, select 1. The number of the selected field is set as the first field number. The value of the best forward reference field work T_FRFW is set as the second field number.

Step 1.5: If the decoded type work DTYW is I or P and the output type work OTYW is B, the value of the best use forward reference field work T_FRFW is set as the first field number, B_FRFW) is set as the second field number.

Step 1.6: If the decoded work (DTYW) is I or P, update the four reference field works (T_FRFW, B_FRFW, T_BRFW, B_BRFW). Specifically, the value of the best-use backward reference field work T_BRFW is set to the best-use forward reference field work T_FRFW, the value of the lowest forward rear reference field work B_BRFW is set to the lowest forward forward reference field work B_FRFW, The first field number is written to the best-practice backward reference field work T_BRFW, and the determined second field number is written to the lowest backward reference field work B_BRFW.

Step 1.7: The reference field for motion compensation is set in the internal register 36 of the bitstream analysis unit 12. [ More specifically, the value of the best forward reference field work T_FRFW is set to the highest priority forward reference field register T_FRFR, the value of the lowest forward forward reference field work B_FRFW is set to the lowest forward forward reference field register B_FRFR, The value of the reference field work T_BRFW is written to the best-suited backward reference field register T_BRFR and the value of the lowermost rear reference field work B_BRFW is written to the lowermost rearward reference field register B_BRFR, respectively.

Step 1.8: A field for writing is set in the internal register 36 of the bitstream analysis unit 12. [ Concretely, when the decoded work DTYW is I or P, the value of the best-use backward reference field register T_BRFR is set to the best use write field register T_WFDR and the value of the lowest use rearward reference field register B_BRFR is set to the lowest Write field register B_WFDR, respectively. When the decoded work DTYW is B, the number of fields not included in any one of the four reference field registers T_FRFR, B_FRFR, T_BRFR, and B_BRFR is selected and the number of the selected field is stored in the most- T_WFDR and the lowest write field register B_WFDR.

Step 1.9: Write 0 to the decode row number work (DRNW).

Step 1.10: Set the write slot. Details of the write slot setting procedure will be described later.

Step 1.11: The decoding of the macroblocks for 16 lines is performed by the bitstream analyzing unit 12.

Step 1.12: When decoding of one frame is finished, return to step 1.1.

Step 1.13: Increase decode row number work (DRNW) by one.

Step 1.14: The procedure returns to Step 1.10 to decode the next 16 macroblocks.

However, the control unit 14 controls the image output unit 13 by executing the interrupt processing routine. Specifically, first, it is determined whether or not the output image is a B picture, the read field number is determined, and then an operation for determining the slot number is performed. If the I picture or the P picture is being decoded, an image decoded earlier may be output, which is an image used for forward reference. The B picture is output one frame later than the start of the decoding. Therefore, if the B picture is being decoded when the next frame is output, the B picture can be output. More specifically, the next steps 2.1 to 2.6 are executed for each occurrence of an interrupt.

Step 2.1: If the output low register (ORWR) is 0 and the output parity register (OPYR) is 0 (output of the best field), the value of the decode type work (DTYW) is written to the output type work (OTYW). The output type work OTYW means whether the coding type of the image to be output next is B or B or not. More specifically, when the decoded work DTYW is I or P, IP is written to the output type work OTYW, and B is written to the output type work OTYW when the decoded type work DTYW is B .

Step 2.2: If the output low register (ORWR) is 0, the output parity register (OPYR) is 0 (output of the best field), and the output type work (OTYW) is IP, the value of the best forward reference field work (T_FRFW) And the value of the lowest forward forward reference field work B_FRFW to the lowest use output field work B_OFDW, respectively. At this time, the picture for forward reference is an I-picture or a P-picture decoded before. Therefore, during the decoding of the I picture or the P picture, the decoded picture is output before that.

Step 2.3: If the output low register (ORWR) is 0, the output parity register (OPYR) is 0 (output of the best field is output), and the output type work (OTYW) is B, then 4 reference field works (T_FRFW, B_FRFW, T_BRFW, and B_BRFW), and writes the number of the selected field to the best effort output field work T_OFDW and the lowest use output field work B_OFDW. The field in which the image of the coding type B in the frame memory 11 is written will not be present in any of the forward prediction output field and the backward prediction output field.

Step 2.4: The values of the best-effort output field work T_OFDW and the minimum field work B_OFDW that have been determined are set in the image output unit 13. More specifically, if the output parity register OPYR is 0 (outputs the next best field), the field number of the best output field work T_OFDW is set to the read field register RFDR of the image output unit 13. [ If the output parity register OPYR is 1 (the lowest field is output next), the field number of the lowest use output field work B_OFDW is set to the read field register RFDR of the image output unit 13. [

Step 2.5: Set the read slot. Details of the read slot setting procedure will be described later.

Step 2.6: Terminate interrupt processing.

9 is a schematic operation of the MPEG decoder 10 of FIG. 1, that is, a handling diagram of a write field and a read field. In FIG. 9, it is assumed that the input bit stream IN is given to the bit stream analyzing unit 12 in order of I0, P1, P4, B2, B3, and P5. In addition, the expressions I0 and P4 denote coding types with the first I, P, and B symbols, and show the order of display (output) by the following numbers.

At the decode period of the picture I0, that is, at the start time of the period 1, the value of the output type work OTYW is IP. Therefore, in period 1, the values of the four reference field works T_FRFW, B_FRFW, T_BRFW, and B_BRFW are updated to 2, 3, 4, and 0, respectively, based on the initial values of period 0 and in accordance with steps 1.4 and 1.6. The value of the best-use backward reference field work T_BRFW, that is, 4 is the number of the best-effort writing field, and the value of the lowest forward rearward reference field work B_BRFW, that is, 0 is the number of the lowest writing field. Therefore, the picture 10 is written to the fields 4 and 0 in the frame memory 11.

The values of the four reference field works T_FRFW, B_FRFW, T_BRFW, and B_BRFW are updated to 4, 0, 1, and 2, respectively, in the decode period of the picture P1, Then, the value of the best-use backward reference field work T_BRFW, that is, 1 is the number of the best-fit write field, and the value of the lowest use backward reference field work B_BRFW, that is, 2 is the number of the lowest write field. Therefore, the motion compensated picture P1 is written to the field 1 and the field 2 by referring forward to the picture I0 in the fields 4 and 0.

The values of the four reference field works T_FRFW, B_FRFW, T_BRFW, and B_BRFW are updated to 1, 2, 3, and 4, respectively, in the decode period of the picture P4, that is, The value of the best-use backward reference field work T_BRFW, that is, 3 is the number of the best-effort writing field, and the value of the lowest forward rearward reference field work B_BRFW, that is, 4 is the number of the lowest writing field. Therefore, the motion compensated picture P4 is written in the fields 3 and 4 by forward reference to the field 1 and the double picture P1.

The four reference field works T_FRFW, B_FRFW, T_BRFW, and B_BRFW are not updated in the decoding period of the picture B2, that is, in the period 4. The number of fields not included in any of the four reference field works, that is, 0 is the number of the write fields for the best use and the lowest use. Therefore, the picture B2 is motion-compensated in the field 0 and three additional slots in the frame memory 11. At this time, the picture P1 in the fields 1 and 2 is referred to in the forward direction, and the picture P4 in the fields 3 and 4 is referred to in the backward direction.

The four reference field works T_FRFW, B_FRFW, T_BRFW, and B_BRFW are not updated in the decoding period of the picture B3, that is, in the period 5. The number of fields not included in any of the four reference field works, that is, 0 is the number of the write fields for the best use and the lowest use. Therefore, the motion compensated picture B3 is written in the field 0 and three additional slots. At this time, the picture P1 in the fields 1 and 2 is referred to in the forward direction, and the picture P4 in the fields 3 and 4 is referred to in the backward direction.

The value of the output type work OTYW is B at the decode period of the picture P5, that is, at the start time of the period 6. Therefore, in period 6, the values of the four reference field works T_FRFW, B_FRFW, T_BRFW, and B_BRFW are updated to 3, 4, 1, and 2, respectively, in accordance with steps 1.5 and 1.6. The value of the best-use backward reference field work T_BRFW, that is, 1 is the number of the best-effort write field, and the value of the lowest forward reference field work B_BRFW, that is, 2 is the number of the lowest write field. Therefore, the motion compensated picture P5 is written in fields 1 and 2. At this time, the picture P4 in the fields 3 and 4 is referred to forward.

7 (b), the value of the output parity register OPYR in the image output unit 13 becomes 0 if it is in the state of outputting the next best field immediately before interruption of output of each field , And is 1 if the lowest field is output next. As a result, the value '1' of the output parity register OPYR changes from 0 to 0 at the output start time of the best field. Fig. 9 shows that the top output field work (T_OFDW) and the lowest use output field work (B_OFDW) are updated in synchronization with the change from 1 to 0 of the output parity register (OPYR).

More specifically, the values of the two output fieldworks (T_OFDW and B_OFDW) are set to 4 and 0, respectively, using the values of the two forward reference field works (T_FRFW, B_FRFW) . Therefore, the best field of the picture I0 stored in the field 4 of the frame memory 11 is output in the second half of the period 2 and the lowest field of the picture I0 stored in the field 0 in the first half of the period 3 respectively .

At the start of the second half of period 3, the values of the two output fieldworks T_OFDW and B_OFDW are updated to 1 and 2, respectively, in accordance with step 2.2. The minimum field of the picture P1 in which the best field of the picture P1 stored in the field 1 is stored in the second half of the period 3 and the picture P1 that is stored in the field 2 is outputted in the first half of the period 4 respectively.

(T_OFDW, B_OFDW), which is not included in any one of the four reference field works (T_FRFW, B_FRFW, T_BRFW, B_BRFW) according to step 2.3, . Therefore, the output of the picture B2 is started at a half frame later than the start of the decoding of the picture B2, the highest field of the picture B2 is in the second half of the period 4 and the lowest field of the picture B2 is in the period 5 Respectively.

(T_OFDW, B_OFDW), which is not included in any of the four reference field works (T_FRFW, B_FRFW, T_BRFW, B_BRFW) according to the procedure 2.3 at the start of the second half of period 5, . Therefore, the output of the picture B3 is started at a half frame later than the start of the decoding of the picture B3, the highest frame of the picture B3 is in the latter half of the period 5, the lowest field of the picture B3 is in the period 6 Respectively.

At the start of the second half of the period 6, the values of the two output fieldworks T_OFDW and B_OFDW are updated to 3 and 4, respectively, by using the values of the two forward reference field works T_FRFW and B_FRFW in accordance with the procedure 2.2. Therefore, the lowest field of the picture P4 stored in the field 4 and the highest field of the picture P4 stored in the field 3 are output respectively in the latter half of the period 6 and the first half of the next period.

As described above, according to Fig. 9, an output image is obtained in the order of I0, P1, B2, B3, and P4. 9, immediately after the best field of the picture I0 is read from the field 4 in the second half of the period 2, it is found that the minimum field of the picture P4 is written in the field 4 in the first half of the period 3 . As a result, efficient use of the field in the frame memory 11 is being achieved. In addition, writing and reading to the field 0 and three additional slots are simultaneously performed for the coding type B picture from the start time of the second half of the period 4 to the end time of the second half of the period 5, There is no problem.

Here, the details of the write slot setting procedure in step 1.10 will be described. The control unit 14 sets the first and second write slot registers WSR1 and WSR2 in the bitstream analyzing unit 12 and waits until a predetermined condition is satisfied.

First, the setting of the first and second write slot registers WSR1 and SWR2 will be described. If the decoded work DTYW is I or P and the decode structure work DSTW indicates the frame structure, the value of the decode row number work DRNW is written to the first and second write slot registers WSR1 and WSR2 . If the decoded work DTYW is I or P and the decode structure work DSTW indicates the field structure, (the value of the decode row number work DRNW) x2 is written to the first write slot register WSR1 (Value of the decode row number work DRNW) x 2 + 1 into the second write slot register WSR2, respectively. However, if DRNW × 2 ≥ N, DRNW × 2 is changed to DRNW × 2-N and DRNW × 2 + 1 is changed to DRNW × 2 + 1-N, respectively. When the decoded work DTYW is I or P, the frame structure or the field structure is not required. In this case, the best field is the field indicated by the best write field register (T-WFDR) (B-WFDR), respectively. Within each field, the first 8 lines are written to slot 0, and the next 8 lines are written to slot 1.

If the decoded work DTYW is B, the control unit 14 sets the first and second write slot registers WSR1 and WSR2 by executing the following procedures 3.1 to 3.7.

Step 3.1: The first slot number X is fetched from the work of the address indicated by the pointer WSLP for the write slot in the slot management memory (SCM).

Step 3.2: fetch the second slot number Y from the next address word in the slot management memory (SCM).

Step 3.3: Write the first slot number X into the first write slot register WSR1.

Step 3.4: Write the second slot number Y to the second write slot register WSR2.

Step 3.5: Write the first slot number (X) in the slot management memory (SCM) for the setting reference of the read slot register (RSLR). More specifically, when the decode structure work DSTW indicates a frame structure, the value of the write slot pointer WSLP + Sn- (the value of the decode row number work DRNW) in the slot management memory SCM And writes the first slot number (X) in the word of the indicated address. When the decode structure work DSTW indicates the field structure, the first slot number X is written into the word of the address indicated by (the value of the write slot pointer WSLP) + Sn in the slot management memory SCM .

Step 3.6: Similarly, to prepare for setting of the read slot register (RSLR), write the second slot number (Y) in the slot management memory (SCM). Specifically, when the decode structure work DSTW indicates the frame structure, the value of the write slot pointer WSLP + Sn- (the value of the decode row number work DRNW) in the slot management memory SCM + N And writes the second slot number (Y) in the word of the address indicated by the second slot number. When the decode structure work DSTW indicates the field structure, the second slot number Y is written into the word of the address indicated by (the value of the write slot pointer WSLP) + Sn + 1 in the slot management memory SCM .

Step 3.7: Increase the write slot pointer (WSLP) by 2.

The control unit 14 sets the first and second write slot registers WSR1 and WSR2 in the bitstream analyzing unit 12 and then selects one of the following conditions 1 to 4 Wait until. This is necessary to prevent writing to the slot before data is read from any slot.

Condition 1: If the decoded work DTYW is I or P, the value of the best use output field work T_OFDW is different from the value of the best use backward reference field work T_BRFW and the value of the lowest use output field work B_OFDW is Must be different from the value of the Lowest Rear Reference Fieldwork (B_BRFW). When this condition is satisfied, the output of the field to be written is already finished, so that writing into that field can be performed.

Condition 2: If the decoded work (DTYW) is I or P and the decode structure work (DSTW) indicates the frame structure and the value of the output row register (ORWR) is larger than the value of the decode row number work (DRNW) that. When this condition is satisfied, a slot having a number smaller than the value of the output low register (ORWR) is output, and writing can be performed.

Condition 3: The decoded work (DTYW) is I or P, the decode structure work (DSTW) indicates the field structure, and the decode period of the best field. When decoding of the coding type I or P is performed, at least one field is usable. Since the best field is written in the usable field, writing can be performed.

Condition 4: The decoded work (DTYW) is B and the value of the read slot pointer (RSLP) is greater than the value of the write slot pointer (WSLP). The slot number stored in the address indicated by the pointer (RSLP) for the read slot in the slot management memory (SCM) is the slot number during the current output as described later. Therefore, since the slot having the number smaller than the number is already read out, the writing can be performed.

Next, the details of the read slot setting procedure in step 2.5 will be described. If the output type work OTYW is an IP, the control unit 14 merely sets the value of the output row register ORWR of the same image output unit 13 to the read slot register RSLR of the image output unit 13 . At the time of image output of the coding type I or P, the number of reading slots in the reading field may be incremented by one in order from zero. However, the output row number in the output row register (ORWR) also increases by one from 0 in order. Therefore, the value of the output row register ORWR may be set in the read slot register RSLR.

If the output type work (OPTW) is B, the control unit 14 sets the read slot register (RSLR) by executing the following simple procedure from step 4.1 to step 4.3. This is the result of the preparatory process shown in the above-mentioned steps 3.5 and 3.6. In addition, the pointer RSLP for the read slot is incremented by one every eight line outputs.

Step 4.1: Retrieve the slot number (Z) from the word of the address indicated by the pointer (RSLP) for the read slot in the slot management memory (SCMN).

Step 4.2: Write the slot number (Z) in the read slot address (RSLR).

Step 4.3: Increase the pointer for the read slot (RSLP) by one.

Figs. 10 to 17 show detailed operational examples of the MPEG decoder 10 of Fig. Figs. 10 to 15 are diagrams for explaining the updating process of various registers, and Fig. 16 and Fig. 17 are diagrams showing the updating process of the slot management memory (SCM), respectively. 10 to 17, the input bit stream IN is given to the bit stream analyzing unit 12 in the order of I0, P1, P4, B2, B3, and P5, as in the case of FIG. It is assumed that the pictures P1, P4, B2 and P5 have a frame structure and the pictures I0 and B3 have a field structure. For the sake of simplicity, N = 6 or Sn = 9. It is assumed that 16 lines of decode takes 16 lines of output time and decode is not performed in the vertical retrace interval.

10 to 15, each period numbered from 0 to 71 corresponds to the 8-line output period of the image, and the output (display) is made in the order of I0, P1, B2, B3, and P4. The decode start timing of each frame is the output start timing of the last 8 lines of the best field (OPYR = 0). The FR in the Decode Structure Work (DSTW) field indicates the frame structure, and the FD indicates the field structure. Here, paying attention to period 24 in Fig. 12, the value of the lowest use write field register B_WFDR and the value of the read field register RFDR are all 4. As a result, data is written to the field 4 and data is read from the field 4 at the same time. However, in the period 24, since the value of the read slot register RSLR already reaches 5 and the values of the first and second write slot registers WSR1 and WSR2 are all 0, no problem occurs at all. From the period 43 to the period 59, the value of the most useful write field register T_WFDR, the value of the least use write field register B_WFDR, and the value of the read field register RFDR are all zero. As a result, writing and reading to and from the field 0 and three additional slots are performed simultaneously for the coding type B picture. However, there is no problem since the value of the first write slot register WSR1 differs from the value of the second write slot register WSR2 and the value of the read slot register RSLR in any period from period 43 to period 59 Do not.

Figs. 16 and 17 show the write slot setting procedure (procedure 3.1 to procedure 3.7) in the case where the decoded work DTYW is B and the read slot setting procedure in the case where the output type work OTYW is B 4.3) to update the slot management memory (SCM). 16 and 17, each row represents the word of the slot management memory (SCM), and each column corresponds to the 8-line output period of the image. As shown in the leftmost column in FIG. 16, each word from addresses 0 to 8 of the slot management memory (SCM) stores an initial value incremented by one from 0 to 8 in order. The initial value of the write slot pointer WSLP is 0, and the initial value of the read slot pointer RSLP is 9. It is assumed that the slot management memory (SCM) has a capacity of 20 words.

The second and third columns from the left in Fig. 16 correspond to the decode periods of the first 16 lines of the picture (B2) having the frame structure, that is, the periods 36 and 37 in Fig. In these periods, slot number 0 is extracted from the address (address 0) indicated by the write slot pointer WSLP in the slot management memory SCM and slot number 1 is extracted from the next address (address 1) A slot number 0 is written to the first write slot register WSR1 and a slot number 1 is written to the second write slot register WSR2. At this time, in order to prepare for the setting of the read slot register (RSLR) in accordance with steps 3.5 and 3.6, the slot number 0 is assigned to the word of the address 9 in the slot management memory (SCM) Respectively. Then, the write slot pointer WSLP is increased by two.

The ninth column from the left in Fig. 16 corresponds to the output period of the first eight lines of the best field of the picture B2, that is, the period 43 in Fig. In this period, the slot number 0 is extracted from the word (address 9) indicated by the pointer RSLP for the read slot in the slot management memory SCM and the slot number 0 is written in the read slot register RSLR. The slot number 0 is a number written in the slot management memory (SCM) by a half frame period before. Then, the pointer (RSLP) for the read slot is incremented by one.

The 14th and 15th columns from the left in Fig. 16 correspond to the decode periods of the first 16 lines of the picture (B3) having the field structure, that is, the periods 48 and 49 in Fig. In these periods, slot number 6 is fetched from the address (address 12) indicated by pointer WSLP for write slot in the slot management memory SCM and slot number 8 is fetched from the next address (address 13) The slot number 6 is written to the first write slot register WSR1 and the slot number 8 is written to the second write slot register WSR2. At this time, in order to prepare for setting of the read slot register (RSLR) in accordance with steps 3.5 and 3.6, slot number 6 is written to the address 1 word and slot number 8 is written to the word address 2 respectively in the slot management memory (SCM) . Then, the write slot pointer WSLP is increased by two.

The fifth column from the left in Fig. 17 corresponds to the output period of the first eight lines of the best field of the picture B3, that is, the period 55 in Fig. In this period, slot number 6 is extracted from the word (address 1) indicated by the pointer RSLP for the read slot in the slot management memory SCM and slot number 6 is written in the read slot register RSLR. The slot number 6 is a number written in the slot management memory (SCM) by a half frame period. Then, the pointer (RSLP) for the read slot is incremented by one.

16 and 17, the address of the word assigned with the code R (one word used for the setting of the read slot) than the address of the word given the code W (two words used for setting the write slot) Is always large. Therefore, it is guaranteed that the slot to be written is the slot whose reading has been completed. Even if the high-speed decode is achieved and the decode period of the 16 lines becomes shorter than the output period of the 16 lines, no problem arises because the control section 14 is in the standby state until the above condition 4 is satisfied.

As described above, according to the MPEG decoder 10 of Fig. 1, under the condition that the input bit stream IN is not decoded in the vertical retrace interval, the memory blocks 20, 21, 22, 23, and 24 and the three additional slots 25, the capacity of the frame memory 11 is reduced to the capacity of about 2.5 frames.

In addition, the number of additional slots can be reduced. Fig. 18 shows a case where interlace conversion of a B picture is realized by N + 2 slots (N = 6). However, the fifth column from the right in Fig. 18 requires a decoding speed twice as high as the other columns. FIG. 19 shows a case where interlace conversion of a B picture is realized by N + 1 slots (N = 6). However, the fifth column from the right in Fig. 19 requires that decoding for 16 lines be completed within the horizontal retrace period. Therefore, the interlace conversion of B pictures can be realized with N + 3 slots, and even in this case, one memory chip having a storage capacity of 16 Mbits can realize all the memory functions required for MPEG2.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is evident that many alternatives, modifications, and variations will be apparent to those skilled in the art.

Claims (12)

An image processing apparatus including a frame memory, Wherein the frame memory is composed of five blocks each having a capacity of 1/2 frame of image and one additional block having a capacity smaller than that of each of the five blocks, First, four blocks selected from the five blocks are used for storing I and / or P pictures for reference of motion compensation, and one block and one additional block used for interlace conversion of the B picture Constitute a data memory, And then the data memory for the B picture is reconstructed into one block in the four blocks and the additional block. The method according to claim 1, Each of the five blocks and the additional block are each divided into a plurality of slots each having a constant capacity, The image processing apparatus includes: A slot management memory for storing a slot number, Further comprising a controller for controlling reading of the data memory using the slot number stored in the slot management memory, Wherein the controller writes the slot number used for writing the data memory into the slot management memory for reading the data memory. 3. The method of claim 2, A write slot pointer for designating a storage position in the slot management memory of a slot number used at the time of writing the data memory, Further comprising a pointer for a read slot for specifying a storage position in the slot management memory of a slot number used in reading the data memory, And the pointer for the write slot and the pointer for the read slot are updated so that writing of the slot is performed immediately after any slot of the data memory is read. 3. The method of claim 2, Wherein each slot of said data memory has a capacity of data units of a number of lines corresponding to 1/2 of the number of lines of decoding processing units. 5. The method of claim 4, Wherein the controller controls writing of the data memory using two slot numbers. 5. The method of claim 4, When the B picture has the frame structure, the controller controls the writing of the data memory using the two slot numbers so that the data units of the top field and the bottom field constituting the B picture are respectively written to the data memory The image processing apparatus comprising: The method according to claim 6, Wherein the controller separates the two slot numbers by an amount that the pointer for the read slot changes from the data read of the best field to the data read of the lowest field and writes the two slot numbers into the slot management memory. 5. The method of claim 4, The controller controls writing of the data memory by using two slot numbers so that upper and lower half data units constituting the B picture are written in the data memory when the B picture has a field structure, To the image processing apparatus. 9. The method of claim 8, Wherein the controller writes two slot numbers into the slot management memory so that the pointer for the read slot continuously reads the field data of the upper half and the lower half. 3. The method of claim 2, A write slot pointer for designating a storage position in the slot management memory of a slot number used at the time of writing the data memory, Further comprising a pointer for a read slot for specifying a storage position in the slot management memory of a slot number used in reading the data memory, Wherein the controller controls the value of the write slot pointer to be always smaller than the value of the pointer for the read slot. A data memory having a plurality of slots, A slot management memory for storing a slot number, And a controller for controlling reading and writing of the data memory using the slot number stored in the slot management memory, The controller controls reading and writing of the data memory so that the slot is written immediately after reading of any slot in the data memory and reading of another slot is performed in the first half of the time allotted to the writing And the image processing apparatus. 12. The method of claim 11, Further comprising a frame memory composed of five blocks each having a capacity of an image of one half frame and one additional block having a capacity smaller than that of each of the five blocks, First, the four blocks selected from the five blocks are used for storing I and / or P pictures for reference of motion compensation, and the remaining one block and the additional block are used for interlace conversion of the B picture Configure the memory, And then the data memory for the B picture is reconstructed into one block in the four blocks and the additional block.