KR100943563B1 - Moving picture encoding device and moving picture decoding device - Google Patents

Abstract

A video encoder (70) for encoding a moving picture, comprising: a buffer (16c) having a plurality of memory areas capable of storing image data of a frame including a top field and a bottom field, and image data stored in the memory area. The motion prediction unit 19, the motion compensator 16d, the difference unit 11, the transform unit 13, and the quantization unit for encoding the input image in field units while performing motion prediction and motion compensation by referring to the unit 14), a memory management unit 71 for managing a plurality of memory areas on a frame basis, and a plurality of memories under the management by the memory management unit 71 to decode the image data of the coded field and manage the image data of the decoded field. The inverse quantization part 16a, the inverse discrete cosine transform part 16b, etc. for storing in any one area are provided.

Description

MOVING PICTURE ENCODING DEVICE AND MOVING PICTURE DECODING DEVICE

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to memory management control of a multi-frame buffer used for storing reference picture data in video encoding and decoding, and more particularly, to a memory management method for interlaced video data.

Motion pictures are being employed in a number of applications that range from video technology and video conferencing to DVDs and digital televisions. In order to transmit a moving picture, it is necessary to transmit a considerable amount of data through a conventional transmission path having a limited effective frequency band. In order to transmit digital data in a limited transmission band, it is essential to compress or reduce the amount of transmission data.

In order to enable mutual use among a number of systems designed for applications by different manufacturers, an image coding standard has been devised that compresses the amount of image data in a common manner. The video encoding standards are H.261, H.263 of ITU, and MPEG-1, MPEG-2 and MPEG-4 of ISO / IEC.

The encoding approach that is the basis of many standards consists of the following main stages.                 

1. Each picture is divided into blocks composed of pixels so that processing of the pictures constituting the video can be made at the block level. A picture represents a frame or field.

2. The spatial redundancy of the picture is reduced by performing transformation, quantization, and entropy encoding on the video data of one block.

3. The difference between successive frames is encoded using the correlation between successive pictures.

This is accomplished by the technique of motion prediction and compensation. In order to determine the motion vector representing the predictive image data with strong correlation between frames for each block, the encoder performs motion detection for searching for a highly correlated image data position within the encoded frame. In addition, the encoder and the decoder perform motion compensation for extracting the predictive picture data corresponding to the motion vector.

An example of the configuration of a video encoder (video encoding device) is as shown in FIG. The illustrated image encoder includes a transformer 13 for transforming spatial image data into a frequency domain, a quantizer 14 for quantizing transform coefficients obtained by the transformer 13, and a variable length for entropy encoding the quantized transform coefficients. The encoder 15 includes an image buffer 17, a decoder 16, and a motion predictor 19 for supplying the compressed video data having a variable bit rate to the transmission path in accordance with the transmission rate.

In the video data 10 of the encoder shown in FIG. 1, pixel values are input to PCM (pulse code modulation). The difference unit 11 calculates a difference value between the image data 10 and the motion compensation image 12. The motion compensation image 12 is obtained by decoding the already encoded image and performing motion compensation ("decoded image of the object"). This is done by the decoder 16 corresponding to the video encoder. The decoder 16 reverses the encoding order, i.e., the decoder 16 adds the dequantized unit Q-1, the inverse transform unit IDCT, and the decoded difference with the motion compensation image to obtain the same transposition as that obtained at the decoding side. Before) an adder for generating an image.

In motion compensation encoding, motion compensation data of a target picture is generated from picture data of a corresponding decoded picture based on prediction of motion between the picture and the decoded picture. The predicted value of the motion is represented by a two-dimensional motion vector representing the displacement of the pixel between the decoded picture and the target picture. In general, motion prediction is performed in units of blocks. In other words, the block having the strongest correlation with the block of the target frame in the decoded picture becomes a motion compensation image. The encoder includes a motion predictor 19 for performing motion prediction and a motion compensator MC for generating a motion compensated image from a picture decoded corresponding to the motion vector.

The video encoder shown in FIG. 1 operates as follows. The video image of the video signal 10 is divided into a certain number of small block groups, commonly called macro blocks. For example, the video image 20 shown in FIG. 2 is divided into a plurality of macro blocks 21. Each macro block typically has a pixel size of 16x16.

The picture is also divided into any number of slices 22. Each slice consists of a number of macroblocks and is a unit of synchronization return in case of data loss. In addition, the arrangement of macroblocks constituting one slice need not be composed of only macroblocks in the same row as shown in FIG. 2, and includes macroblocks in multiple rows, or different slices in the middle of a row. There may be a short circuit.

When image data of an image is encoded only by reducing the spatial redundancy in the image, the resulting picture is called an I picture. The I picture is encoded with reference only to pixel values in the picture. Since the coded I picture cannot use temporal information for reducing the amount of data, the size of the coded data is large.

In order to perform efficient compression using the temporal redundancy existing between successive pictures, predictive encoding is performed between successive pictures based on motion prediction and compensation. When the reference picture selected in the motion prediction is one picture that has already been encoded and decoded, it is called a P picture. In addition, when there are two reference pictures (normally, the display order is forward and backward with respect to a target picture), it is called B picture.

In the H.26L standard, which is currently being developed, the motion compensation in each 16x16 macroblock can be performed using a different block size. Individual motion vectors can be determined for blocks of 4x4, 4x8, 8x4, 8x8, 8x16, or 16x16 pixels, which are block sizes of motion compensation. The usefulness of making a small motion compensation block is that it can describe fine movement.

Based on the result of the motion prediction process, the motion compensation process performs prediction based on the determined motion vector. The information included in the prediction error block obtained from the predicted block is converted into a transform coefficient by the transform unit 13. In general, two-dimensional DCT (Discrete Cosine Transform) is often used. The obtained transform coefficients are quantized, and finally, entropy coding (VLC) is performed in the entropy coding unit 15. The motion vector calculated by the motion predictor 19 is used for motion compensation and is included in the compressed video data 18 via the variable length encoder 15 and the image buffer 17 and output.

The transport stream of the compressed video data 18 is transmitted to a decoder (video decoding device), where a sequence of encoded video images is reproduced based on the received data. The configuration of the decoder corresponds to the configuration of the decoder 16 included in the video encoder shown in FIG.

In the new video encoding scheme, in order to enable more efficient encoding of an image, a plurality of reference pictures between pictures may be provided. For this reason, the motion predictor and the motion compensator include a multi-frame buffer for providing various reference pictures. Information representing individual reference pictures is added to the motion vector.

The internal structure of the multi-frame buffer is as shown in Fig. 3, and reference numeral 30 denotes the whole. The multi-frame buffer is composed of a plurality of memory areas 31 and 32 that store frames of video signals. The memory area of the multi-frame buffer 30 includes two different kinds of memory areas, namely, a short time area 33 in which a reference picture mainly used as a reference picture is stored, and a reference picture mainly used for a long time. It is divided into a long time region 34.                 

The multi-frame buffer stores appropriately selected reference pictures for encoding or decoding a particular picture. The storage order of the reference picture is divided into two processing stages: (1) replacement of the reference picture and (2) buffering of the reference picture.

(1) Reference pictures are listed based on reference picture order information transmitted to the slice layer. The order setting of the reference picture affects only the encoding or decoding processing of the macroblock group included in one slice. The purpose of this process is to reduce the number of bits of information indicating the reference picture to be referred to at the time of motion compensation by assigning a small number to a frequently referenced picture, and assigning a code having a shorter code length as the smaller number.

(2) In the buffering of the reference picture, the buffering of the picture to be encoded or decoded is controlled when updating the reference picture stored in the multi-frame buffer for each encoding / decoding process.

One of two different memory management control modes, i.e., "shift window buffering mode" or "adaptive memory control buffering mode", can be used for buffering the reference picture.

In the shift window buffering mode, each encoded or decoded image of the object is stored in addition to the multi frame buffer. The pictures in the short time region of the multi-frame buffer are periodically replaced by new images in a first-in first-out (FIFO) method. As long as the buffer has an unused memory area of sufficient capacity, it is not necessary to erase any image data in order to store image data of the image currently being processed. If the unused area of the multi-frame buffer becomes full with new image data that has already been processed, the earliest stored image data is replaced with image data of the new image currently encoded / decoded.

In the adaptive memory control buffering mode, each picture stored in addition to or deleted from the multi-frame buffer is explicitly selected. Memory control is performed in accordance with memory management control processing parameters which ensure the corresponding memory management control on the encoding and decoding side. In order to perform such image replacement processing, a unique identification number for explicitly designating a target picture is assigned to each memory area. In addition, an index indicating an order after replacing the reference picture of (1) is assigned to each memory area, which is called a reference index.

Several problems remain in the above memory management control mode. In the conventional memory management control mode, interlace image data cannot be handled effectively. The interlaced video data has a problem that memory management is complicated because each frame is composed of two fields (top field and bottom field) having different spatial positions in time and vertical direction, and can be encoded in units of fields.

Accordingly, an object of the present invention is to provide a moving picture encoding device, a moving picture decoding device, a memory management device, and the like capable of efficiently managing picture memory in encoding and decoding interlaced video data.

In order to achieve the above object, a moving picture encoding apparatus according to the present invention includes a storage means having a plurality of memory areas capable of storing image data of a frame including a top field and a bottom field, and image data stored in the memory area. Encoding means for encoding an input image on a field basis while performing motion prediction and motion compensation by referring to a field unit, and memory management means for managing the plurality of memory areas on a frame basis. And storing means for decoding the image data of the field encoded by the encoding means and storing the image data of the decoded field in any one of the plurality of memory areas under management by the memory management means. .

Furthermore, the moving picture decoding apparatus according to the present invention refers to storage means having a plurality of memory areas capable of storing image data of a frame including a top field and a bottom field, and image data stored in the memory area on a field basis. Decoding means for decoding an input image on a field basis while performing motion compensation, memory management means for managing the plurality of memory areas on a frame basis, and image data of a field decoded by the decoding means on the memory management means. And storage means for storing in any one of the plurality of memory areas under management by the controller.

Furthermore, the memory management apparatus according to the present invention is a memory management apparatus in an apparatus for encoding or decoding a moving image, comprising: storage means having a plurality of memory regions capable of storing image data of a frame including a top field and a bottom field; Management means for managing the plurality of memory areas on a frame-by-frame basis; and storing image data of a field in any one of the plurality of memory areas under management by the management means; And memory access means for reading from one.

According to these moving picture coding apparatuses, moving picture decoding apparatuses, and memory management apparatuses, the storage of image data into each memory area capable of storing one frame data or two field data (top field data and bottom field data), and each Regardless of whether the reading of the image data from the memory area is performed in either the frame unit or the field unit, memory management is always performed in the frame unit, so that memory management in encoding and decoding of interlaced video data is simplified and efficient. In particular, in the case where a picture encoded in interlace video data and a picture encoded in field units are mixed, only memory management in a unified frame unit is solved, so that efficient memory management is realized.

The memory management means manages whether or not image data can be stored for each of the plurality of memory areas on a frame-by-frame basis, and the storage means is stored in the memory area managed by the memory management means. You may store the image data of a field. That is, the memory management means manages the memory area by setting a storage permission flag corresponding to each of the plurality of memory areas to " use " or " unused, " and the storage means indicates that the storage permission flag indicates " unused. &Quot; You may store the image data of the said field in a memory area. For example, when the image management data of a frame, a top field or a bottom field is stored in the memory area, the memory management means sets a storage permission flag corresponding to the memory area to "use", and the image data is stored in the memory area. When not stored, when the stored frame becomes unnecessary, or when both the stored top field and the bottom field become unnecessary, the storage permission flag corresponding to the memory area is set to "unused". As a result, a 1-bit storage permission flag is allocated to the memory area storing the two field data, and whether or not the image data is allowed to be stored in each memory area only by determining or changing the state of the storage permission flag of the one bit. Management can be simplified, and the processing relating to management is simplified, and the memory capacity for management is reduced.

In addition, the memory management means may manage, for each of the plurality of memory regions, whether a "short time region" for a short time reference or a "long time reference" for a long time reference is performed on a frame basis. That is, the memory management means may manage the memory area by setting a long time flag corresponding to each of the plurality of memory areas in a "short time area" or a "long time area". For example, when the memory management means changes the image data of the frame, the top field, or the bottom field stored in the memory area in which the long time flag is set in the "short time area", for a long time reference, the long time flag is changed to "long time area." Set to "." As a result, a 1-bit long time flag is allocated to the memory area storing two field data, and the reference attribute of each memory area is determined by simply determining or changing the state of the 1-bit long time flag. Management for " long term reference " can be simplified, and the processing for management can be simplified and the memory capacity for management can be reduced. Moreover, the image data stored in the "short time area" can be moved to the "long time area" only by changing a long time flag from a "short time area".

The present invention can be realized not only as the above-described moving picture coding apparatus, the moving picture decoding apparatus, and the memory management apparatus but also as the moving picture coding method, the moving picture decoding method, and the memory management method, as a program, or a computer on which a program is recorded. It can also be realized as a readable recording medium.

1 shows a block diagram of a motion compensated DPCM image encoder.

2 shows secondary division of a video image for encoding and decoding processing.

3 shows an overall configuration of a multi frame buffer used for motion prediction and motion compensation.

4 shows data stored in association with one memory area.

5 shows a block diagram of a motion prediction / motion compensation unit.

6 shows an example of a control method of a multi frame buffer.

FIG. 7 is a block diagram showing the structure of a video encoder in Embodiment 2. FIG.

8 is a diagram illustrating a type of an input video signal or a coding unit.                 

8A illustrates an example of a video signal encoded in frame units.

8B illustrates an example of a video signal encoded in field units.

8C shows an example of a video signal encoded in a mixed state of a frame and a field.

9 is a flowchart showing an operation procedure of the memory management unit.

10 is a table showing an example of memory management by the memory manager;

FIG. 10 (a) shows an example of memory management corresponding to FIG. 8 (a),

FIG. 10 (b) shows an example of memory management corresponding to FIG. 8 (b),

Fig. 10 (c) shows an example of memory management corresponding to Fig. 8 (c).

11 is a diagram illustrating a storage state of a reference picture in a buffer,

FIG. 11A shows a state of storing a buffer when an input video signal is encoded in units of frames.

FIG. 11B shows a state of storing a buffer when an input video signal is encoded in units of fields.

FIG. 11C shows a state of storing a buffer when an input video signal is encoded in a state where a frame and a field are mixed.

12 is a block diagram showing the configuration of a video decoder according to the present invention.

13 is an explanatory diagram of a recording medium for storing a program for realizing a video encoder and a video decoder according to the present invention by a computer system.

14 is a block diagram showing an overall configuration of a content supply system for realizing a content delivery service according to the present invention.

15 is a diagram showing an example of a mobile telephone according to the present invention.

Fig. 16 is a block diagram showing the structure of the mobile telephone.

17 is a diagram showing the configuration of a digital broadcasting system according to the present invention.

(Embodiment 1)

Hereinafter, the video encoder (video encoding device) according to the first embodiment of the present invention will be described with reference to the drawings. In addition, the video encoder in the present embodiment has the configuration shown in FIG. 1 and the characteristic memory management function shown in FIG. 5 described later. The following description will focus on differences from the conventional video encoder.

As described above, FIG. 3 shows the overall configuration of the multi frame buffer. The multi-frame buffer 30 has a plurality of memory areas 31 and 32 for storing a reference picture consisting of two fields, namely, a top field and a bottom field. Data stored in each of the memory areas 31 and 32 is as shown in FIG. Each memory area 41 is used to store the top field 42 and the bottom field 43, where each field data is written or read independently.

Generating an appropriate reference picture for motion prediction and motion compensation is performed by the memory control section 53 shown in FIG. When the encoding or decoding of each picture is completed, the memory unit (multiframe buffer) 52 is updated according to the memory control information. The memory area used to store a new image, for example, image data to be encoded or decoded, is marked by setting a storage permission flag 44 assigned to each memory area one by one. The reference picture required for motion prediction or motion compensation of the series of pictures is held in the frame buffer by releasing the storing permission flag. When deleting an image from the memory unit (multi-frame buffer) 52, the storage permission flag 44 is set to the "unused" state. Even if any one field, both fields, or a frame of a frame is stored, the storage permission flag 44 is in a use state. The storage permission flag 44 indicates the state of the frame, but what is stored is managed in each memory (field) unit. When only one of the top field or the bottom field is stored, the other side is stored. It can also indicate that the field of can be stored.

The storage permission flag can be stored in the memory section (multi-frame buffer) 52 or in the memory control section 53 associated with the memory area corresponding to the multi-frame instead. Only one storage permission flag 44 is associated with each top and bottom field pair, so that addition of hardware for handling interlace video data is unnecessary.

In addition, one long time flag 45 may be assigned to each memory area 41. This flag 45 is stored in the memory unit multi frame buffer 52 or the memory control unit 53 associated with each memory area. By using this long time flag 45, it is shown that each memory area is stored in a long time reference image. By setting the long time flag 45 in this manner, the shift processing (movement) of the image from the short time region 33 to the long time region 34 is realized.                 

Further, in the embodiment of the present invention, the long time memory area 34 stores the top or bottom field in either of the top and bottom field positions 42 and 43, thereby improving the efficient storage of the long time image data. . This is achieved by providing two storage permission flags for each long time memory area, such as enabling each field position to indicate "unused".

An example of the memory management control method is shown in FIG. When the encoding or decoding process of the image in step 61 ends, the process of memory control for the multi-frame buffer starts as shown in step 62. In step 63, first, a memory area storing two fields that are not required in later field processing is selected, and then in step 64, the storage permission flag 44 is set to "unused". If there is a memory area in which one field data constituting the same frame is stored, and the other field area of the memory area is unused, the other field data may be stored in the unused field area. For example, if the top field data of the same frame is stored in the memory area and the bottom field data is not stored, the bottom field may be stored in the uncontained bottom field area of the memory area.

In this way, new image data can be written for each memory space of an unused field pair. This selection is made based on the memory control information sent from the video encoder. The video encoder can generate control information based on the definition in the coding standard employed.

The memory control information also selects a memory area having image data for a long time (step 65). In step 66, the long time flag 45 of these memory areas is set.

In step 67, the field to be processed in accordance with the memory control parameter is stored in each "unused" field position in the memory area, and then in step 68, the encoding or decoding process is resumed.

As described above, the present invention relates to a motion prediction and motion compensation unit using a plurality of reference pictures, and the plurality of reference pictures are stored in a memory unit having a memory area of each reference picture. Each memory area is configured to store video data of one frame including a top field and a bottom field. The storage of the video data into the memory unit is controlled by the memory control unit. One storage permission flag that permits storing image data of two consecutive fields in one memory area is assigned to each memory area. As a result, when the video signal is interlaced video data, the coding, decoding, motion prediction, and motion compensation of the image are performed on a field basis or on a frame basis. Since it is always performed on a frame basis, the management process for memory management is simplified.

In addition, although the storage permission flag 44 and the long time flag 45 were demonstrated simultaneously in Embodiment 1, only one of them may be used and the other may not be used, or it may substitute by another method.

(Embodiment 2)                 

Next, a video encoder and a video decoder in Embodiment 2 of the present invention will be described. This embodiment corresponds to a detailed embodiment 1 and is common to the first embodiment in that memory management is performed in units of frames.

7 is a block diagram showing the configuration of the video encoder 70 according to the second embodiment. The video encoder 70 is characterized in that the memory management for the multi-frame buffer is performed frame by frame even if the structure and coding unit of the input video signal 10 are frame unit, field unit, and a mixture thereof. (11), a converter (13), a quantizer (14), a variable length encoder (15), an image buffer (17), a decoder (16), a motion predictor (19), and a memory manager (71). . The decoder 16 includes an inverse quantization unit 16a, an inverse discrete cosine transform unit 16b, a buffer 16c, and a motion compensation unit 16d. In addition, the same code | symbol is attached | subjected to the same component as the conventional video encoder shown in FIG. 1, and the video encoder in Embodiment 1, and the description is abbreviate | omitted.

The video encoder 70 of the present embodiment includes a characteristic memory manager 71. The memory manager 71 moves to be the same processing unit as that unit based on external information indicating in which unit (frame unit, field unit, or mixture thereof) the image of the input video signal 10 is encoded. The compensator 16d and the motion predictor 19 are instructed, and the reference picture stored in the multi-frame buffer, that is, the buffer 16c, is frame-by-frame. Specifically, as shown in FIG. 8A, when the picture of the input video signal 10 is encoded in units of frames in a group of picture (GOP) or the like, the motion compensation unit 16d and the motion prediction While processing in the unit 19 is performed in units of frames, memory management is performed in units of frames, and as shown in Fig. 8B, the picture of the input video signal 10 is a GOP (or sequence). In the case of encoding in units of fields in the above, the control is performed so that the processing in the motion compensator 16d and the motion predictor 19 is performed in units of fields, and the memory management is performed in units of frames. As shown in c), when a picture of the input video signal 100 is encoded with a mixture of frames and fields for each picture, GOP (or sequence), etc., the picture may be corresponding to the coding unit. As at the same time, the processing in the motion compensation unit (16d) and the motion predicting unit 19 for performing control such that in a frame unit or a field unit to the memory management is performed on a frame-by-frame basis.

The contents of the memory management are basically the same as those in the first embodiment, where a 1-bit storage permission flag for each memory area, which is a unit for storing one frame data or two field data (top field data and bottom field data), and The 1-bit long time flag is allocated, and the storage area is either "Long Time" / "Either of" Use "or" Unused "for each image data (in this embodiment, every frame or every pair of top and bottom fields). It is either determined or recorded in the "short time" area. These flags are provided in the buffer 16c or the memory manager 71. In addition, "unused" and "used" indicate that image data (frame data, top field data, or bottom field data) can be stored in the corresponding memory area, and that the storage is prohibited. The "long time area" and the "short time area" correspond to respective areas in the case where the buffer 16c is divided into two types of storage areas, that is, a storage area for mainly long-term reference and a storage area for mainly short-term reference. .

9 is a flowchart showing an operation procedure of the memory manager 71. The memory manager 71 determines the unit of coding of the input video signal 10 based on the input external information (step S70), and when the picture of the video signal 10 is encoded in units of frames (step S70). (Frame unit)), the motion compensation by the motion compensator 16d and the motion prediction by the motion predictor 19 are controlled to be in units of frames, and the reference image is stored in the buffer 16c in units of frames. / Removal and storage in the long time area / short time area (step S71), and on the other hand, when the picture of the video signal 10 is encoded in field units ("field units" in step S70), the motion compensation unit 16d The motion compensation and the motion prediction by the motion prediction unit 19 are controlled to be in the field unit, and the storage / removal of the reference image to the buffer 16c in units of frames and the long time / short time area Storage is carried out (step S72).

10 is a table showing an example of memory management by the memory manager 71. When the input video signal 10 is encoded in units of frames, the memory manager 71 stores a storage permission flag and a long time flag corresponding to each frame (i.e., a memory area) as shown in FIG. By using it, it is recorded, referred to, or updated in which state of " use " / " unused " and in which " long time " / " short time "

On the other hand, when the input video signal 10 is encoded in units of fields, the memory management unit 71 stores the top field and the bottom field corresponding to each pair (i.e., memory area) as shown in Fig. 10B. By using the permission flag and the long time flag, etc., it is recorded, referred to, or updated in which state of the "use" / "unused" state is stored for each pair, i.e., frame-by-frame, and which area is stored in the "long time" / "short time" area. do. This detailed procedure is as shown in the flowchart of FIG. 6 in Embodiment 1. FIG.

When the input video signal 10 is encoded in a state where the frame unit and the field unit are mixed, the memory management unit 71 stores the storage permission flag associated with each frame with respect to the frame as shown in Fig. 10C. And a long time flag or the like, and a storage permission flag and a long time flag corresponding to each pair of the top field and the bottom field for the field, and the like. Or "unused" and which one of the "long time" and "short time" areas is stored, referenced, or updated.

Here, the specific control procedure of the storing permission flag is as follows. In other words, when image data is not yet stored in the memory area, frame data stored in the memory area becomes unnecessary (it is determined not to be used), or top field data and bottom field data stored in the memory area. When both are unnecessary (it is determined that neither field data is used), the storage permission flag corresponding to the memory area is set to "unused". This makes it possible to store new frame data or field data in the memory area.

On the other hand, when new frame data, top field data or bottom field data is stored in the "unused" memory area, the storage permission flag corresponding to the memory area is set to "used". As a result, the storage of other frame data or field data into the memory area is prohibited, and it is ensured that the frame data or field data already stored is held in the memory area. When the top field data or the bottom field data is stored in the "unused" memory area, the storage permission flag corresponding to the memory area is set to "used", but the data of the bottom field or the top field constituting the same frame is set. Is controlled to be stored in the field area on the other side of the memory area.

In addition, the specific control procedure of a long time flag is as follows. In other words, the image data stored in the memory area (if frame data is stored, the corresponding frame data, if only the top field data is stored, the corresponding top field data, and if only the bottom field data is stored, the bottom field data is stored. When both the top field data and the bottom field data are stored, when the top field data and the bottom field data are both used for a short time reference, the long time flag corresponding to the memory area is set in the "short time area". do. This makes it possible to use the buffer memory of the FIFO at the same time as the memory areas of other "short-time areas".

On the other hand, the image data stored in the memory area (if frame data is stored, the corresponding frame data, if only the top field data is stored, the corresponding top field data, and if only bottom field data is stored, the corresponding bottom field data) When both the top field data and the bottom field data are stored, when the top field data and the bottom field data are both used for long-term reference, the long time flag corresponding to the memory area is set in the "long time area". do. This ensures that the image data is held in the memory area until it is explicitly deleted ("unused") in the memory area. Moreover, when changing the image data of the frame, the top field, or the bottom field stored in the memory area in which the long time flag is set in the "short time area" for long time reference, the long time flag is set in the "long time area". As a result, the image data can be easily moved from the "short time zone" to the "long time zone".

FIG. 11 is a diagram illustrating a storage state of a reference picture in a buffer. FIG. Fig. 11 (a) shows the storing state of the buffer 16c when the input video signal 100 is encoded in units of frames, and Fig. 11 (b) shows that the input video signal 10 is in units of fields. The storage state of the buffer 16c in the case of encoding is shown, and FIG. 11 (c) shows the storage state of the buffer 16c in the case where the input video signal 10 is encoded in a state in which frames and fields are mixed. Shows.

As described above, according to the video encoder 70 according to the present embodiment, even if the coding unit of the input video signal 10 is any of a frame unit, a field unit, and a mixture thereof, memory management is performed in a frame unit. As a result, the processing load and circuit size for memory management are reduced compared to the complicated memory management method in which the frame unit and the field unit are mixed.

In addition, the memory manager 71 according to the present embodiment can be applied not only to a video encoder but also to a video decoder (video decoding apparatus). FIG. 12 is a block diagram showing the configuration of the video decoder 100 including the memory manager 109 having the same function as the above-described memory manager 71. The video decoder 100 performs decoding corresponding to the encoding by the input buffer 102 temporarily holding the input encoded video signal 101 and the variable length encoder 15 included in the video encoder 70. A variable length decoder 103, an inverse quantization unit 104 for inverse quantization corresponding to quantization by the quantization unit 14 included in the video encoder 70, and a conversion unit included in the video encoder 70 ( 13, an inverse discrete cosine transform unit 105 that performs inverse transformation of the transformation by the transform unit, an adder 106 that adds an image to output the video signal 110, and a buffer 107 as a multi-frame buffer which stores the reference image. And a motion compensation unit for acquiring a motion vector included in the encoded video signal 101 through the variable length decoder 103 and performing motion compensation on the reference image stored in the buffer 107 using the motion vector ( 108), memory management And a 109.

The memory manager 109 based on the external information indicating in which unit (frame unit / field unit) the picture of the input encoded video signal 101 is encoded, the motion compensator 108 to be the same processing unit as the unit. ), And manages the reference image stored in the multi-frame buffer, that is, the buffer 107 on a frame-by-frame basis. Specifically, in the case where the input picture of the encoded video signal 101 is encoded in units of frames in a GOP (or sequence) or the like, the motion compensation unit 108 controls the processing to be performed in units of frames. Is performed in units of frames, and when the picture of the input coded video signal 101 is encoded in field units in a GOP (or sequence) or the like, the control in the motion compensator 108 is performed in units of fields. At the same time, memory management is performed in units of frames. Further, regarding the details of the memory management in the video decoder 100, except that the motion compensation is performed by using the motion vector included in the input encoded video signal 101, the video encoder 70 Same as memory management.

As mentioned above, although the moving picture coding apparatus, the moving picture decoding apparatus, and the memory management concerning the present invention have been described based on the two embodiments, the present invention is not limited to these embodiments.

It is not necessary to simultaneously provide the storing permission flag 44 and the long time flag 45 like this embodiment, and may use only one and do not use the other, or may substitute by another method.

If emphasis is placed on improving memory management and improving the use efficiency of buffer frames rather than simplifying memory control, memory management may not be fixed on a frame-by-frame basis, but frame units and field units may be mixed. For example, by assigning a 1-bit storage permission flag and a 2-bit long time flag to each memory area, the management of "used" / "unused" is carried out in units of frames, and the management of "short time area" / "long time area". It may be performed in field units.

In the above embodiment, memory management ("use" / "not used", "long time area" / "short time area") is realized by a flag corresponding to a picture (frame or field). It is not limited to management. For example, a management table that records the number (or reference index or number representing each area of the buffer) in the "used" (or "unused") state among the pictures stored in the buffer, or the "short-term area" The same memory management may be performed by providing a management table which records the number (or reference index or number indicating each area of the buffer) stored in the picture (or "long time area").

The video encoder and video decoder having the memory management function shown in the above embodiments can also be realized as a program. By recording such a program on a recording medium such as a flexible disk and distributing it, a general-purpose computer installed at every place can function as the video encoder or video decoder according to the present invention.

FIG. 13 is an explanatory diagram of a recording medium for storing a program for realizing the video encoder and video decoder of the above embodiment by a computer system. Fig. 13A shows an example of a physical format of a flexible disk as a main body of a recording medium. Fig. 13B shows the appearance, the cross-sectional structure, and the flexible disk seen from the front of the flexible disk. The flexible disk FD is embedded in the case F, and a plurality of tracks Tr are formed on the surface of the disk in a concentric manner from the outer circumference toward the inner circumference, and each track has sixteen sectors Se in the angular direction. It is divided into Therefore, in the flexible disk which stores the said program, the video encoder and the video decoder as the said program are recorded in the area | region allocated on the said flexible disk FD. Fig. 13C shows a configuration for recording and reproducing the program on the flexible disk FD. When the program is recorded on the flexible disk FD, the video encoder and the video decoder as the program are written from the computer system Cs via the flexible disk drive. When the video encoder and the video decoder are built in the computer system by the program in the flexible disk, the program is read out from the flexible disk by the flexible disk drive and transferred to the computer system. The recording medium storing the program is not limited to a flexible disk, but may be an optical disk such as a CD-ROM or a DVD-ROM, a memory card, a ROM cassette, or the like.

In addition, the video encoder and the video decoder according to the present invention can be applied to various devices or systems. An application example of the video encoder and the video decoder in the above embodiment will be described below.

Fig. 14 is a block diagram showing the overall configuration of a content supply system ex1OO that realizes a content delivery service. The area for providing communication service is divided into cells of desired size, and base stations ex107 to ex110 which are fixed wireless stations are provided in respective cells. This content supply system ex1OO is connected to a computer ex111 and a PDA (personal digital assistant) ex112 via the Internet service provider ex102, the telephone network ex104, and the base stations ex107 to ex110, for example, to the Internet ex1O1. Each device such as a camera ex113, a mobile phone ex114, a mobile phone with a camera ex115 and the like is connected. The content supply system ex100 is not limited to the combination as shown in Fig. 14, and may be connected in any combination. Further, each device may be directly connected to the telephone network ex104 without passing through the base stations ex107 to ex110 which are fixed wireless stations.                 

The camera ex113 is a device capable of shooting a moving picture such as a digital video camera. In addition, the mobile phone may be a personal digital communications (PDC) method, a code division multiple access (CDMA) method, a wideband-code division multiple access (W-CDMA) method, or a global system for mobile communications (GSM) method, or a PHS. (Personal Handyphone System) and the like, which may be any.

The streaming server ex103 is connected from the camera ex113 via the base station ex109 and the telephone network ex104, and live delivery based on the encoded data transmitted by the user using the camera ex113 can be performed. The encoding process of the photographed data may be performed by the camera ex113, or may be performed by a server or the like which performs the data transmission process. The moving picture data shot by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device capable of capturing still images and moving pictures such as digital cameras. In this case, the encoding of the moving picture data may be performed by the camera ex116 or by the computer ex111. The encoding process is performed by the LSI ex117 included in the computer ex111 or the camera ex116. The software for picture coding and decoding may be stored in any storage medium (CD-ROM, flexible disk, hard disk, etc.) which is a recording medium that can be read by a computer ex111 or the like. The moving picture data may also be transmitted to the cellular phone ex115 with the camera. The moving picture data at this time is data encoded by the LSI included in the cellular phone ex115.

In the content supply system ex1OO, the streaming server ex103 performs a coding process on the content (for example, a video shot of music live, etc.) that the user is photographing with the camera ex113, the camera ex116, or the like as in the above embodiment. On the other hand, the streaming server ex103 streams the content data to the client that made the request. The client includes a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114 and the like capable of decoding the encoded data. In this manner, the content supply system ex100 can receive and reproduce the encoded data at the client, and can also receive and decode and reproduce the encoded data in real time at the client, thereby realizing personal broadcasting.

What is necessary is just to use the video encoder or video decoder shown by said each embodiment for the encoding and decoding of each device which comprises this system.

As an example, a mobile phone will be described. Fig. 15 is a diagram showing the cell phone ex115 using the video encoder and video decoder described in the above embodiments. The mobile phone ex115 captures images of an antenna ex201 for transmitting and receiving radio waves between the base station ex110, a CCD camera, and the like, a camera unit ex203 capable of taking still images, and a camera unit ex203. , A display unit ex202 such as a liquid crystal display for displaying the decoded data such as an image received by the antenna ex201, a main body composed of a group of operation keys ex204, and an audio output unit such as a speaker for audio output ( ex208), to store encoded data or decoded data, such as a voice input unit ex205 such as a microphone for voice input, data of a captured moving picture or still picture, data of a received mail, data of a moving picture or still picture, etc. And a slot section ex206 for attaching the recording medium ex207 to the mobile phone ex115. The recording medium ex207 stores a flash memory device, which is a type of electrically erasable and programmable read on memory (EPEROM), which is a nonvolatile memory that can be electrically rewritten or erased in a plastic case such as an SD card.

The mobile phone ex115 will also be described with reference to FIG. The cellular phone ex115 controls the power supply circuit ex310, the operation input control unit ex304, and the power control unit ex310 to the main control unit ex311 which collectively controls each unit of the main body unit including the display unit ex202 and the operation key ex204. An image coding unit ex312, a camera interface unit ex303, an LCD (Liquid Crystal Display) control unit ex302, an image decoding unit ex309, a multiple separation unit ex308, a recording / reproducing unit ex307, a modulation / demodulation circuit unit ex306 ) And voice processing unit ex305 are connected to each other via a synchronization bus ex313.

When the call termination and the power key are turned on by the user's operation, the power supply circuit unit ex310 activates the digital cellular phone ex115 with the camera in an operable state by supplying power to each unit from the battery pack.

The cellular phone ex115 uses the voice processing unit ex305 to collect a voice signal collected by the voice input unit ex205 in the voice call mode under the control of the main control unit ex311 including the CPU, ROM, RAM, and the like. The data is converted into data and subjected to spectrum spread processing by the modulation and demodulation circuit section ex306, and then subjected to digital analog conversion processing and frequency conversion processing by the transmission / reception circuit section ex301 and then transmitted via the antenna ex201. In addition, the mobile phone ex115 amplifies the received data received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog digital conversion processing, and performs a spectrum despreading process in the demodulation circuit unit ex306, thereby processing the voice processing unit. After conversion to analog voice data by ex305, it is output through the voice output unit ex208.

When the electronic mail is transmitted in the data communication mode, the text data of the electronic mail input by the operation of the operation key ex204 of the main body is sent out to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spectrum spread processing on the text data by the modulation / demodulation circuit unit ex306 and transmits the digital data to the base station ex11O through the antenna ex201 after performing digital analog conversion processing and frequency conversion processing on the transmission / reception circuit unit ex301. .

When image data is transmitted in the data communication mode, the image data shot by the camera unit ex203 is supplied to the image coding unit ex312 via the camera interface unit ex303. When the image data is not transmitted, it is also possible to display the image data shot by the camera unit ex203 directly on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

The picture coding unit ex312 is configured with the video encoder described in the present invention, and compresses and encodes the picture data supplied from the camera unit ex203 by the coding method used for the video encoder shown in the above embodiment to convert the coded picture data into coded image data. It converts and sends it to the multiple separation part ex308. At this time, the mobile phone ex115 simultaneously transmits the voices collected by the voice input unit ex205 while recording in the camera unit ex203 to the multiplexing unit ex308 as digital voice data through the voice processing unit ex305.

The multiplexer ex308 multiplexes the encoded image data supplied from the image encoder ex312 and the audio data supplied from the voice processor ex305 in a predetermined manner, and the resulting multiplexed data is subjected to the demodulation and demodulation circuit ex306. After spread spectrum processing, the transmission / reception circuit unit ex301 performs digital analog conversion processing and frequency conversion processing, and then transmits them through the antenna ex201.

In the case of receiving data of a moving picture file linked to a homepage or the like in the data communication mode, the received data received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading by the modulation / demodulation circuit unit ex306, resulting in multiplexing. The data is sent to the multiple separator ex308.

In addition, in order to decode the multiplexed data received through the antenna ex201, the multiplexing unit ex308 divides the multiplexed data into a bit stream of image data and a bit stream of voice data, and the corresponding multiplexing unit via the sync bus ex313. The coded image data is supplied to the picture decoding unit ex309, and the voice data is supplied to the voice processing unit ex305.

The picture decoding unit ex309 has a structure provided with the video decoder described in the above embodiments, generates reproduced moving picture data by decoding the bit stream of the picture data with a decoding method corresponding to the coding method shown in the above embodiments, and Moving data is supplied to the display unit ex202 via the LCD control unit ex302, whereby moving picture data included in, for example, a moving picture file linked to the home page is displayed. At this time, the voice processing unit ex305 converts the voice data into analog voice data, and then supplies it to the voice output unit ex208, whereby the voice data contained in, for example, the moving image file linked to the home page is reproduced.

Moreover, not only the example of the said system but digital broadcasting by satellite and terrestrial wave has become a hot topic in recent years, As shown in FIG. It may include. Specifically, the broadcast station ex409 transmits the bit stream of the video information to the communication or broadcast satellite ex410 via radio waves. The broadcast satellite ex410 that has received this transmits a broadcast radio wave, and receives the radio wave through an antenna ex406 of a home having a satellite broadcast reception facility, thereby receiving a television (receiver) ex401 or a set-top box (STB) ex407. A device such as a device decodes a bit stream and reproduces it. In addition, the video decoder shown in the above-described embodiments can be provided in the reproducing apparatus ex403 which reads and decodes the bit stream recorded on the storage medium ex402 such as a recording medium CD or DVD. In this case, the reproduced video signal is displayed on the monitor ex404. In addition, a configuration may be considered in which a video decoder is packaged in a set-top box ex407 connected to a cable ex405 for cable TV or an antenna ex406 for satellite / terrestrial broadcasting, and reproduced by a television monitor ex408. In this case, the video decoder may be included in the television, not in the set top box. Also, in a car ex412 having an antenna ex411, a signal is received from a satellite ex410, a base station ex107, or the like to reproduce a moving picture on a display device such as car navigation ex413 of the car ex412. It is possible.                 

It is also possible to code an image signal with the video encoder shown in the above embodiment and record it on a recording medium. As a specific example, there is a recorder ex420 such as a DVD recorder for recording an image signal on a DVD disk ex421 or a disk recorder for recording on a hard disk. It is also possible to record to the SD card ex422. If the recorder ex420 includes the video decoder shown in the above embodiment, the picture signal recorded on the DVD disk ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.

In addition, the structure of the car navigation ex413 can be considered, for example, except for the camera unit ex203, the camera interface unit ex303, and the image coding unit ex312 among the components shown in FIG. 16. ), Television (receiver) ex401 and the like can also be considered.

In addition, the terminal of the mobile phone ex114 or the like may consider three types of packaging formats, in addition to a transmitting / receiving terminal having both an encoder and a decoder, a transmitting terminal having only an encoder and a receiving terminal having only a decoder.

In this manner, the video encoder or video decoder shown in the above embodiments can be used for all the above-mentioned devices and systems, and the effects described in the above embodiments can be obtained by doing so.

In addition, this invention is not limited to the said embodiment, A various deformation | transformation or correction is possible without deviating from the range of this invention.

The video encoding device, the video decoding device, the memory management device, and the like according to the present invention can be used as an image encoder and a video decoder realized by software executed on an electronic circuit such as an LSI or a computer. For example, it is useful as a video encoder and a video decoder provided in a computer, a PDA, a digital broadcast transmitter, a mobile phone, etc. which encode a moving picture or decode and reproduce the encoded moving picture.

Claims (23)

An apparatus for managing the storage of picture data into a memory in an apparatus for encoding or decoding a moving image. A memory having a plurality of memory areas which are areas for storing one frame of data; Storage means for storing data in the memory; Each memory area has memory management means for managing whether data is prohibited or whether data can be stored; The memory management means is adapted to record the second field data paired with the first field data when the first field data is recorded in the first memory area in which data can be stored. Allowing additional data writing to the first memory area and prohibiting writing of additional data to the first memory area when the second field data paired with the first field data is not recorded. Device characterized in that. A method of managing the storage of picture data into a memory in an apparatus for encoding or decoding a moving picture, A storage step of storing data in a memory having a plurality of memory areas which are areas for storing one frame of data; Each memory area has a memory management step of managing whether data storage is prohibited or data can be stored; The memory management step is performed when the second field data paired with the first field data is recorded when the first field data is recorded in the first memory area in which data can be stored. Allowing additional data writing to the first memory area and prohibiting writing of additional data to the first memory area when the second field data paired with the first field data is not recorded. Characterized in that the method. delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete

Images