JP2001223987A - Memory managing method, picture encoding method, picture decoding method, picture display method, memory managing device and memory managing program recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the possibility of the erasure of screen data required for encoding, decoding and display by enhancing memory bank usage efficiency. SOLUTION: FM1a, FM1b, FM2a and FM2b are respectively the first screen area of a first picture group, the second screen area of the first picture group, the first screen area of a second picture group and the second screen area of the second picture group. Besides, AD12a and AD12b are respectively the first address start position of the first picture group and the second address start position of the first picture group and, then screen sizes SZ1a and SZ1b are secured from each start position toward a high-order address concerning the first picture group. In the second picture group, the screen sizes SZ2a and SZ2b are secured toward the high-order addresses from AD12b-SZ2a and AD34a- SZ2b.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to a memory management method,
Image encoding method, image decoding method, image display method, memory management device, image encoding device, image decoding device, image display device, memory management program recording medium, image encoding program recording medium, image decoding program recording The present invention relates to a medium and an image display program recording medium, and in particular, to a memory management method capable of saving a memory capacity required for simultaneously encoding, decoding, or displaying a plurality of image sequences, and image encoding using the memory. Image decoding, image display methods, these memory management methods, image encoding
The present invention relates to an apparatus corresponding to an image decoding / image display method and a recording medium on which a program for executing the method is implemented by software.

[0002]

2. Description of the Related Art In recent years, the multimedia era in which voice, image, and other pixel values are handled in an integrated manner has been entered, and conventional information media, that is, information such as newspapers, magazines, televisions, radios, and telephones are transmitted to humans. Means have been taken up as a subject of multimedia. Generally, multimedia means not only characters but also graphics, sounds, and especially images, etc. are simultaneously associated with each other. To make the above-mentioned conventional information media the target of multimedia, the information must be expressed in digital form. Is an essential condition.

By the way, when the information amount of each information medium is estimated as a digital information amount, the amount of information per character is 1 to 2 bytes, while the amount of information per character is 1 to 2 bytes. Is 64 kbi per second
At the same time, for a moving image of the current television reception quality, an information amount of 100 Mbit or more per second is required, and it is not realistic to handle the huge amount of information in the above-mentioned information medium in digital form. For example, a videophone
Integrated Services Digital Network (ISDN) with a transmission speed of 64 kbps to 1.5 Mbps
etwork) has already been put into practical use,
It is impossible to send the camera image as it is via ISDN. Therefore, information compression technology is needed. For example, in the case of a videophone, H.264 standardized by ITU-T (International Telecommunication Union Telecommunication Standardization Sector) has been adopted. 261 and H.E. H.263 video compression technology is used. According to the information compression technology of the MPEG1 standard, it is also possible to record image information together with audio information on a normal music CD (compact disc).

Here, MPEG (Moving Picture Exper)
ts Group) is an international standard for pixel value compression of moving images, and MPEG1 is a standard that can compress moving image pixel values to 1.5 Mbps, that is, information of television signals to about 1/100. In addition, since the transmission speed for the MPEG1 standard is mainly limited to about 1.5 Mbps, the moving picture pixel value is compressed to 2 to 15 Mbps in the MPEG2 standardized to meet the demand for higher image quality. You.

Further, at present, MPEG1, MPEG2
And the working group that has promoted standardization (ISO / IEC JTC1 / SC2
9 / WG11), encoding and
MPEG4, which enables operation and realizes new functions required in the multimedia age, is being standardized.
MPEG4 initially aimed to standardize low bit rate coding methods, but is now being extended to more general coding standards, including interlaced image coding and high bit rate coding. .

One of the features of MPEG4 is a mechanism for simultaneously encoding and transmitting a plurality of image sequences. This makes it possible to compose one image scene from a plurality of images. For example, it is possible to make the foreground and the background different from each other and change their frame frequency, image quality, and bit rate individually. It is possible to arrange a plurality of image sequences in a horizontal or vertical direction like a multi-screen in a television receiver or the like, so that a user can extract only a desired image sequence or enlarge and display it.

FIG. 15 shows a typical example in which an image sequence is subjected to MPEG coding. This is MPEG
1 or MPEG2 frame structure
EG4 corresponds to the VOP reference structure of a certain object. In MPEG coding, a frame (I frame) that is coded only by intra-screen calculations, a frame (P frame) that performs inter-screen prediction coding using correlation between screens, and a past frame and a future frame There is a frame (B frame) in which bidirectional prediction encoding is performed based on the frame (B).

FIG. 15 shows, for example, eight consecutive frames FR.
1, FR2,..., FR8 when MPEG coding is performed, where frames FR1 and FR5 are I frames and the remaining frames, ie, FR2, FR3, FR4 and FR6, FR7, FR8.
Is a P frame. The I frame can be decoded by itself, but the P frame cannot be correctly coded / decoded if the screen at the time before reference cannot be coded / decoded correctly. Therefore, when a transmission error of a bit stream or a failure at the time of decoding occurs, correct image data cannot be obtained until the next I frame. By the way, a reference screen is necessary for encoding / decoding of a P frame, and together with one screen for recording data of the encoding / decoding process of the screen, a memory for a total of two screens is one image series. Necessary for encoding / decoding of (object). Similarly, encoding / decoding of n image sequences requires a memory area for 2n screens. Therefore, conventionally, the memory address space is divided into 2n pieces, and each divided area (bank) is used as a memory for one screen, and the image data is stored in this.

FIG. 16 shows an example of a conventional method of dividing a memory address. In the figure, FM1a, FM1b,
FM2a, FM2b, FM3a, and FM3b are a first screen area of a first image sequence, a second screen area of a first image sequence, a first screen area of a second image sequence, and a second image, respectively. A second screen area of a sequence, a first screen area of a third image sequence, and a second screen area of a third image sequence. Also, A
D1a, AD1b, AD2a, AD2b, AD3a, AD3b are the first
A first address start position of the image sequence, a second address start position of the first image sequence, a first address start position of the second image sequence, a second address start position of the second image sequence, The first address start position of the third image sequence and the second address start position of the third image sequence
D1b-1 is the memory bank of the 1a, and AD1b to AD2a
-1 to the first memory bank, and AD2a to AD2b-
1 to the second memory bank, and AD2b to AD3a-1.
Up to correspond to the second memory bank, and AD3a to AD3b-1 correspond to the third memory bank. Also,
SZ1a, SZ1b, SZ2a, SZ2b, SZ3a, SZ3b are the first
Screen size of the first image sequence, second screen size of the first image sequence
Screen size, first screen size of second image sequence, second screen size of second image sequence, first screen size of third image sequence
And the second screen size of the third image sequence.

Here, how to store the image data of each image series will be described by taking as an example a case where the frames FR1, FR2,..., FR8 in FIG. 15 are decoded. These frames
FR1, FR2,..., FR8 constitute one image sequence (first image sequence). This image series is MPEG
4 corresponds to one object, and MPEG2 corresponds to a GOP. First, a frame FR1, which is the first I frame in the first image sequence, is decoded and stored in the screen area FM1a. Next, for frame FR2,
The decoding is performed with reference to the decoded image data of the frame FR1 stored in the screen area FM1a, and the decoding result is stored in the screen area FM1b.

Next, regarding the frame FR3,
The decoding is performed with reference to the decoded image data of the frame FR2 stored in the screen area FM1b, and the decoding result is stored in the screen area FM1a. This is because the image data of the frame FR1 is no longer needed to decode the frame FR3, so that the screen area FM1a may be overwritten. Further, for the frame FR4, decoding is performed with reference to the decoded image data of the frame FR3 stored in the screen area FM1a, and the decoding result is stored in the screen area FM1b.

Hereinafter, a frame FR belonging to the first image sequence will be described.
Similarly for 5 to FR8, two screen areas FM1a, F
Decoding is performed using M1b alternately. Further, decoding is similarly performed for frames belonging to the second and third image sequences. That is, two screen areas FM2a and FM2b are alternately used for frames belonging to the second image sequence, and two screen areas FM3a and FM3b are alternately used for frames belonging to the third image sequence. To perform the respective decoding in the same manner. Further, when performing coding such as inter-screen prediction coding or displaying an image, similarly, coding and image display are performed by alternately using two mutually adjacent screen areas in the memory.

The I frame and the P frame are referred to for decoding the P frame and the B frame, but the B frame is not referred to for decoding other B frames and the P frame. , B frames are not stored in the memory. Therefore, if a B frame exists between the frames FR2 and FR3, the B frame is stored in the frame FR2 stored in the screen area FM1b.
Then, decoding is performed with reference to the frame FR3 stored in the screen area FM1a.

Here, each screen area FM1a, FM1b, FM2a,
FM2b, FM3a, FM3b are the address start positions AD1a, AD1b, AD
With reference to 2a, AD2b, AD3a, and AD3b as starting points, as shown by the arrow "→" in FIG. 16, the data is stored in each area (bank) of the image memory toward its higher address.
Therefore, each screen size SZ1a, SZ1b, SZ2a, SZ2b, SZ3a, S
Even if Z3b increases, as long as the size of each bank of the divided memory is not exceeded, the image data necessary for encoding / decoding the image sequence is not destroyed by the image data of another image sequence. Absent. For this reason, the encoding and
When the size of the memory area required for decoding is fixed, no problem occurs in the configuration of the example of dividing the memory address in FIG.

FIG. 17 is a block diagram of a conventional image coding apparatus. In the figure, the motion detection and motion compensation (MEMC) unit U1
Reads a reference screen FMout from the memory unit FM with respect to the image signal Vin, performs motion detection and motion compensation with reference to this, and outputs a motion vector MV and a motion compensation screen Ref. The subtracter U2 calculates a difference value between the image signal Vin and the motion compensation screen Ref, and outputs the difference value to the discrete cosine transform (DCT) unit U3.
The discrete cosine transform (DCT) unit U3 performs a DCT transform on a block basis, the quantization (Q) unit U4 quantizes the result of the conversion, and the variable length coding (VLC) unit U5 converts the quantized value into a variable length code. Into an encoded stream Vbin.

On the other hand, the quantized value output from the quantizer (Q) unit U4 is inversely quantized by an inverse quantizer (IQ) unit U6 and inverse DCT transform in block units by an inverse discrete cosine transform (IDCT) unit U7. Is performed, the result is added to the motion compensation screen Ref by the adder U8 to form the recording screen FMin of the memory unit FM. It should be noted that a reference screen is not required for encoding an I frame, and in this case, the motion compensation screen Ref treats all pixel values as “0”.

An image sequence number Ob is stored in the memory unit FM.
The jID and the size ObjSZ of each screen are input. The image sequence number ObjID is used to identify the image sequence, and the size ObjSZ of each screen is used to specify the size of the memory area required to record the recording screen FMin.
For example, in FIG. 16, ObjID and ObjSZ of FM1a are ObjID =
1, ObjSZ = SZ1a, ObjID of FM2b, ObjSZ is Obj
It is represented as ID = 2, ObjSZ = SZ2b.

FIG. 18 is a block diagram of a conventional image decoding apparatus. In the figure, a variable length decoding (VLD) unit U11 performs variable length decoding of an encoded stream Vbin, and performs inverse quantization (I
Q) The unit U6 inversely quantizes this, and uses the inverse discrete cosine transform (IDC
T) The unit U7 computes the inverse D
Perform CT conversion. On the other hand, the memory unit FM outputs a reference screen FMout corresponding to the sequence number ObjID of the image, performs motion compensation according to the motion vector MV in the motion compensation (MC) unit U12, and generates a motion compensation screen Ref, The adder U8 adds this to the output of the inverse discrete cosine transform (IDCT) unit U7 to obtain a decoded screen signal Vout. This screen signal Vout is stored in the memory unit FM in the memory area specified by the image sequence ObjID in a recording screen F including image data of the size ObjSZ.
Recorded as Min. Note that a reference screen is not required for decoding an I frame, and in this case, the motion compensation screen Ref treats all pixels as “0”.

FIG. 19 is a block diagram of a conventional image display device. In the figure, the object selector U20 sequentially instructs the memory unit FM as the ObjID of the image sequence to be displayed in accordance with the image sequence information ObjSel to be displayed, which is specified by the user. The image output FMout output by the memory unit FM in response to this command is input to the synthesizer U23 and is synthesized and displayed with another image sequence output from the memory unit FM, or selected by the switch U21. The display is synthesized with the default background screen BG and output to the switch U24. When all the image sequences to be displayed have been read from the memory unit FM, the object selector U20 notifies the switch U24 that the reading of the image sequence to be displayed has been completed with the completion signal ObjEnd, whereby the switch U24 The composite image is output to the display device U25 at a timing synchronized with the synchronization signal Display for display.

In a general configuration, the image decoding device shown in FIG. 18 and the image display device shown in FIG. 19 are often configured as an integrated device. In this case, the image decoding device shown in FIG. Since the image display device shares the memory unit FM, it is often necessary to use only half of the memory when these are configured as individual devices.

FIG. 20 is a block diagram of a conventional memory unit FM. When recording to memory unit FM,
The bank selector M1 specifies the memory bank corresponding to the sequence number ObjID of the image from the divided memory bank, and the address generator M2 determines the address corresponding to the memory area of the size specified by the image size ObjSZ. Generate and memory M4
Records the screen signal FMin in the address space.

Similarly, when reading from the memory unit FM, the memory area corresponding to the sequence number ObjID of the image is specified from the memory banks divided by the bank selector M1, and the size of the image is determined by the address generator M2. An address corresponding to the memory area of the size specified by ObjSZ is generated, and the screen signal FMout is read from the address space of the memory M4. Therefore, at the time of decoding, the read / write operation is performed by alternately using the respective screen areas in two memory banks adjacent to each other as described above by using the access function of the memory unit FM in units of memory banks. , It is possible to decode the image sequence for each object.

[0023]

A conventional memory management method corresponding to an image sequence is configured as described above.
The management can be performed such that a bank corresponding to a memory for a total of two screens required for encoding / decoding of the image sequence is secured in the memory. However, FIG.
In the conventional memory address management method of No. 6, there is a problem that if a time-divided screen signal exceeds the maximum value of each memory, it cannot be recorded. That is, MPEG1 or MPE
In G2, the size of the screen does not change in the middle of the image sequence, but in MPEG4, the size of the screen is allowed to change depending on the time, and the memory where the size of the screen signal to be recorded is divided If the size of the bank is exceeded, there is a possibility that screen data necessary for encoding / decoding / display is lost. To avoid this problem, it is sufficient to secure enough memory banks so that each divided memory bank has the maximum allowable screen size, but this requires an enormous amount of memory capacity. Accordingly, there is a problem that power consumption of a device in which a memory is incorporated increases. Conventionally, since the size of the screen signal does not change depending on the time, no countermeasure has been taken at all when the screen data necessary for decoding / display is lost in the decoding device or the display device. Was.

The present invention has been made in view of the above points, and has been made in consideration of the above circumstances.
An object of the present invention is to provide a memory management method, a memory management device, and a memory management program recording medium that can significantly reduce the possibility of losing screen data required for display. In addition, when screen data necessary for decoding / display is lost, an image decoding method, an image decoding device, an image decoding program recording medium, an image display method, which reduces image quality deterioration,
Image display device, image display program recording medium, image decoding method capable of reducing the amount of calculation, image decoding device, image decoding program recording medium, image encoding method for performing encoding corresponding to the decoding, image It is an object to realize an encoding device and an image encoding program recording medium.

[0025]

According to a first aspect of the present invention, there is provided a memory management method, comprising: a first to an n-th image sequence (n is an integer of 2 or more); Is simultaneously recorded in the memory, and the area in the memory is changed to an address from address ADstart [i] to address ADend [i] (i is an integer satisfying 1 ≦ i ≦ n / 2). i
, The memory address space is divided into first to n / 2th memory address spaces. The region toward either the upper or lower memory address in the memory address space of i is defined as a k-th image sequence (k = i × 2−2).
It is used as an area for recording the image data of 1), and is directed from the other end of ADstart [i] or ADend [i] to either the lower or upper memory address in the ith memory address space. All the areas are used as areas for recording image data of the (k + 1) -th image sequence.

According to a second aspect of the present invention, there is provided a memory management method according to the first aspect,
Whether one image data overwrites the other image data by recording both the image data of the k-th image sequence and the image data of the (k + 1) -th image sequence in the i-th memory address space Is monitored, and when image data is read from the memory, whether or not the image data of the image series to be read has been destroyed by the overwriting is notified to the outside.

According to a third aspect of the present invention, there is provided a memory management method according to the first aspect,
Whether one image data overwrites the other image data by recording both the image data of the k-th image sequence and the image data of the (k + 1) -th image sequence in the i-th memory address space If the image data can be destroyed by the above overwriting, the image data of high importance is overwritten by overwriting to destroy the image data of low importance by referring to the externally input importance information of the image sequence. Is to protect.

The image encoding method according to the invention of claim 4 of the present application is an image encoding method for simultaneously encoding image data of first to n-th image sequences (n is an integer of 2 or more). Thus, when performing inter-picture prediction encoding, predicted image data is recorded in a memory managed by the memory management method according to claim 1.

An image decoding method according to a fifth aspect of the present invention is an image decoding method for simultaneously decoding image data of first to n-th image sequences (n is an integer of 2 or more). Thus, when performing inter prediction decoding, predictive image data is recorded in a memory managed by the memory management method according to claim 1.

An image decoding method according to a sixth aspect of the present invention is an image decoding method for simultaneously decoding image data of first to n-th image sequences (n is an integer of 2 or more). When performing inter-picture prediction decoding, predictive image data is recorded in a memory managed by the memory management method according to claim 2, and image data in the memory required for decoding the image sequence is stored. If the data is destroyed, the decoding process is interrupted until decoding becomes possible without referring to the image data in the memory.

An image decoding method according to a seventh aspect of the present invention is an image decoding method for simultaneously decoding image data of first to n-th image sequences (n is an integer of 2 or more). When performing inter-picture prediction decoding, predictive image data is recorded in a memory managed by the memory management method according to claim 3 with an importance of an important image sequence being increased.
In this method, image data necessary for decoding an important image sequence is hardly destroyed.

An image display method according to the invention of claim 8 of the present application is an image display method for simultaneously displaying image data of first to n-th image series (n is an integer of 2 or more). The image data to be displayed is recorded in a memory managed by the memory management method described in Item 1.

The image display method according to the ninth aspect of the present invention is an image display method for simultaneously displaying image data of first to n-th image series (n is an integer of 2 or more). When the image data in the memory necessary for displaying the image sequence is destroyed, the image data at the latest undestructed time of the image sequence is displayed instead of the destroyed image data. It was done.

A memory management device according to a tenth aspect of the present invention is a memory management device for simultaneously recording image data of first to n-th image sequences (n is an integer of 2 or more) in a memory. By allocating areas in the memory from addresses ADstart [i] to addresses ADend [i] (i is an integer satisfying 1 ≦ i ≦ n / 2) as a ith memory address space, A memory area dividing means for dividing the memory address space into an n / 2th memory address space, and ADstart [i] or ADend of the ith memory address space.
A region from one end of [i] to either the upper or lower memory address in the i-th memory address space is defined as an image of a k-th image sequence (k = i × 2-1). It is used as an area for recording data, and an area from the other end of ADstart [i] or ADend [i] to either the lower or upper memory address in the ith memory address space. , (K +
Address generating means for generating an address to be used as an area for recording image data of the image series of 1).

A memory management program recording medium according to the invention of claim 11 is a memory management program for simultaneously recording image data of first to n-th image sequences (n is an integer of 2 or more). A memory management program recording medium on which a memory management program for executing the method is recorded, wherein an area in the memory is changed from an address ADstart [i] to an address ADend [i] (i is an integer satisfying 1 ≦ i ≦ n / 2). Address up to
i is divided into first to n / 2th memory address spaces by allocating them as memory address spaces. A region toward either the upper or lower memory address in the ith memory address space is defined as a k-th image sequence (k = i × 2).
-1) to be used as an area for recording image data, and from the other end of ADstart [i] or ADend [i] to either the lower or upper memory address in the ith memory address space. The area for
A memory management program for executing a memory management method used as an area for recording image data of the (k + 1) th image sequence is recorded.

In the memory management method according to the twelfth aspect of the present invention, the memory management method according to the twelfth aspect of the present invention provides an empty state or i image data (i
(Integer of 1 to 3) is stored in the memory area where the newly input image data is stored, at least when the image data can be stored at the lowest or highest address of the address of the memory area. Storing the newly input image data at the lowermost or uppermost side of the address of the memory area, and storing the newly input image data at the lowermost side of the address of the memory area;
And a memory management method for storing and managing the memory by storing the newly input image data in an intermediate portion thereof when image data cannot be stored in the uppermost side. When storing the image data, the newly input image data is stored in an area that is in contact with one of the image data stored on the uppermost side and the lowermost side, or the address center of the memory area is The newly input image data is stored in an area adjacent to the position.

The memory management method according to a thirteenth aspect of the present invention is the memory management method according to the twelfth aspect, wherein all of the image data stored in the memory area and the newly input image data are used. When the image data has a rectangular shape, when storing the image data in the intermediate portion, the newly input image is stored in an area which is in contact with one of the image data stored in the uppermost side and the lowermost side. The rearrangement of the memory space position of the already stored image data is performed so as to store the data.

According to a fourteenth aspect of the present invention, in the memory management method according to the thirteenth aspect, the rearrangement of the memory space position of the already stored image data is newly input. This is performed only when the memory for image data is insufficient.

A memory management method according to a fifteenth aspect of the present invention is the memory management method according to the twelfth aspect, wherein the image data stored in the memory area and the newly input image data are stored in the memory area. When at least one is image data of an arbitrary shape, when storing the image data in the intermediate portion, the newly input image data is stored in an area adjacent to an address center position of the memory area, The rearrangement of the memory space position of the already stored image data is performed.

In a memory management method according to a sixteenth aspect of the present invention, in the memory management method according to the fifteenth aspect, the rearrangement of the memory space position of the already stored image data is newly input. This is performed only when the memory for the image data is insufficient.

A memory management method according to a seventeenth aspect of the present invention is the memory management method according to the twelfth aspect, wherein the image data is stored in the uppermost and lowermost sides when the image data is stored in the intermediate portion. When memory data for storing newly input image data is stored in an area in contact with any of the stored image data, when image data of an arbitrary shape is newly input, the image data is stored in the central portion. At this time, the memory space position of the already stored image data is rearranged so that the newly input image data is stored in an area adjacent to the address center position of the memory area.

A memory management method according to an eighteenth aspect of the present invention is the memory management method according to the seventeenth aspect, wherein after the rearrangement of the memory space position accompanying the input of image data of an arbitrary shape, a new input is performed. When storing image data in the central portion only when the memory for the input image data is insufficient, the image data already stored is stored so as to store the newly input image data in an area adjacent to the address central position of the memory area. The rearrangement of the memory space position of the image data is performed.

[0043]

Embodiments of the present invention will be described below with reference to the drawings. (Embodiment 1) In Embodiment 1, in a memory management method, data of two image series is recorded in upper address sides and lower address sides in two memory banks adjacent to each other, respectively. , The use efficiency of the memory bank is improved. FIG. 1 shows an example of an image composed of a plurality of image sequences. Obj
1, Obj2, Obj3, Obj4 are image sequences. As shown in FIG. 1, for example, four image sequences Obj1, Obj2, Obj3, and Obj4 are encoded in image sequence units, and the image display device synthesizes and displays each image sequence and displays it as one screen. Naturally, the size of the screen of each image series increases or decreases depending on the time. So, MP
In EG4, the memory capacity necessary for the reference screen in the inter-picture predictive coding is set by previously determining the “upper limit of the sum” of the screen sizes of all image sequences at an arbitrary time.

This is because the image display is a display of a composite of the respective image sequences. Therefore, even if the overlapping of the objects is taken into account, the sum of the screen sizes of all the image sequences can be compared with the time variation. Set the upper limit of the screen size for each image sequence (for example, the total amount of reference memory required for all image sequences is generally about 1.5 times the display screen) (for example, This is because the required memory capacity can be saved as compared with the case where the size of the display screen is set as the upper limit.

FIG. 2 shows an example in which memory addresses are divided based on the memory management method according to the first embodiment of the present invention. The figure shows a situation where the first image sequence and the second image sequence are recorded in the same memory bank. In FIG. 2, FM1a, FM1b, FM3a, and FM4a denote a first screen area of a first image sequence, a second screen area of a first image sequence, a first screen area of a third image sequence, respectively. It is the 1st screen area of the 4th image series. AD12a, AD12b, AD3
4a and AD34b are respectively a first address start position of the first image sequence, a second address start position of the first image sequence,
A first address start position of the third image sequence, a second address start position of the third image sequence, and AD12a to AD12
b-1 to the first memory bank, AD12b to AD34a-
1 are stored in the memory bank of the 1a, AD34a to AD34b-1.
Up to correspond to the 2a-th memory bank.

The first image sequence includes the start positions AD12a and AD12b.
, The screen sizes SZ1a and SZ1b are secured from the top to the upper address. In the third image sequence, the screen size SZ3a is secured from the start position AD34a toward the upper address.
On the other hand, the second image sequence includes the start positions AD12b-1,
Screen size SZ from AD34a-1 to lower address
2a and SZ2b are secured. In the fourth image sequence, the screen size SZ4a is secured from the start position AD34b-1 toward lower addresses.

Note that the second image sequence and the fourth image sequence actually correspond to the screen sizes SZ2a, SZ2b and SZ4.
Since a is known, each start position AD12b-1, AD34a
Address AD12 obtained by subtracting the values corresponding to these screen sizes SZ2a, SZ2b and SZ4a from -1 and AD34b-1 respectively.
b-SZ2a, AD34a-SZ2b and AD34b-SZ4a are calculated, and the screen size is secured from this to the upper address.

Here, how to store the image data of each image series will be described by taking as an example the case of decoding the frames FR1, FR2,..., FR8 in FIG. These frames
FR1, FR2,..., FR8 constitute one image sequence (first image sequence) corresponding to one object. First, a frame FR1, which is the first I frame in the first image sequence, is decoded, and is decoded into a screen area FM1a.
To be stored. This screen area FM1a is located on the address start position AD12a side of the memory address space 1a, that is, on the lower address side.

Next, for the frame FR2, decoding is performed with reference to the decoded image data of the frame FR1 stored in the screen area FM1a, and the decoding result is stored in the screen area FM1b. This screen area FM1b is located on the address start position AD12b side of the first memory address space, that is, on the lower address side. And then, the frame FR3
, Decoding is performed with reference to the decoded image data of the frame FR2 stored in the screen area FM1b in the first memory address space, and the decoding result is stored in the screen area FM1a in the first memory address space. I do. This is because the image data of the frame FR1 is no longer needed to decode the frame FR3, so that the screen area FM1a may be overwritten. Further, for the frame FR4, decoding is performed with reference to the decoded image data of the frame FR3 stored in the screen area FM1a, and the decoding result is displayed in the screen area FM1a.
Store in FM1b.

Hereinafter, the frame FR belonging to the first image sequence will be described.
Similarly, with respect to 5 to FR8, the first a,
On the lower address side of the 1b memory bank, 1
Decoding is performed using the two screen areas FM1a and FM1b assigned one by one alternately. Note that an I frame or a P frame is referred to for decoding a P frame or a B frame, but a B frame is not referred to for decoding another B frame or a P frame. Is never stored in memory. Therefore, if the frame
If a B frame exists between FR2 and frame FR3, the B frame is decoded with reference to the frame FR2 stored in the screen area FM1b and the frame FR3 stored in the screen area FM1a.

The decoding is similarly performed on the frames belonging to the second and third image sequences. That is,
As for the frames belonging to the second image sequence, two screen areas FM2 each assigned one to the upper address side of the first memory banks 1a and 1b adjacent to each other.
a, FM2b are used alternately, and for frames belonging to the third image sequence, the adjacent 2a, 2b
The two picture areas FM3a, FM3b (one adjacent to the right side of FM4a in FIG. 2 but not shown) are alternately assigned to the lower address side of the memory bank of I do.

Further, when coding such as inter-picture predictive coding or displaying an image, similarly, two adjacent screen areas in the memory are used alternately to perform coding or image display. I do. The features of this memory management method are as follows:-Screens of different image series are recorded in the same memory bank-Screens of one image series are recorded in the lower address side in the same memory bank, and screens of the other image series are recorded Are recorded on the upper address side. Therefore, in the above-described decoding example, the first image sequence is alternately recorded in FM1a and FM1b, and the second image sequence is alternately recorded in FM2a and FM2b. This can use any screen area in the memory bank, for example, alternately recording the first image sequence on FM1a and FM2b, and alternately recording the second image sequence on FM2a and FM1b You may do so.

As described above, in MPEG4, since the upper limit of the sum of the screen sizes of the entire image sequence is determined, the MPEG4
The change in the sum of the sizes of the screens belonging to a plurality of image sequences, which is coded according to the coding rule of 4, is smaller than the change in the screen size of each screen alone. In the example of FIG. 2, the recorded screen will not be damaged by overwriting unless the sum of the screen sizes of the first image series and the second image series exceeds the size of the memory bank. Therefore, even if the screen area FM1a occupies a large area in the bank or the screen area FM2b occupies a large area as shown in FIG. The screen does not break. The secured memory area is released when it is no longer needed (when encoding / decoding completes reference by predictive coding, and when displaying images, when output to the display device is completed), and the next screen is displayed. It is reused as a recording area.

However, if the memory bank is not large enough, even if the sum of the screen sizes of the entire image sequence is less than the upper limit value, the sum of the two image sequences in the image sequence will not be as shown in FIG. It may be large as in b). FIG.
In the example of (b), since a part of the memory area of the screen areas FM1a and FM2a overlaps, it is inevitable that one of the screen data is damaged. In this case, it is possible to reduce image quality deterioration by detecting a damaged screen and performing appropriate image modification described later at the stage of decoding and display.

As described above, according to the memory management method corresponding to a plurality of image sequences according to the first embodiment, the image of the object of the different sequence is stored in the lower address side and the upper address side in each memory bank. Since the data is recorded, when recording the data of two objects, the frequency of destruction of the data of the other object by the data of one object can be significantly reduced, the use efficiency of the memory bank can be increased,
It is possible to obtain a memory management method capable of reducing power consumption when incorporated in a device and greatly reducing the possibility of losing screen data necessary for decoding and display. In addition, the present invention is particularly useful in a device such as a portable terminal which can be equipped with only a small-capacity memory due to power consumption and the like.

(Embodiment 2) Embodiment 2 shows the configuration of a memory management device (memory unit) that implements the memory management method according to Embodiment 1. FIG. 4 is a block diagram of the memory unit of the present invention. In the figure, devices that operate in the same manner as in the block diagram of the conventional memory unit in FIG. 20 are denoted by the same reference numerals, and description thereof will be omitted. In the figure, M10 is a bank selector that specifies a memory bank and notifies the lowest address or the highest address Adr of the memory bank, and corresponds to a memory area dividing unit. M12 is the highest address or the lowest address of the memory bank notified by the bank selector M10
A subtractor, M1, that subtracts between Adr and the screen size ObjSZ
A switch 3 is provided between the bank selector M10 and the address generator M14, and selects one of the address Adr as a memory bank output and the output of the subtractor M12 based on the output LHsel of the bank selector M10. is there. M14 is an address generator, which corresponds to an address generator. M15 is the screen data importance Pri and sequence number ObjID.
Address management device that performs address management based on
This corresponds to a recording monitoring unit, a destruction notification unit, and a data protection unit. M16 is an address generator M14 and memory M1
7, which is turned on or off based on the output NoObj of the address manager M15.
M1 is a bank selector M10, a subtractor M12, and a switch M1.
3, a memory management device including an address generator M14, an address management device M15, and a switch M16; M17, a memory managed by the memory management device M1; Simply referred to as memory).

Next, the operation will be described. First, the operation at the time of recording will be described. The bank selector M10 as a memory area dividing means specifies a memory bank corresponding to the image sequence number ObjID from the divided memory banks. this is,
The entire address space of the memory M17 is changed from ADstart [i] to ADend
[i] (i is an integer that satisfies 1 ≦ i ≦ n) by dividing into memory address spaces corresponding to addresses. Each memory bank has two ObjIDs (ie, two image sequence screens)
Record At this time, the bank selector M10 notifies the lowest address of the memory bank as Adr in the case of a screen to be recorded on the lower address side of the memory bank. The highest address is notified as Adr. In addition, the bank selector M10 is used to select whether the screen is to be recorded on the upper address side or the lower address side in the memory bank.
Specify with LHsel. Note that the memory management device according to the second embodiment of the present invention separately describes the reference image storage memory and the decoded image storage memory,
The bank selector M10 as a memory area dividing means converts the entire address space of the memory M17 from ADstart [i] to ADend [i].
N corresponding to an address up to (i is an integer satisfying 1 ≦ i ≦ n)
However, if a memory management method focusing on either the reference image storage memory or the decoded image storage memory is considered, the memory area division of the memory management device is considered. The bank selector M10 as a means performs ADstart on the entire address space of the memory M17.
[i] to ADend [i] (i is an integer satisfying 1 ≦ i ≦ n / 2)
Is divided into n / 2 memory address spaces corresponding to the addresses up to.

The selection of the memory area of the bank selector M10 operates so as to select a memory area which has been used in the past with the same image sequence number ObjID and is now open. Therefore, when the screen recording position is on the lower address side in the memory bank, the lowest address of the memory bank is input to the address generator M14 by the switch M13, while the screen recording position is on the upper address side in the memory bank. In the case of, the value obtained by subtracting the screen size ObjSZ from the highest address in the memory bank is selected by the switch M13 and input to the address generator M14. Address generator M14 as address generation means
Is the size ObjSZ from the address position selected by switch M13
Is used as an area for recording an image of the i-th image sequence from the screen data ADstart [i] externally input to the memory M17 toward the upper memory address, and ADend [ It is used as an area for recording an image of the (i + 1) -th image sequence from i] to the lower memory address.

Address manager M15 as recording monitoring means
Monitors whether the one destroys the other by recording the image of the i-th image sequence and the image of the (i + 1) -th image sequence in the same address space. This is performed by monitoring the addresses output by the address generator M14 and checking whether or not screens with different image sequence numbers ObjID are written at the same address position. Different image sequence numbers
When the screen of ObjID is to be written to the same address position, the importance Pri of the screen data to be written and the importance Pri of the already written screen data are determined. If the importance Pri of the screen data to be written is higher than the importance Pri of the already written screen data, the address manager M15 as the data protection means switches the switch M16.
To ON to execute writing to the memory M17. Also,
If the importance Pri of the screen data to be written is lower than the importance Pri of the already written screen data, the switch M
Turn OFF 16 to interrupt writing to memory M17.

The importance Pri of the screen data is given in advance by the creator of the content. For example, a low importance is given to the background and the like, and a high importance is given to the image of the object in the screen. As a result, highly important data is stored in the memory M17.
While data is protected by writing to the memory M17, data of low importance is destroyed without being written to the memory M17, so complete decoding and display operations have to be interrupted. Therefore, by performing the decoding operation and the display operation without referring to this, the decoding is performed only by causing the image quality deterioration that does not hinder the practical use, for example, the image sharpness is slightly reduced.・ Display becomes possible. In the case where the screens of different image sequence numbers ObjID are written at the same address position, when the importance Pri is the same or when the importance Pri is not externally given, the switch M16 is turned off in advance. Or,
It is uniquely determined whether to turn ON.

Next, the operation at the time of reading will be described. From the output of the address generator M14 monitored at the time of recording, the address manager M15 instructs the address generator M14 with the position signal StartEnd of the start / end position of the memory address corresponding to the image sequence number ObjID. Address generator M14 is a position signal
A memory address is instructed to the memory M17 according to StartEnd, and desired screen data is output from the memory M17 as FMout. The address manager M15 as a destruction notification means
From the output of the address generator M14 monitored during recording,
It stores whether or not the screen data corresponding to the image sequence number ObjID to be read has been destroyed by data of another image sequence. Therefore, when the screen data specified by the image sequence number ObjID is destroyed, the fact that the image data is destroyed is notified to NoObj. This makes it possible to notify whether or not the accessed image sequence image is destroyed when reading the image data from the memory.

If the address manager M15 has a function of holding the address of the screen data at the latest time that has not been destroyed by the same image sequence number ObjID, Star
By indicating the address of the screen data at the latest time that has not been destroyed as tEnd and turning on the switch M16, it is possible to read from the memory M17 a screen of the same image sequence that is close in time and is preferable as an alternative screen. . NoObj can also notify that an alternative screen is being output.

With the above configuration, the memory management method of the first embodiment can be realized as hardware, and when data of two objects is recorded, data of one of the objects is used to record data of the other object. Can significantly reduce the frequency of overwriting data and destroying it, increase the efficiency of memory bank use, reduce power consumption when embedded in equipment, and save screen data necessary for encoding, decoding, and display. Can greatly reduce the possibility of image loss, and monitor whether image data of one of the image series destroys the other when recording two different image series in the same memory address space. Memory management that can notify the outside if damage has occurred and, if destroyed, destroy less important data and protect more important data Device can be obtained.

Further, the memory management device shown in FIG.
7, the image decoding device of FIG. 18, the image decoding device of FIG.
When encoding, decoding, and displaying an image by replacing the memory FM in the image display device of
When the data of one object is recorded in the memory, the frequency of destruction of the data of the other object by the data of one object can be significantly reduced, the utilization efficiency of the memory bank can be increased, and the use of the conventional memory unit can be improved. Also saves memory space, reduces power consumption when embedded in equipment, greatly reduces the possibility of losing screen data required for encoding, decoding, and display, and also provides two different images. When recording a series of images in the same memory address space, it is possible to monitor whether one will destroy the other, to notify the outside if corruption has occurred, and to indicate the degree of importance when destroying. An image encoding device, an image decoding device, and an image display device that can destroy low data and protect high importance data can be configured.
In addition, the present invention is particularly useful in a device such as a portable terminal which can be equipped with only a small-capacity memory due to power consumption and the like.

(Embodiment 3) Embodiment 3 shows an execution procedure when recording in the memory of the memory management device according to Embodiment 2 is realized by software. FIG. 5 is a flowchart in the case where the recording procedure of the memory unit of the present invention is realized by software. FIG. 5 shows a procedure for recording the screen Obj1 when the screen Obj1 of the image series 1 and the screen Obj2 of the image series 2 are recorded in the same memory bank in the example of dividing the memory address as shown in FIG. First, a memory range FM1a necessary for recording Obj1 is calculated (step W1). Next, it is checked whether the address of FM1a overlaps with the memory area used by Obj2 (step W2). If they overlap,
The importance Pri of Obj1 and Obj2 is compared (step W3), and Obj1 is compared.
If the priority Pri is lower than the priority Pri of Obj2, the process ends without writing anything. Otherwise, the memory of FM2a in which Obj2 is recorded is released and secured as a memory area of Obj1 (step W4). . Finally, the screen data is recorded in the memory area of FM1a (step W5), and the processing ends.

As described above, the recording procedure of the memory unit described in the second embodiment can be realized by software, and when recording data of two objects, data of one object is used to record data of the other object. Can greatly reduce the frequency of destruction of data, increase the efficiency of memory bank use, reduce power consumption when embedded in equipment, and greatly reduce the possibility of losing screen data required for encoding, decoding, and display. It is possible to obtain a memory management method that can be reduced.

(Embodiment 4) Embodiment 4 shows an execution procedure when the read procedure of the memory management device according to Embodiment 2 is realized by software.
FIG. 6 is a flowchart in the case where the read procedure of the memory unit of the present invention is realized by software. FIG. 6 shows a screen Obj1 of the image sequence 1 and a screen Obj2 of the image sequence 2 in the same memory bank in the division example of the memory address in FIG.
It shall be recorded in 1a.

First, the memory area in which the latest Obj1 is recorded is selected from FM1a or FM1b, and this is written in the writing area.
Set FM1 (step R1). Next, it is determined whether or not FM1 is FM1a (step R2). Here, originally FM1a should be selected, but if FM1a is destroyed,
Instead, FM1b will be selected. If FM1 is FM1a, since FM1a has not been destroyed, NoObj is set to “0” and an external notification that FM1a can be read correctly is sent to the outside (step R3). Otherwise, whether FM1 is FM1b is determined. Determine (Step R4), if FM1 is FM1b, set NoObj to “1”
To the outside that FM1a has been destroyed and FM1b has been output instead (step R5). On the other hand, FM1 is FM1a
However, if it is neither FM1b, it notifies the outside that No.Obj is “2” to the effect that the image sequence is completely destroyed (step R6). Finally, FM1 is output as the read-out screen FMout (step R7). Since FM1 is the latest screen that has not been destroyed, even if NoObj = 1, a screen with relatively little deterioration in image quality will be output even if it is used for display instead of the original screen. Note that the value of NoObj is merely an example, and other values may be used or a plurality of signals having the same meaning (for example, whether or not FM1 is the same as FM1a and whether FM1 is output to FMout May be used.

As described above, the read procedure of the memory unit described in the second embodiment can be realized by software, and when data of two objects is recorded, data of one object is used to record data of the other object. Can greatly reduce the frequency of destruction of data, increase the efficiency of memory bank use, reduce power consumption when embedded in equipment, and greatly reduce the possibility of losing screen data required for encoding, decoding, and display. This makes it possible to realize a memory management method that can be reduced to a minimum and a reading method corresponding to the memory management method, and has an effect that information indicating whether or not the read data has been destroyed can also be output. The memory management method (reading procedure) shown in FIG. 6 is combined with the memory management method (writing procedure) shown in FIG.
19, the image encoding device performs image encoding, image decoding, and image display with a reduced amount of memory compared to using a conventional memory unit by directly applying to the memory FM in the image display device of FIG. It is possible to realize an image encoding method, an image decoding method, and an image display method that are capable of being performed.

(Fifth Embodiment) A fifth embodiment is an example in which the configuration of a memory management device (memory unit) according to the second embodiment is simplified. FIG. 7 is a block diagram of the memory unit of the present invention. 4 is different from the block diagram of the memory unit in FIG.
Obj and the switch M16 are omitted in the present embodiment. Memory M17 inside the memory unit
Is large enough to guarantee that the memory area in the memory bank is not destroyed, there is no need to output this because NoObj is always 0. Therefore, in such a situation, the present memory unit from which NoObj is omitted is sufficient.

With the above configuration, the memory management method of the first embodiment can be realized as hardware, and when recording data of two objects, data of one object is used to record data of the other object. Can greatly reduce the frequency of destruction of data, increase the efficiency of memory bank use, reduce power consumption when embedded in equipment, and greatly reduce the possibility of losing screen data required for encoding, decoding, and display. When recording two different image sequences in the same memory address space, it is possible to monitor whether one destroys the other, and if it is destroyed, destroy less important data In addition, a memory management device having a simpler configuration than that of the second embodiment can be obtained, which can protect data having high importance.

The memory management device shown in FIG.
7, the image decoding device of FIG. 18, the image decoding device of FIG.
When the data of two objects is recorded in the memory when encoding, decoding, and displaying an image by directly replacing the memory FM in the image display device of Can significantly reduce the frequency of destruction of data, increase the efficiency of memory bank usage,
Compared to using a conventional memory unit, the amount of memory can be reduced, the power consumption when built into the device can be reduced, and the possibility of losing screen data required for encoding, decoding, and display is greatly increased. While recording images of two different image sequences in the same memory address space, it is possible to monitor whether one destroys the other,
An image encoding device that can destroy less important data and protect more important data when it is destroyed,
A decoding device and an image display device can be configured.

(Embodiment 6) Embodiment 6 shows the configuration of an image decoding apparatus using a memory management apparatus (memory unit) according to Embodiment 5. FIG. 8 is a block diagram of the image decoding apparatus of the present invention. In the figure, devices having the same configuration as those in the block diagram of the image decoding device in FIG. 18 are denoted by the same reference numerals, and description thereof will be omitted. The memory unit FM in the figure is the same as the memory unit in FIG. The importance Pri of the image sequence, which has been set externally in advance, is input to the memory unit FM, and this is stored in the memory unit FM.
Refer to when writing data to the unit FM. That is, when trying to write a screen of a different image sequence number ObjID at the same address position in the memory, the importance Pri of the screen data to be written and the importance Pri of the already written screen data are determined, and the screen to be written is determined. Importance Pri of screen data in which data importance Pri has already been written
If higher, write to memory is performed. If the importance Pri of the screen data to be written is lower than the importance Pri of the already written screen data, the writing to the memory is interrupted. As a result, it is possible to prevent the important image series from being destroyed, and to correctly decode the important image series. In this image decoding apparatus, the importance level Pri
The image sequence having a low image sequence has a high possibility of being destroyed, and the image quality may be degraded by decoding the destructed image.

With the above-described configuration, when the data of two objects is recorded in the memory when the image is displayed for decoding, the frequency of destruction of the data of the other object by the data of one object is greatly increased. Reduces the amount of memory used compared to conventional memory units, reduces power consumption when embedded in equipment, and eliminates screen data required for decoding. In addition, when two different image sequences of images are recorded in the same memory address space, it is possible to monitor whether one destroys the other. An image decoding apparatus capable of destroying low data and protecting high importance data can be configured. When an image sequence with a low priority level Pri is destroyed, the decoding is stopped once, the image decoded before is continuously output, and the decoding is restarted when the next I frame appears. You may. The image decoding apparatus according to the sixth embodiment of the present invention can use a memory unit other than that shown in FIG. 8 as long as it is a memory unit that can use the importance level Pri.

(Embodiment 7) Embodiment 7 shows a simplified structure of a memory management device (memory unit) according to Embodiment 2. FIG. 9 is a block diagram of the memory unit of the present invention. The difference between this figure and the block diagram of the memory unit in FIG. 4 is that the input of the importance Pri from the outside is omitted. If the importance is not assigned to the image sequence in advance, the information of the importance cannot be used. Therefore, in such a situation, a memory unit in which the importance Pri is omitted is sufficient.

FIG. 10 is a block diagram of an image decoding apparatus using the memory unit shown in FIG. In the figure, devices having the same configuration as those in the block diagram of the image decoding device in FIG. The memory unit FM in the figure uses the memory unit in FIG. When NoObj notified from the memory unit FM notifies that the screen to be referred to is destroyed, the switch U10 is turned off to stop decoding the screen. That is, if the reference image is destroyed, it is impossible to perform the necessary decoding correctly based on the reference image, and thus, by turning off the switch U10, the input of the Vbin of the screen is stopped, and the VLD device U11, inverse quantizer U6, IDCT unit U7, adder U8, MC
The operation of the device U12 can be stopped to reduce the amount of calculation and the power consumption.

With the above configuration, the memory management method according to the first embodiment can be realized as hardware, and when data of two objects is recorded, data of one object is used to record data of the other object. Can significantly reduce the frequency of destruction, increase the efficiency of memory bank usage, reduce power consumption when embedded in devices, and greatly reduce the possibility of losing screen data required for encoding, decoding, and display. In addition, when two different image series of images are recorded in the same memory address space, it is possible to monitor whether one destroys the other, and to notify the outside if the corruption occurs. , A memory management device having a simpler configuration than that of the second embodiment can be obtained.

The memory management device shown in FIG.
7, the image decoding device of FIG. 18, the image decoding device of FIG.
When the data of two objects is recorded in the memory when encoding, decoding, and displaying an image by directly replacing the memory FM in the image display device of Can significantly reduce the frequency of destruction of data, increase the efficiency of memory bank usage,
Compared to using a conventional memory unit, the amount of memory can be reduced, the power consumption when built into the device can be reduced, and the possibility of losing screen data required for encoding, decoding, and display is greatly increased. While recording images of two different image sequences in the same memory address space, it is possible to monitor whether one destroys the other,
When a destruction occurs, an image encoding device, an image decoding device, and an image display device having a simpler memory management device capable of notifying the fact to the outside can be configured. In addition,
The image decoding apparatus according to the seventh embodiment can also use a unit other than that shown in FIG. 9 as long as it is a memory unit that outputs NoObj.

(Eighth Embodiment) An eighth embodiment shows a configuration of an image display device using the memory management device (memory unit) according to the seventh embodiment. FIG. 11 is a block diagram of the image display device of the present invention. In the figure, devices having the same configuration as those in the block diagram of the image display device in FIG. 19 are denoted by the same reference numerals. The FM is a memory unit as a memory unit capable of notifying whether or not image data in a memory required for displaying an image sequence has been destroyed. U20
Instructs to display the image as it is if the image data of the image sequence is not destroyed, and if the image data of the image sequence is destroyed, the image data of the latest time in which the image sequence is not destroyed Is an object selector as an object selecting means for instructing to display instead. U25 is a display device as display means for displaying the image data selected by the object selection means.

The memory unit FM shown in FIG. 9 uses the memory unit shown in FIG. The switch U22 is switched according to the value of NoObj output from the memory unit FM. When the memory area is not destroyed and the value of NoObj turns on the switch U22, the image display device operates in the same manner as the conventional one shown in FIG. That is, the object selector U20 sequentially assigns the number of the image sequence to be displayed to ObjI according to the image sequence information ObjSel to be displayed specified by the user.
Command D as memory unit FM. The image output FMout output by the memory unit FM in response to this command is input to the synthesizer U23 and is synthesized and displayed with another image sequence output from the memory unit FM, or selected by the switch U21. The display is synthesized with the default background screen BG and output to the switch U24. When all the image sequences to be displayed have been read from the memory unit FM, the object selector U20 notifies the switch U24 that the reading of the image sequence to be displayed has been completed with the completion signal ObjEnd, whereby the switch U24 Sync signal D for display
The composite image is output to the display device U25 at the timing according to isplay. On the other hand, if NoObj cannot output the screen as FMout, that is, if the memory area is destroyed and the screen of the same image sequence cannot be displayed, synthesizing with the synthesizer U23 will result in synthesizing a stuttered screen. The value of NoObj turns off switch U22.

On the other hand, when NoObj notifies that some screen is to be output to FMout, that is, when the screen is not destroyed or when another screen of the same image series (not destroyed) is read, the above-mentioned operation is performed. By turning on the switch U22 and synthesizing the image, it is possible to obtain a correct synthesized image if it is not destroyed, and a synthesized image with less unnaturalness when reading another screen of the same image series. Can be. In this case, the address manager has a function of holding the address of the screen data at the latest time that has not been destroyed by the same image sequence number. A composite image that can be displayed and has less unnaturalness can be obtained.

With the above-described configuration, when data of two objects is recorded in the memory when displaying an image, the frequency of destruction of data of the other object by data of one object can be greatly reduced. By increasing the efficiency of memory bank utilization, it is possible to reduce the amount of memory compared to using conventional memory units, reduce power consumption when embedded in equipment, and reduce the possibility of losing screen data required for display. When recording two different image sequences in the same memory address space, it is possible to monitor whether one will destroy the other, and notify the outside if it has occurred if it occurs. It is possible to configure an image display device having a simpler memory management device. Note that the image display device according to the eighth embodiment can also use a device other than that shown in FIG. 11 as long as it is a memory unit that outputs NoObj.

Ninth Embodiment A ninth embodiment shows a further simplified configuration of the memory management device (memory unit) according to the second embodiment. FIG.
2 is a block diagram of the memory unit of the present invention. The difference between this figure and the block diagram of the memory unit in FIG.
Input of the importance Pri from the outside is omitted, and the memory M
NoObj notifying that the data read from 17 is destroyed, and switch M16 are omitted in the present embodiment.

With the above configuration, the memory management method of the first embodiment can be realized as hardware, and when recording data of two objects, data of one object is used to record data of the other object. Can significantly reduce the frequency of destruction, increase the efficiency of memory bank usage, reduce power consumption when embedded in devices, and greatly reduce the possibility of losing screen data required for encoding, decoding, and display. And a memory management device with a simpler configuration than in the fifth and seventh embodiments can be obtained.

The memory management device shown in FIG. 12 includes the image encoding device shown in FIG. 17, the image decoding device shown in FIG.
9 when the data of two objects is recorded in the memory when encoding, decoding, and displaying an image by directly replacing the memory FM in the image display device of No. 9 with the data of one object. Dramatically reduce the frequency of destruction of object data, increase the efficiency of memory bank utilization, save memory space compared to using conventional memory units, and reduce power consumption when embedded in equipment. , An image encoding device, an image decoding device, and an image display device having a memory management device that can greatly reduce the possibility of losing screen data necessary for encoding, decoding, and display, and have a simpler configuration. can do.

(Embodiment 10) Embodiment 10 has the configuration of Embodiments 6 and 7. FIG. 13 is a block diagram of an image decoding apparatus according to Embodiment 10 of the present invention. In the figure, devices having the same configuration as those in the block diagram of the image decoding device in FIG. Memory unit in the figure
FM uses the memory unit of FIG. Therefore, since both the importance levels Pri and NoObj are provided, the advantages of the respective image decoding apparatuses shown in FIGS. 8 and 10 are obtained. It should be noted that the image decoding apparatus according to the tenth embodiment uses the importance Pri and NoObj
It is also possible to use a memory unit other than that shown in FIG.

(Embodiment 11) A data management apparatus according to Embodiment 11 of the present invention will be described below with reference to FIGS. In the first to tenth embodiments, the memory for storing the decoded image and the memory for storing the reference image are separately described as shown in FM1a and FM1b in FIG. Since the usage of the reference memory and the use of the reference image storage memory are completely the same, in the following description, the decoded image storage memory and the reference image storage memory are not divided, and the image storage memory is simply used. Will be described as a management method. MPEG1 and MPEG
In 2, the size of the image data (hereinafter referred to as an object) does not change in the middle of the image sequence,
In No. 4, the size of the object is allowed to change in the middle of the image sequence. Therefore, in the eleventh embodiment of the present invention, when the data stored in the image storage memory is only the object data whose size does not change (hereinafter, referred to as a simple profile),
A case in which the memory management method of the image storage memory is changed and the memory is managed when at least one object data having a variable size is included (hereinafter, referred to as a core profile) will be described.

FIG. 21 is a diagram showing an example of a memory management method at the time of a simple profile (hereinafter, referred to as a simple profile mode), and FIG.
FIG. 9 is a diagram illustrating an example of a memory management method at the time of profiling (hereinafter, referred to as a core profile mode). In the eleventh embodiment of the present invention, a case will be described in which the number of image sequences constituting an image is at most four. As shown in FIGS. 21 and 22, when the number of image sequences constituting an image is a maximum of four, securing of the memory address space depends on the number of image sequences,
The uppermost position of the memory and the central position of the memory (the upper end in FIGS. 21 and 22 correspond to the lowermost position of the memory address) in this order, and the object data of each image sequence is stored in the reserved memory space. Store. That is, in the simple profile mode, the memory space is secured when the number of objects is 1 as shown in FIG.
In this case, the lowest position of the memory (FIG. 21A)
When the number of objects is two, a memory space is secured at the lowest position of the memory and at the highest position (FIG. 21B). When the number of objects is three, the memory space is located at the lowest position of the memory, at the highest position, and at a position adjacent to the object data stored at the lowest position of the memory (the end position of the memory) (FIG. 21C). When the number of objects is four, a memory space is further secured at a position (FIG. 21D) adjacent to the object data stored at the uppermost position of the memory which is the other end of the memory.

By performing such memory management,
It is possible to increase the efficiency of use of the memory area when only the object of the same size is handled, and it is possible to greatly reduce the possibility that decoding becomes impossible due to lack of memory required for decoding. As shown in FIG. 22, when the number of objects is one, the memory space in the core file mode is secured at the lowest position in the memory (FIG. 22).
In (a)), when the number of objects is two, a memory space is secured at the lowest position of the memory and at the highest position (FIG. 22B). When the number of objects is three, a memory space is secured in the lowest direction from the lowest position of the memory, the highest position, and the center position of the memory where the image memory is divided into two (FIG. 22C). When the number of objects is four, the memory space is set so that one end of the object data is adjacent to the memory center position where the image memory is divided into two from the memory center position and the memory top position, respectively. Is secured (FIG. 22D).

By performing such memory management,
When at least one object whose size changes is included, a memory area can be dynamically secured even if the size of each object slightly changes. The possibility that decoding becomes impossible can be greatly reduced. In general, considering the complexity of processing when the size of an object changes, the total amount of memory required for decoding each object in the core profile is larger than the total amount of memory required in the simple profile. Often set smaller. Accordingly, in the case of the core profile, even if the memory is not secured without a gap as shown in FIG. 21, it is practically sufficient to make the sum of the memory amounts of the two objects constant as shown in FIG. It is.

These are examples of ideal memory arrangement positions in the simple profile mode and the core profile mode, and a memory space is secured so that the arrangement is as possible as possible. Although the ideal arrangement may be different at the time of object disappearance, when allocating memory for a new object, FIG.
1. A memory for a new object is secured at a location having a memory arrangement close to that of FIG. 21 and FIG.
In FIG. 2, Obj1, Obj2, Obj3, and Obj4 do not indicate respective object IDs, but simply indicate memory locations. That is, the object IDs and the memory locations, and the numbers of Obj1, Obj2, Obj3, and Obj4 do not have a fixed correspondence. Also, in FIG. 22, the arrow indicates the dynamic memory expansion direction, and is conceptually the same as the memory management method described in the first embodiment, and thus the description is omitted here.

Next, switching processing from the simple profile mode to the core profile mode in the memory management method according to the eleventh embodiment of the present invention will be described with reference to FIG.
This will be described with reference to FIG. FIG. 23 shows Embodiment 11 of the present invention.
9 is a flowchart for explaining a process of switching from the simple profile mode to the core profile mode in the memory management method according to the first embodiment. The memory management method according to the eleventh embodiment of the present invention operates in the simple profile mode first, and the image sequence to be processed includes an image sequence of an arbitrary shape in which the size of an object changes. When a newly detected object of any shape, which is an object whose size has changed, included in an image sequence of any shape whose object size changes, the core profile is switched from the simple profile mode to the core profile. -Mode switching processing is performed. If the image sequence to be processed does not include an image sequence in which the size of the object changes,
Switching from the simple profile mode to the core profile mode is not performed, and the memory area is secured in the simple profile mode. In FIG. 23, when an object having an arbitrary shape is newly input, first, it is determined whether or not data is stored at the position Obj3 shown in FIG. 21 (S101).
If the data is stored, the data is moved to the memory center position (see FIGS. 22C and 22D) (S10).
2).

Next, it is determined whether or not data is stored at the position Obj4 shown in FIG. 21 (S103). If data is stored, the data is moved to the central position of the memory (FIG. 22 (c)). ) (D)) (S104). As a result, the allocation of the memory address space becomes the allocation of the core and the profile mode (see FIG. 22), so that memory for the subsequent object data can be secured in the core profile mode. In general, it is difficult to know in advance whether a stream is a simple profile or a core profile, and a simple profile requires a larger total amount of memory for decoding each object than a core profile. large. Therefore, until an object of an arbitrary shape is input, it operates in the simple profile mode requiring a larger amount of memory, and secures a larger amount of memory required by the simple profile. Also, by inputting an object of an arbitrary shape, the memory usage method is changed from the simple profile mode to the core profile mode, so that the sum of the memory amounts of the two objects is within the upper limit. It is freely changeable and can use enough memory area for decoding with little relocation of memory locations.

When relocating the memory,
Since the decoding process is interrupted, the decoding process for data input during this time is interrupted if there is not sufficient decoding processing capability. Therefore, in that case, the decoding process is suspended until the next I frame data is input, and the next I frame data is input.
The securing of the memory address space may be resumed from the frame reception in the core profile mode. After switching to the core profile mode, the mode is not switched to the simple profile mode on the way. This is because when an image sequence to be processed includes an image sequence in which the size of an object changes, there is a high possibility that an object of an arbitrary shape will be input thereafter, and a simple profile is used in the core profile. This is because it does not require a large amount of memory. In addition, simple profile mode and core profile mode
When the mode is switched, the possibility that the decoding process is interrupted is high, and the disadvantage that the decoding is temporarily interrupted is greater than the advantage that the amount of memory that can be temporarily used is increased by the mode switching. Note that if there is sufficient decoding capability and memory relocation capability, switching from the core profile mode to the simple profile mode may be performed.

Next, a simple profile mode, which is one of the memory storage methods according to the eleventh embodiment of the present invention, will be described with reference to FIG. 21 and FIGS. It should be noted that the eleventh embodiment of the present invention
The memory space allocation in the profile mode is intended to increase the efficiency of memory usage in the simple profile mode and to reduce the number of times of performing memory relocation processing with interruption of decoding processing.
This will be described with reference to FIG. FIG. 24 is a flowchart for explaining an example of a procedure for securing a memory address space in the simple profile mode, and is started when new rectangular object data whose size does not change is input. When new rectangular object data is input, first, it is determined whether or not data input in the Obj1 position can be stored in the open area (S201), and data input in the Obj1 position can be stored. In this case, the newly input data is stored in the Obj1 position (S202).

On the other hand, if the Obj1 position is not in the open area or the input data cannot be stored, it is determined whether the Obj2 position can store the input data in the open area (S203). If the input data can be stored in the open area, the newly input data is stored in the Obj2 position (S204). If the Obj2 position is not in the open area or the input data cannot be stored, it is determined whether the Obj3 position can store the input data in the open area (S205), and the Obj3 position is not in the open area. If you can store the entered data,
The newly input data is stored in the Obj3 position (S20
6). If the Obj3 position is not in the open area or the input data cannot be stored, it is determined whether the Obj4 position can store the input data in the open area (S207), and the Obj4 position is in the open area. If the input data can be stored, the newly input data is stored in the Obj4 position (S208). On the other hand, if the position of Obj4 is not a blank area or if the input data cannot be stored, the current memory layout indicates that Obj1, Obj2, Obj
3, the newly input object data cannot be stored in any position of Obj4. Therefore, the memory address space is rearranged, and a process for securing a memory address space for storing newly input object data is performed.

Since the memory area to be reserved is reserved so as to store the object data of all the image series, newly input data is stored in any of the positions Obj1, Obj2, Obj3, and Obj4. The fact that storage is not possible means that an empty area exists somewhere between the Obj positions. Therefore, the rearrangement of the memory address space may be performed so as to fill the empty area. That is, first, it is checked whether or not data has been stored at the Obj1 position (S209). If data has not been stored at the Obj1 position, the data at the Obj2, Obj3, and Obj4 positions shown in FIG. , And the newly input data is stored in the Obj1 position (S210). The details of the relocation of the data at the positions Obj2, Obj3, and Obj4 in step S210 will be described later with reference to FIG.

On the other hand, if data has been stored at the Obj1 position, it is checked whether data has been stored at the Obj2 position (S211). If data has not been stored at the Obj2 position, Obj3 shown in FIG. After rearranging the data at the Obj4 position, the newly input data is stored at the Obj2 position (S212). Obj in step S212
Details of the rearrangement of data at the position 3 and the position Obj4 will be described later with reference to FIG. Obj2
If data has been stored at the position, it is checked whether data has been stored at the Obj3 position (S213).
If data has been stored in the location,
After rearranging the data at the j3 position, the newly input data is stored at the Obj4 position (S214). The details of the relocation of the data at the position Obj3 in step S214 will be described later with reference to FIG. On the other hand, if data has not been stored in the Obj3 position, FIG.
After rearranging the data at the position Obj4 shown in FIG.
The newly input data is stored in the position (S215).
The details of the relocation of the data at the position Obj4 in step S215 will be described later with reference to FIG.

As described above, first, it is determined whether or not the memory for the new object data can be secured. By performing the rearrangement, it is possible to reduce the rearrangement process of the memory in which the decoding process is likely to be interrupted, and it is possible to increase the use efficiency of the memory.
The possibility that decoding becomes impossible due to lack of memory required for decoding can be greatly reduced, and a memory management method with high practical value can be realized.

Next, Obj2, Obj3, Obj4 in the simple profile mode in step S210 described above.
The process of relocating the position data will be described with reference to FIG. Figure 25 shows Ob in simple profile mode.
It is a flowchart for demonstrating the rearrangement of the data of j2, Obj3, Obj4 position. First, it is checked whether the Obj2 position is empty (S301), and then, it is checked whether the Obj4 position is empty (S302, S3).
03). At this time, if both the Obj2 and Obj4 positions are not empty, that is, if data is stored in both the Obj2 and Obj4 positions, the data stored in the Obj4 position shown in FIG. 21 is stored in the Obj2 position. The data is moved to a position adjacent to the data (S304). If the Obj2 position is empty and the Obj4 position is not empty, the data stored in the Obj4 position is moved to the Obj2 position (S305). Next, it is checked whether the Obj3 position is empty or not (S306, S307). When data is stored in the crab, Ob shown in FIG.
The data stored at the j3 position is moved to a position adjacent to the new data stored at the Obj1 position (S308).

On the other hand, when data is stored in the Obj3 position and no data is stored in any of the Obj2 and Obj4 positions, the data stored in the Obj3 position is moved to the Obj2 position. (S309).
If data larger than the amount of memory that can be processed by the decoder is input, in this step S308, Ob
When the data stored at the j3 position is moved to a position adjacent to the new data stored at the Obj1 position, the Obj2 position,
Alternatively, the data stored at the position Obj3 is duplicated with the data stored at the position Obj4. In such a case, the data stored at the position Obj3 is not moved.

Next, it is checked whether or not the newly input data can be stored in the Obj1 position (S310). If the newly input data can be stored in the Obj1 position, the newly input data is stored in the Obj1 position. It is stored (S311). On the other hand, if the newly input data cannot be stored in the Obj1 position, it is determined that data larger than the memory amount that can be processed by the decoder has been input, and the decoding impossible processing is performed (S31).
2). In the non-decoding process, the decoding process is interrupted for the newly input data object until the next I-frame data is input, and the memory address space is newly secured from the next I-frame reception. This is the process to be performed.

Next, the process of relocating the data at the positions Obj3 and Obj4 in the simple profile mode in step S212 will be described with reference to FIG. FIG. 26 shows Obj3 and Obj in the simple profile mode.
It is a flowchart for demonstrating the rearrangement of data of four positions. First, it is checked whether or not the Obj3 position is empty (S401). If the Obj3 position is not an empty area, that is, if data is stored in the Obj3 position, as shown in FIG. Store the stored data in Obj
The data is moved to a position adjacent to the new data stored at one position (S402). Next, it is checked whether or not the Obj4 position is empty (S403). If the Obj4 position is not an empty area, that is, if data is stored in the Obj4 position, as shown in FIG. Is moved to a position adjacent to the new data to be stored at the Obj2 position (S404). If data larger than the memory amount that can be processed by the decoder is input, the data stored at the position Obj4 is moved to a position adjacent to the new data stored at the position Obj2 in this step S404. Since the data stored in the Obj1 position or the Obj3 position is duplicated, in such a case, the data stored in the Obj4 position is not moved.

Next, it is checked whether or not the newly input data can be stored in the Obj2 position (S405). If the newly input data can be stored in the Obj2 position, the newly input data is stored in the Obj2 position. It is stored (S406). On the other hand, if the newly input data cannot be stored in the Obj2 position, it is determined that data larger than the memory amount that can be processed by the decoder has been input, and the decoding impossible processing is performed (S40).
7). In the non-decoding process, the decoding process is interrupted for the newly input data object until the next I-frame data is input, and the memory address space is newly secured from the next I-frame reception. This is the process to be performed.

Next, the process of relocating the data at the position Obj3 in the simple profile mode in step S214 described above will be described with reference to FIG. FIG.
9 is a flowchart for explaining relocation of data at the position Obj3 in the simple profile mode.
This is a process that occurs when data is stored in all of the positions Obj1, Obj2, and Obj3, and the newly input data cannot be stored in the free space at the position Obj4. First, the data stored at the position Obj3 shown in FIG. 21 is moved to a position adjacent to the data stored at the position Obj1 (S501). Next, it is checked whether the newly input data can be stored in the Obj4 position (S50).
2) If the newly input data can be stored in the Obj4 position, the newly input data is stored in the Obj4 position (S503). On the other hand, if the newly input data cannot be stored in the Obj4 position, it is determined that data larger than the amount of memory that can be processed by the decoder has been input, and non-decoding processing is performed (S504). Note that this non-decryptable process is performed on the newly input data object in the next I
In this process, the decoding process is interrupted until frame data is input, and the memory address space is newly secured from the time of receiving the next I frame.

Next, the process of relocating the data at the Obj4 position in the simple profile mode in step S215 will be described with reference to FIG. FIG.
FIG. 7 is a flowchart for explaining relocation of data at the Obj4 position in the simple profile mode.
This is a process that occurs when data is stored at the positions Obj1 and Obj2 and the newly input data cannot be stored at the positions Obj3 and Obj4. First, it is checked whether or not the Obj4 position is empty (S601).
If no data is stored at the position, it is determined that data larger than the memory amount that can be processed by the decoder has been input, and the process proceeds to step S605. On the other hand, if the Obj4 position is not a free area, that is, if data is stored in the Obj4 position, the data stored in the Obj4 position shown in FIG. 21 is moved to a position adjacent to the data stored in the Obj2 position. (S602).

Next, it is checked whether the newly input data can be stored in the Obj3 position (S603). If the newly input data can be stored in the Obj3 position, the newly input data is stored in the Obj3 position. It is stored (S604). On the other hand, if the newly input data cannot be stored in the Obj3 position, it is determined that data larger than the memory amount that can be processed by the decoder has been input, and the process proceeds to step S605. When data larger than the amount of memory that can be processed by the decoder is input, a decoding impossible process is performed (S605). In the non-decoding process, the decoding process is interrupted for the newly input data object until the next I-frame data is input, and the memory address space is newly secured from the next I-frame reception. This is the process to be performed.

Next, a core profile mode which is one of the storage methods according to the eleventh embodiment of the present invention will be described with reference to FIGS. 22, 29 to 31. FIG. It should be noted that securing the memory space in the core profile mode according to the eleventh embodiment of the present invention increases the efficiency of use of the memory in the core profile mode, and increases the possibility that the decoding process is likely to be interrupted. This is for reducing the number of times the rearrangement process is performed, and the details will be described with reference to FIG. FIG. 29 is a flowchart for explaining an example of a memory address space securing procedure in the core profile mode, and is started by inputting new object data after switching to the core profile mode. I do. When the new object data is input after switching to the core profile mode, first, it is determined whether or not the data input in the open area of the Obj1 position can be stored (S7).
01), if the data input in the Obj1 position can be stored in the open area, the newly input data is stored in the Obj1 position (S702).

On the other hand, if the Obj1 position is not in the open area or the input data cannot be stored, it is determined whether the Obj2 position can store the input data in the open area (S703). If the input data can be stored in the open area, the newly input data is stored at the position Obj2 (S704). If the Obj2 position is not in the open area or the input data cannot be stored, it is determined whether the Obj3 position can store the input data in the open area (S705), and the Obj3 position is not in the open area. If you can store the entered data,
The newly input data is stored in the Obj3 position (S70).
6). If the Obj3 position is not in the open area or the input data cannot be stored, it is determined whether the Obj4 position can store the input data in the open area (S707), and the Obj4 position is stored in the open area. If the input data can be stored, the newly input data is stored in the Obj4 position (S708). On the other hand, if the position of Obj4 is not a blank area or if the input data cannot be stored, the current memory layout indicates that Obj1, Obj2, Obj
3, the newly input object data cannot be stored in any position of Obj4. Therefore, the memory address space is rearranged, and a process for securing a memory address space for storing newly input object data is performed.

Since the memory area to be reserved in advance is reserved so as to store object data of all the image series constituting the image, Obj1, Obj2, Obj3, O
The fact that newly input data cannot be stored in any position of bj4 means that the number of objects using the memory is one, and the object using the memory is in FIG. d) Obj 3 position or O
This means that it exists at the bj4 position. Therefore, reallocation of the memory address space is
The object at the position j3 or Obj4 may be moved to the Obj1 position or the Obj2 position shown in FIG. This is because, in the memory management method of the present invention, as described above, the memory area to be secured in advance is secured so as to store the object data of all the image series constituting the image. Has a restriction that the sum of the sizes of the two objects does not exceed the total memory capacity. In the core profile mode, when the number of objects is three or four, As described with reference to FIGS. 22 (c) and (d), since the memory is divided into two and used, it is needless to say that the size of one object cannot exceed 1/2 of the total memory capacity. This is because it is assumed that the sum of the sizes of the two objects cannot exceed 1/2 of the total memory. Therefore, first, Ob
It is checked whether data has been stored in the j1 position and the Obj2 position (S701 and S703). If no data has been stored in any of the Obj1 position and the Obj2 position, Ob shown in FIG.
After rearranging the data at the j3 and Obj4 positions, the newly input data is stored at the Obj1 position (S711). The details of step S711 will be described later with reference to FIG.
This will be described with reference to FIG.

Further, the fact that data has already been stored in either the Obj1 position or the Obj2 position and that newly input data cannot be stored means that newly input data is inconsistent with the above-described restrictions. Because the data is normal,
The newly input data cannot be stored in the memory, and it is necessary to perform a non-decodable process (S712).
In the non-decoding process, the decoding process is interrupted for the newly input data object until the next I-frame data is input, and the memory address space is newly secured from the next I-frame reception. This is the process to be performed. As described above, first, it is determined whether or not the memory for the new object data can be secured. If the memory for the new object data cannot be secured, the reallocation of the memory space position of the existing object is performed only when the memory for the new object data cannot be secured. By doing so, it is possible to reduce the amount of memory relocation processing that is likely to interrupt the decoding processing, increase the memory utilization efficiency, and make decoding impossible due to lack of memory necessary for decoding. It is possible to greatly reduce the possibility of the memory management, and to realize a memory management method with high practical value.

Next, the process of relocating the data at the positions Obj3 and Obj4 in the core profile mode in step S711 will be described with reference to FIG. FIG.
FIG. 9 is a flowchart for explaining relocation of data at the positions Obj3 and Obj4 in the core profile mode. First, it is checked whether or not the Obj4 position is empty (S801), and the Obj4 position is not an empty area, that is, Obj4.
If the data is stored at the four positions, the data stored at the Obj4 position shown in FIG. 22 is moved to the Obj2 position (S802). On the other hand, if the Obj4 position is empty, it is checked whether the Obj3 position is empty (S803), and the Obj3 position is not an empty area, that is, Obj
If data is stored at the three positions, the data stored at the Obj3 position shown in FIG. 22 is moved to the Obj2 position (S804), and the process goes to step S805. Also, Obj
If both the positions 3 and Obj4 are free areas, it is determined that data larger than the amount of memory that can be processed by the decoder has been input, and the flow advances to step S807.

In step S805, it is checked whether or not the newly input data can be stored at the position Obj1.
Obj if the location can store the newly entered data
The newly input data is stored at one position (S80).
6). On the other hand, if the newly input data cannot be stored in the Obj1 position, it is determined that data larger than the memory amount that can be processed by the decoder has been input, and the process proceeds to step S807. If data larger than the amount of memory that can be processed by the decoder is input, a decoding impossible process is performed (S807).
In the non-decoding process, the decoding process is interrupted for the newly input data object until the next I-frame data is input, and the memory address space is newly secured from the next I-frame reception. This is the process to be performed.

Next, in the core profile mode, a description will be given of memory relocation processing accompanying a change in the size of object data, with reference to FIGS. 22, 31 and 32. FIG. In core profile mode,
For example, in the case where the memory arrangement shown in FIG. 31A with three objects is taken and the object data at the position Obj1 is lost,
A memory area is secured in the direction of the arrow in FIG. 31B from the position 2 and the position Obj3. However, the core
In the profile mode, when the number of objects to be stored in the memory is reduced to two or less, it is not necessary to use the image storage memory in two parts, and the size of one object or two objects is reduced. Since the sum of the sizes is not required to exceed the total memory capacity, the allowable value of the maximum number of macroblocks per object increases from the point in time when the number of objects stored in the memory is reduced to two or less. Therefore, the size of the input data of the object stored in the memory is one size of the image storage memory.
In the case of / 2 or more, the required memory capacity cannot be secured in the memory arrangement shown in FIG. That is, if the number of objects is reduced from three or more to two or less and the size of the input data of the object is increased, a situation occurs in which the required memory capacity cannot be secured with the existing memory arrangement. In order to secure a large memory capacity, FIG.
The memory is rearranged so as to have the memory arrangement shown in FIG. The memory relocation described below with reference to FIG. 32 is different from the memory relocation necessary only when a new object is generated, unlike the above-described memory relocation required only when a new object is generated. Arrangement.

FIG. 32 is a flowchart for explaining an operation performed every time data of an object is input in the memory management method in the core profile mode. This includes memory relocation processing. First, when the object data is input, it is determined whether a necessary memory area can be secured (S901). If the necessary memory area can be secured, the process proceeds to step S914.
On the other hand, if the necessary memory area cannot be secured, it is checked whether the number of objects is two or less (S90).
2) If the number of objects is not two or less, it is determined that invalid data that cannot be decoded is input, and the process proceeds to step S915. On the other hand, if the number of objects is two or less, it is checked whether data is stored at the Obj1 position (S903). At this time,
If no data is stored in the Obj1 position, it is further checked whether data is stored in the Obj3 position (S904). If data is stored in the Obj3 position, the data is stored in the Obj3 position. The obtained data is moved to the Obj1 position (S905). On the other hand, if data is not stored at the Obj3 position, it is further checked whether data is stored at the Obj4 position (S906).
If data is stored at the j4 position, the data stored at the Obj4 position is moved to the Obj1 position (S9).
07).

Next, it is checked whether data is stored at the position Obj2 (S908). At this time, if no data is stored in the Obj2 position, the Obj3
It is checked whether data is stored at the position (S90).
9) If the data is stored in the Obj3 position, the data stored in the Obj3 position is moved to the Obj2 position (S910). On the other hand, if the data is not stored at the Obj3 position, it is further checked whether data is stored at the Obj4 position (S911). If the data is stored at the Obj4 position, Is moved to the Obj2 position (S91).
2). Next, it is determined whether a necessary memory area can be secured (S913). If the necessary memory area can be secured, the process proceeds to step S914. On the other hand, if the necessary memory area cannot be secured, it is determined that invalid data that cannot be decrypted has been input, and the process proceeds to step S915. If the necessary memory area can be secured, the object data is stored in the memory area (S914).
On the other hand, if the necessary memory area cannot be secured, it is determined that invalid data that cannot be decoded is input, and the decoding is disabled (S915).

In the non-decoding process, the decoding process is suspended for the newly input data object until the next I frame data is input, and the memory address space is renewed from the time of receiving the next I frame. This is the process for securing As described above, it is determined whether or not a memory for storing the data of the object can be secured, and only when the memory cannot be secured,
By relocating the memory space position of the existing object, it is possible to reduce the relocation processing of the memory in which the decoding processing is likely to be interrupted, thereby improving the memory use efficiency and decoding. The possibility that decoding becomes impossible due to a shortage of memory required can be greatly reduced, and a memory management method of high practical value can be realized. In the eleventh embodiment of the present invention, the non-decodable process is described as decoding the newly input data object from the next I frame data. However, the present invention is not limited to this. It may be configured not to decode all the data obtained at all.

(Twelfth Embodiment) A twelfth embodiment is a memory management method according to the first to eleventh embodiments described above.
A program for realizing the image encoding method, the image decoding method, and the image display method is stored in a floppy (registered trademark).
By recording on a recording medium such as a disk, the processing described in each of the above embodiments can be easily performed by an independent computer system.

FIG. 14 shows a memory management method, an image encoding method, an image decoding method, and an image display method according to the first to eleventh embodiments. FIG. 11 is an explanatory diagram of a case where the present invention is implemented by a computer system using a floppy disk storing a decoding program and an image display program. FIG. 14B shows the appearance, cross-sectional structure, and floppy disk of the floppy disk as viewed from the front, and FIG. 14A shows an example of the physical format of the floppy disk which is the main body of the recording medium. The floppy disk FD is built in the case F,
A plurality of tracks Tr are formed concentrically from the outer circumference to the inner circumference on the surface of the disk, and each track is divided into 16 sectors Se in the angular direction. Therefore, in the floppy disk storing the program, at least one of the memory management program, the image encoding program, the image decoding program, and the image display program is recorded in an area allocated on the floppy disk FD. FIG. 14C shows a configuration for recording and reproducing the program on the floppy disk FD. When the above program is recorded on the floppy disk FD, the computer system Cs
Then, the memory management method, the image encoding method, the image decoding method, and the image display method as the above programs are written via a floppy disk drive. When the encoding or decoding device is constructed in a computer system by a program in a floppy disk,
The program is read from the floppy disk by the floppy disk drive and transferred to the computer system.

As a result, the memory management program,
It is possible to obtain a program recording medium that allows an independent computer system to execute at least one of the image encoding program, the image decoding program, and the image display program. In the above description, the description has been made using the floppy disk as the recording medium.
Larger capacity magnetic recording media such as DD and removable disks, CD-ROM, CD-R, CD-RW, PD,
The present invention can be similarly implemented using an optical disk such as a DVD or MO. Further, the recording medium is not limited to these, but can be similarly implemented as long as the program can be recorded, such as various IC cards (memory cards).

[0121]

As described above, according to the memory management method according to the first aspect of the present invention, the image data of the first to n-th image sequences (n is an integer of 2 or more) are simultaneously recorded in the memory. In this method, the area in the memory is changed from the address ADstart [i] to the address ADend [i] (where i is 1 ≦ i ≦ n
(Integer satisfying / 2) is divided into first to n / 2th memory address spaces by assigning addresses as the i-th memory address space, and ADstart [i] or ADend of the i-th memory address space. [i] is defined as an image data of a k-th image sequence (k = i × 2-1) from an end toward either the upper or lower memory address of the i-th memory address space. Used as an area for recording
In the area from the other end of start [i] or ADend [i] to either the lower or upper memory address of the ith memory address space, the image data of the (k + 1) th image sequence is recorded. Used as an area for
As a result, it is possible to realize a memory management method with high practical value, which can increase the use efficiency of the memory area and greatly reduce the possibility of losing screen data necessary for encoding / decoding / display. effective.

According to the memory management method of the second aspect of the present invention, in the memory management method of the first aspect, the image data of the k-th image sequence and the (k +
By recording the image data of the image series of 1) together in the i-th memory address space, it is monitored whether or not one image data overwrites the other image data. It notifies the outside whether the image data of the image sequence to be read has been destroyed by the above overwriting, so that the use efficiency of the memory area is improved, and the screen data necessary for encoding / decoding / display is reduced. This has the effect of significantly reducing the possibility of loss and realizing a highly practical memory management method capable of notifying the outside whether or not the accessed image sequence image has been destroyed.

According to the memory management method of the third aspect of the present invention, in the memory management method of the first aspect,
Whether one image data overwrites the other image data by recording both the image data of the k-th image sequence and the image data of the (k + 1) -th image sequence in the i-th memory address space If the overwriting can cause destruction of the image data, the externally input image sequence importance information is referenced, and the overwriting is performed to destroy the less important image data and the higher importance image data is destroyed. , So that the use efficiency of the memory area can be increased, and the possibility that screen data necessary for encoding / decoding / display is lost can be greatly reduced. In addition, there is an effect that a memory management method that can reduce image quality deterioration and has a high practical value can be realized.

Further, according to the image coding method according to the invention of claim 4 of the present application, the first to n-th image sequences (n is 2
2. An image encoding method for simultaneously encoding image data of the above (integer), wherein when performing inter-screen prediction encoding, predictive image data is recorded in a memory managed by the memory management method according to claim 1. As a result, it is possible to increase the efficiency of use of the memory area and greatly reduce the possibility of losing screen data required for encoding. is there.

Further, according to the image decoding method according to the invention of claim 5 of the present application, the first to n-th image sequences (n is 2
An image decoding method for simultaneously decoding image data of the above (integer), wherein when performing inter-screen prediction decoding, predictive image data is recorded in a memory managed by the memory management method according to claim 1. As a result, it is possible to increase the efficiency of use of the memory area and greatly reduce the possibility of losing the screen data required for decoding. is there.

Further, according to the image decoding method according to the invention of claim 6 of the present application, the first to n-th image sequences (n is 2
An image decoding method for simultaneously decoding image data of the above (integer), wherein when performing inter-screen predictive decoding, predictive image data is recorded in a memory managed by the memory management method according to claim 2. If the image data in the memory required for decoding the image sequence is destroyed, the decoding process is interrupted until decoding becomes possible without referring to the image data in the memory. Therefore, the use efficiency of the memory area can be increased, the possibility of losing the screen data necessary for decoding can be greatly reduced, and when the screen data required for decoding has been lost, the image quality degradation can be reduced. This has the effect of realizing an image decoding method having a high practical value.

According to the image decoding method of the present invention, the first to n-th image sequences (where n is 2
An image decoding method for simultaneously decoding image data of the above (integer), wherein when performing inter-screen prediction decoding, predictive image data is stored in a memory managed by the memory management method according to claim 3. The important image series is recorded with high importance, and the image data necessary for decoding the important image series is hardly destroyed, so the memory area utilization efficiency is improved and the screen data necessary for decoding is The possibility of loss can be greatly reduced, and even when screen data required for decoding is lost, image quality degradation can be reduced.
There is an effect that an image decoding method having a high practical value can be realized.

According to the image display method of the invention of claim 8 of the present application, there is provided an image display method for simultaneously displaying image data of first to n-th image series (n is an integer of 2 or more). Since the image data to be displayed is recorded in the memory managed by the memory management method according to claim 1, the use efficiency of the memory area is increased, and the possibility of losing screen data necessary for display is reduced. There is an effect that an image display method which can be greatly reduced and has high practical value can be obtained.

According to the image display method of the ninth aspect of the present invention, there is provided an image display method for simultaneously displaying image data of first to n-th image series (n is an integer of 2 or more). When the image data in the memory necessary for displaying the image sequence is destroyed, the image data at the latest undestructed time of the image sequence is displayed instead of the destroyed image data. As a result, the use efficiency of the memory area can be increased, the possibility of losing the screen data required for display can be greatly reduced, and when the screen data required for display has been lost, image quality degradation can be reduced. There is an effect that an image display method with high practical value can be realized.

According to the memory management device of the tenth aspect of the present invention, the memory management device simultaneously records the image data of the first to n-th image series (n is an integer of 2 or more) in the memory. And the area in the memory is ADst
The first to n / 2th memory addresses are assigned by assigning addresses from address art [i] to address ADend [i] (i is an integer satisfying 1 ≦ i ≦ n / 2) as the i-th memory address space. Memory area dividing means for dividing into spaces;
ADstart [i] or AD in the ith memory address space
A region from one end of end [i] to either the upper or lower memory address in the i-th memory address space is defined as a k-th image sequence (k = i × 2-1).
Of the ADstart [i] or ADend [i] from the other end to either the lower or upper memory address in the ith memory address space. The region is designated as (k
+1) address generating means for generating an address to be used as an area for recording image data of the image series, so that the use efficiency of the memory area is improved, and the encoding / decoding / display is performed. This has the effect of realizing a memory management device with high practical value, which can greatly reduce the possibility of losing screen data necessary for the operation.

Further, according to the memory management program recording medium of the invention of claim 11 of the present application, the image data of the image sequences of the first to n-th image sequences (n is an integer of 2 or more) are simultaneously recorded. A memory management program recording medium in which a memory management program for executing a memory management method is recorded, wherein an area in the memory is changed from an address ADstart [i] to an address ADen.
The addresses up to address d [i] (where i is an integer satisfying 1 ≦ i ≦ n / 2) are divided into the first to n / 2th memory address spaces by allocating them as the ith memory address space. ADstart [i] in the above memory address space of i
Alternatively, an area from one end of ADend [i] to either the upper or lower memory address in the i-th memory address space is defined as a k-th image sequence (k =
i × 2-1) is used as an area for recording image data, and the above-mentioned ADstart [i] or ADend [
The area from the other end of [i] to either the lower or upper memory address in the ith memory address space is used as an area for recording image data of the (k + 1) th image sequence A memory management program for executing a memory management method to perform the memory management method.
It is possible to provide a program recording medium on which a program capable of causing a computer to execute a memory management method having high practical value, which can significantly reduce the possibility of losing screen data necessary for display, is provided. .

Further, according to the memory management device of the twelfth aspect of the present invention, a new state is input to a memory area storing an empty state or i image data (i = 1 to 3). When storing image data, at least
If the image data can be stored at the lowest or uppermost address of the memory area, the newly input image data is stored at the lower or uppermost address of the memory area, If image data cannot be stored at the lowest and highest addresses of the memory area, a memory management method for storing and managing the memory by storing the newly input image data in an intermediate portion. When storing the newly input image data in the intermediate portion, the newly input image data is stored in an area that is in contact with one of the image data stored on the uppermost side and the lowermost side. Alternatively, the newly input image data is stored in an area adjacent to the address center position of the memory area, so that the memory use efficiency can be improved. , It is possible to greatly reduce the possibility of impossible decoded by insufficient memory required for decoding.

According to the memory management apparatus of the present invention, the image data stored in the memory area and the newly input image data are stored in the memory management method. Are all rectangular image data, when the image data is stored in the intermediate portion, the newly input image data is stored in an area that is in contact with one of the image data stored on the uppermost side and the lowermost side. To relocate the memory space position of the already stored image data so that the stored image data is stored,
Since a larger memory amount can be secured, the use efficiency of the memory area can be increased, and the possibility that decoding cannot be performed due to lack of memory required for decoding can be significantly reduced.

According to the memory management device of the present invention, in the memory management method of the present invention, the rearrangement of the memory space position of the already stored image data is performed by newly inputting. Is performed only when there is a shortage of memory for the decoded image data, so that the number of times of executing the memory rearrangement process, which is likely to interrupt the decoding process, can be reduced, and the memory use efficiency can be reduced. In addition to this, it is possible to significantly reduce the possibility that decoding becomes impossible due to a shortage of memory required for decoding, thereby realizing a memory management method of high practical value.

According to the memory management device of the present invention, the image data stored in the memory area and the newly input image data can be stored in the memory management method. When at least one of the image data is image data of an arbitrary shape, when storing the image data in the intermediate portion, the newly input image data is stored in an area adjacent to an address center position of the memory area. In addition, since the memory space position of already stored image data is rearranged, a memory area can be dynamically secured even if the size of each image data slightly changes. The possibility that decoding becomes impossible due to a shortage of memory required for this can be greatly reduced.

According to the memory management device of the present invention, in the memory management method of the present invention, the rearrangement of the memory space position of the already stored image data is performed by newly inputting. Is performed only when there is a shortage of memory for the decoded image data, so that the number of times of executing the memory rearrangement process, which is likely to interrupt the decoding process, can be reduced, and the memory use efficiency can be reduced. In addition to this, it is possible to significantly reduce the possibility that decoding becomes impossible due to a shortage of memory required for decoding, thereby realizing a memory management method of high practical value.

According to the memory management device of the present invention, when storing the image data in the intermediate portion, the uppermost side and the lowermost side are stored. In the memory management for storing the newly input image data in an area in contact with any of the image data stored on the side, if image data of an arbitrary shape is newly input, the image data is stored in the central portion. When storing, the memory space position of the already stored image data is rearranged so that the newly input image data is stored in an area adjacent to the address center position of the memory area. Until data is entered,
A larger memory amount can be secured, and after the image data of an arbitrary shape is input, the sum of the memory amounts of the two objects can be freely changed within the upper limit range, and the memory position can be re-established. Sufficient memory area for decoding can be used with little placement.

Further, according to the memory management method of the present invention, in the memory management method of the present invention, after the rearrangement of the memory space position accompanying the input of the image data of an arbitrary shape, Only when the memory for the newly input image data is insufficient, when storing the image data in the central portion, to store the newly input image data in an area adjacent to the address center position of the memory area,
Since the memory space position of the already stored image data is rearranged, the number of times of performing the memory rearrangement process which is likely to interrupt the decoding process can be reduced, and the memory utilization efficiency can be reduced. And the possibility that decoding becomes impossible due to lack of memory required for decoding can be greatly reduced, and a memory management method of high practical value can be realized.

[Brief description of the drawings]

FIG. 1 shows an example of an image composed of a plurality of image sequences.

FIG. 2 is a diagram showing an example in which a memory address is divided by the memory management method according to the first embodiment of the present invention;

FIG. 3 is a diagram showing an example in which a memory address is divided by the memory management method according to the first embodiment of the present invention;

FIG. 4 is a block diagram of a memory unit according to a second embodiment of the present invention.

FIG. 5 is a flowchart showing a case where a recording procedure of a memory unit according to a third embodiment of the present invention is realized by software;

FIG. 6 is a diagram showing a flowchart in a case where a read procedure of a memory unit according to a fourth embodiment of the present invention is realized by software;

FIG. 7 is a block diagram of a memory unit according to a fifth embodiment of the present invention.

FIG. 8 is a block diagram of an image decoding apparatus according to a sixth embodiment of the present invention.

FIG. 9 is a block diagram of a memory unit according to a seventh embodiment of the present invention.

FIG. 10 is a block diagram of an image decoding apparatus according to a seventh embodiment of the present invention.

FIG. 11 is a block diagram of an image display device according to an eighth embodiment of the present invention.

FIG. 12 is a block diagram of a memory unit according to a ninth embodiment of the present invention;

FIG. 13 is a block diagram of an image decoding apparatus according to a tenth embodiment of the present invention.

FIG. 14 is a diagram for storing a program for realizing a memory management method, an image encoding method, an image decoding method, and an image display method according to the twelfth embodiment of the present invention by a computer system. Of the recording medium

FIG. 15 is a diagram showing an example of typical MPEG encoding.

FIG. 16 is a diagram showing a conventional example of dividing a memory address;

FIG. 17 shows Embodiments 2, 5, 7, and
9 is a block diagram of an image encoding device

FIG. 18 is a diagram illustrating a conventional and second, fifth, and seventh embodiments of the present invention.
9 is a block diagram of an image decoding apparatus.

FIG. 19 is a diagram showing a conventional and second, fifth, and seventh embodiments of the present invention.
9 is a block diagram of the image display device.

FIG. 20 is a block diagram of a conventional memory unit.

FIG. 21 is a diagram showing an example of a memory management method at the time of a simple profile according to the eleventh embodiment of the present invention;

FIG. 22 is a diagram showing an example of a memory management method at the time of a core profile according to the eleventh embodiment of the present invention.

FIG. 23 is a view illustrating a flowchart for explaining processing for switching from the simple profile mode to the core profile mode in the memory management method according to the eleventh embodiment of the present invention;

FIG. 24 is a flowchart illustrating an example of a memory address space securing procedure in a simple profile mode according to the eleventh embodiment of the present invention;

FIG. 25 is a diagram illustrating a flowchart for explaining relocation of data at Obj2, Obj3, and Obj4 positions in the simple profile mode according to the eleventh embodiment of the present invention.

FIG. 26 is a diagram showing a flowchart for explaining relocation of data at Obj3 and Obj4 positions in the simple profile mode according to the eleventh embodiment of the present invention;

FIG. 27 is a diagram illustrating a flowchart for explaining relocation of data at the Obj3 position in the simple profile mode according to the eleventh embodiment of the present invention.

FIG. 28 is a view showing a flowchart for explaining relocation of data at the Obj4 position in the simple profile mode according to the eleventh embodiment of the present invention;

FIG. 29 is a view showing a flowchart for explaining an example of a memory address space securing procedure in a core profile mode according to the eleventh embodiment of the present invention;

FIG. 30 is a diagram showing a flowchart for explaining relocation of data at Obj3 and Obj4 positions in the core profile mode according to the eleventh embodiment of the present invention.

FIG. 31 is a diagram showing an example of a memory arrangement at the time of a core profile according to the eleventh embodiment of the present invention;

FIG. 32 is a view illustrating a flowchart for describing memory relocation when the allowable value of each object data is changed in the memory management method in the core profile mode according to the eleventh embodiment of the present invention.

[Explanation of symbols]

 M10 Bank selector M12 Subtractor M14 Address generator M15 Address manager M17 Memory FM memory unit U20 Object selector U25 Display device Cs Computer system FD Floppy disk FDD Floppy disk drive

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (reference) H04N 5/907 G09G 5/00 555G 7/32 H04N 7/137 Z

Claims (18)

[Claims] 1. A memory management method for simultaneously recording image data of a first to an n-th image series (n is an integer of 2 or more) in a memory, wherein an area in the memory is shifted from an address ADstart [i]. ADend [i]
(I is an integer satisfying 1 ≦ i ≦ n / 2)
Divides into the first to n / 2th memory address spaces by assigning them as the memory address spaces, respectively, and ADstart [i] or AD in the ith memory address space.
A region from one end of end [i] to either the upper or lower memory address in the ith memory address space is defined as a k-th image sequence (k = i × 2-1).
Of the ADstart [i] or ADend [i] from the other end to either the lower or upper memory address in the ith memory address space. A memory management method, wherein an area is used as an area for recording image data of a (k + 1) -th image sequence. 2. The memory management method according to claim 1, wherein both the image data of the k-th image sequence and the image data of the (k + 1) -th image sequence are recorded in the i-th memory address space. It monitors whether one image data overwrites the other image data and notifies the outside when reading the image data from the memory whether the image data of the image series to be read has been destroyed by the overwriting. A memory management method. 3. The memory management method according to claim 1, wherein both the image data of the k-th image sequence and the image data of the (k + 1) -th image sequence are recorded in the i-th memory address space. It monitors whether one image data overwrites the other image data, and if the overwriting can destroy the image data,
A memory management method characterized by referring to importance information of an image sequence input from the outside and overwriting destroying image data of low importance to protect image data of high importance. 4. An image encoding method for simultaneously encoding image data of first to n-th image sequences (n is an integer of 2 or more), wherein when performing inter-screen predictive encoding, predictive image data Is recorded in a memory managed by the memory management method according to claim 1. 5. An image decoding method for simultaneously decoding image data of first to n-th image sequences (n is an integer of 2 or more), wherein when performing inter-screen predictive decoding, predictive image data Is recorded in a memory managed by the memory management method according to claim 1. 6. An image decoding method for simultaneously decoding image data of first to n-th image sequences (n is an integer of 2 or more), wherein when performing inter-screen predictive decoding, predictive image data Is recorded in a memory managed by the memory management method according to claim 2. If image data in the memory required for decoding the image sequence is destroyed, the image data in the memory is not referred to. An image decoding method, wherein the decoding process is interrupted until decoding can be performed in step (1). 7. An image decoding method for simultaneously decoding image data of first to n-th image sequences (n is an integer of 2 or more), wherein an inter-screen predictive decoding is performed. Predicted image data is recorded in a memory managed by the described memory management method, with the importance of important image sequences being increased, so that image data necessary for decoding important image sequences is hardly destroyed. An image decoding method characterized by the following. 8. An image display method for simultaneously displaying image data of first to n-th image series (n is an integer of 2 or more), wherein the memory managed by the memory management method according to claim 1 includes: An image display method, comprising recording image data to be displayed. 9. An image display method for simultaneously displaying image data of a first to an n-th image series (n is an integer of 2 or more), wherein image data in a memory required for displaying the image series is stored. When destroyed, an image display method characterized by displaying, in place of the destroyed image data, image data of the image sequence at the latest unbroken time. 10. A memory management device for simultaneously recording image data of a first to n-th image series (n is an integer of 2 or more) in a memory, wherein an area in the memory is shifted from an address ADstart [i]. ADend [i]
(I is an integer satisfying 1 ≦ i ≦ n / 2)
A memory area dividing means for dividing the memory address space into first to n / 2th memory address spaces by allocating the memory address space to each of the memory address spaces;
A region from one end of end [i] to either the upper or lower memory address in the i-th memory address space is defined as a k-th image sequence (k = i × 2-1).
Used as an area for recording the image data of the image series, and from the other end of ADstart [i] or ADend [i] to either the lower or upper memory address in the ith memory address space. A memory management device, comprising: an address generation unit configured to generate an address for using an area directed to the area as an area for recording image data of a (k + 1) -th image sequence. 11. Simultaneously recording image data of image sequences of first to n-th image sequences (n is an integer of 2 or more),
There is a memory management program recording medium in which a memory management program for executing a memory management method is recorded, and an area in the memory is changed from an address ADstart [i] to an address ADend [i].
(I is an integer satisfying 1 ≦ i ≦ n / 2)
Divides into the first to n / 2th memory address spaces by assigning them as the memory address spaces, respectively, and ADstart [i] or AD in the ith memory address space.
A region from one end of end [i] to either the upper or lower memory address in the ith memory address space is defined as a k-th image sequence (k = i × 2-1).
Of the ADstart [i] or ADend [i] from the other end to either the lower or upper memory address in the ith memory address space. A memory management program for executing a memory management method that uses an area as an area for recording image data of a (k + 1) -th image sequence. 12. When storing newly input image data in an empty state or in a memory area storing i pieces of image data (i = 1 to 3), at least the address of the memory area When image data can be stored in the lower or uppermost side, the newly input image data is stored in the lowermost or uppermost side of the address of the memory area, and the lowermost of the address in the memory area is stored. A memory management method for managing storage of a memory by storing the newly input image data in an intermediate part thereof when image data cannot be stored in the intermediate part. When storing the input image data, the newly input image data is stored in an area that is in contact with one of the image data stored on the uppermost side and the lowermost side. Luke, or wherein the region in contact with the address central location in the memory area for storing a new input image data, a memory management method, characterized in that. 13. The memory management method according to claim 12, wherein the image data stored in the memory area and the newly input image data are all rectangular image data. When storing image data in the
Rearranging the memory space position of the already stored image data so as to store the newly input image data in an area that is in contact with any of the image data stored at the lowermost side, How to manage memory. 14. The memory management method according to claim 13, wherein the rearrangement of the memory space position of the already stored image data is performed only when the memory for the newly input image data is insufficient. A memory management method comprising: 15. The memory management method according to claim 12, wherein at least one of the image data stored in the memory area and the newly input image data is image data of an arbitrary shape. When storing the image data in the intermediate portion, the rearrangement of the memory space position of the already stored image data is performed so that the newly input image data is stored in an area adjacent to the address center position of the memory area. Performing a memory management method. 16. The memory management method according to claim 15, wherein the rearrangement of the memory space position of the already stored image data is performed only when the memory for the newly input image data is insufficient. A memory management method comprising: 17. The memory management method according to claim 12, wherein when the image data is stored in the intermediate part,
At the time of memory management for storing newly input image data in an area in contact with any of the image data stored at the lowermost side, if image data of an arbitrary shape is newly input, an image is displayed in the central portion. When storing data, a memory space position of already stored image data is rearranged so that newly input image data is stored in an area adjacent to an address center position of the memory area. Management method. 18. The memory management method according to claim 17, wherein after rearranging a memory space position accompanying input of image data of an arbitrary shape, only when a memory for newly input image data is insufficient, When storing the image data in the central portion, the memory space position of the already stored image data is rearranged so as to store the newly input image data in an area adjacent to the address central position of the memory area. A memory management method comprising: