JPH10210481A - Memory controller - Google Patents

Abstract

PROBLEM TO BE SOLVED: To convert interlaced scanning into a block form and to read out the reference image data with small memory capacity by shifting the position of an image data storage area by an extent that is decided by a prescribed calculation and for every input image. SOLUTION: The output 'addr' of an address generator 102 is added to the offset value by an adder 104. At the same time, the output of the adder 104 that exceeds a range of the full memory capacity is rounded by a remainder computing element 103, to be included in a range of H× 2V+(N+3B)/4}. Then the offset value showing the head position of an area, where the image data equivalent to a single screen, is changed respectively by -H×(N+3B)/4 every time the offset value returns to the head of an address space, and the position of a memory bank is shifted by -H×(N+3B)/4. Thus, it is possible to effectively use a memory whose read-out operation is finished by shifting the position of the memory bank as the time elapses.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a memory control device in a motion vector detector of a moving picture coding device.

[0002]

2. Description of the Related Art In recent years, MPE has been used as a moving picture coding method.
G-1 (ISO / IEC11172), MPEG-2
(ISO / IEC13818) and other inter-frame coding systems using motion compensation prediction are being used in the fields of storage, communication, and broadcasting. In these systems, motion-compensated prediction is performed in which each image of a moving image sequence is divided into coding blocks, and a prediction block is obtained using a motion vector detected from a reference image for each coding block.

FIG. 14 shows a conceptual diagram of interframe coding.

In the encoding process, first, an I picture 1401 is encoded. An I picture is a frame in which all encoded blocks are intra-frame encoded, and can be decoded using only its own encoded data.

[0005] P picture 14 following I picture 1401
In 02, motion compensation prediction from the I picture 1401 is performed, and only the prediction error is encoded. Therefore,
Decoding the P picture 1402 requires its own encoded data and the decoding result of the immediately preceding I picture 1401.

[0006] In the subsequent P picture 1403, motion compensation prediction is performed from the immediately preceding P picture 1402, so that decoding of the P picture 1402 requires its own encoded data and the decoding result of the immediately preceding P picture 1401. Becomes Hereinafter, similar processing is continued.

FIG. 15 shows the relationship between the coded block and the motion vector search area. What is motion vector detection?
In this process, a block having the highest correlation with the coding block is found from the motion vector search area on the reference image, and the relative position between the block and the coding block is used as a motion vector. In FIG. 15, the block size is B × B, and the size of the motion vector search area is M × N.
And

FIG. 7 is a block diagram showing an image input unit and its surroundings in the motion compensated prediction interframe coding apparatus. 701
Is a frame memory, 702 is a memory controller for managing the frame memory 701, 703 is a motion vector detector, 7
Reference numeral 04 denotes a moving image encoder that encodes an encoded block using the motion vector information calculated by the motion vector detector 703.

[0009] Input image data input from an input terminal 705 is temporarily written into a frame memory 701 by a memory controller 702, and is read out at required timings as coded block data and reference image data. 703 and a video encoder 704. The motion vector detector 703 detects motion vector information from the reference image data and the coded block data, and transmits the motion vector information to the video coder 704. Video encoder 704
Encodes encoded block data using the motion vector information detected by the motion vector detector 703, and outputs encoded data from an output terminal 706.

FIG. 8 shows a memory controller 702 in FIG.
An example of the configuration will be described.

Reference numeral 801 denotes a cache memory for temporarily storing input image data, 802 a cache memory for temporarily storing encoded block data read from the frame memory 701, and 803 a temporary storage of reference image data read from the frame memory 701. Is a cache memory stored in the cache memory.

Reference numeral 804 denotes a memory write address generator, reference numeral 805 denotes an encoded block read address generator, reference numeral 806 denotes a reference image read address generator, and reference numeral 80 denotes a reference image read address generator.
Reference numeral 7 denotes a selector for switching the three addresses and outputting the selected address to the frame memory. Reference numeral 808 denotes a bidirectional buffer for controlling the flow of data on the data bus.

Writing and reading of data to and from the frame memory 701 are performed in a time sharing manner. That is, input image data, which is data to be written to the frame memory 701, is temporarily stored in the internal cache memory 801 and is simultaneously written to the frame memory 701 during a writing operation. Similarly, read data is collectively read from the frame memory 701 during the read operation, and is temporarily stored in the internal cache memories 802 and 803. Thereafter, the data is read out from the cache memories 802 and 803 at a necessary timing and the motion vector
03, transmitted to the video encoder 704. A read address and a write address are selected by the selector 807 and input to the frame memory 701 in synchronization with the read operation and the write operation, and the data transfer direction is controlled by the bidirectional buffer 808 at the same time.

Next, the operation of the conventional memory controller will be described.

The input image data (digital video signal) input to the encoding apparatus is generally input in the order of interlaced scanning as shown in FIG. That is, FIG.
2, the odd-numbered lines (1, 2, 3,
…: Odd field) pixel data is input to the bottom of the screen, and then, again from the top of the screen, pixel data of even lines (1 ′, 2 ′, 3 ′... Is done.

In encoding, processing is performed in block units, so that data input by interlaced scanning is
It is necessary to rearrange the data into a block format as shown in FIG. For this reason, the input digital video signal is temporarily written to the frame memory 701 in the order of interlaced scanning, and is read out from the frame memory 701 in a block format, whereby the rearrangement into the block format is executed. FIG. 13 shows an example in which the block size B = 8.

Here, in the frame picture structure of MPEG-2, the coding block is generated so as to have an interlaced scanning structure. That is, the odd lines in the coding block are composed of odd fields, and the even lines are composed of even fields. Therefore, in order to form an encoded block, data (frame data) of both the odd field and the even field must be aligned on the frame memory. For this reason, once data for one frame is once written in the frame memory, reading of the encoded block is started.

When all the frames to be encoded are composed of I pictures, the conversion from interlaced scanning to block format divides the frame memory 701 into two memory banks and alternately switches between the writing side and the reading side. Can be performed by That is, the operation of writing interlaced scan data to one memory bank and reading block data from the other memory bank is as follows.
What is necessary is just to alternately repeat every frame time.

On the other hand, as shown in FIG. 14, when the coding of an I picture and the coding of a P picture are combined, when a motion vector is detected from an original image (when the original image is used as a reference image in motion vector detection),
The I picture or the P picture read as a coded image becomes a reference picture of a subsequent P picture after the coding process is completed, and is thus read again.

In the above-described two-bank division where only the I picture is encoded, the next input image is immediately written into the read area after the reading of the encoded block, so that the reference image cannot be read. For this reason, it is necessary to divide the frame memory 701 into three memory banks for use. That is, as shown in FIG. 10, the input image writing is performed for one frame period,
The reading of the coded block data is performed during the frame period, the reading of the reference image data is performed during the last one frame period, and these processes are shifted by one frame period in each of the three memory banks, so that a series of processes is performed without interruption. be able to.

FIG. 9 is an example of a block diagram of the write address generator 804, the coded block read address generator 805, and the reference image read address generator 806 for realizing the above-mentioned memory control. here,
The address generator 901 includes the write address generator 8.
04, a write address of interlaced scan data is generated, an encoded block read address generator 805 generates an encoded block read address, and a reference image read address generator 806 generates a read address of reference image data. . The operation of the other parts is common to the address generators 804, 805, and 806.

The adder 902 adds the offset value selected by the selector 903 to the address generated by the address generator 901.

Here, when one word of the frame memory corresponds to one pixel,

[0024]

(Equation 1)

[0025]

(Equation 2)

## EQU1 ## Note that H is the number of pixels in the horizontal direction of the image, V is the number of lines in the vertical direction of the frame image, and H
× V is the number of words in one bank.

At this time, the output ad of the address generator 901
Since dr accesses only data in one bank, if one word of the frame memory corresponds to one pixel,
The following expression is satisfied.

[0028]

(Equation 3)

Therefore, when the head address of the memory bank 1 is added to the output of the address generator 901, the output out of the address generator becomes

[0030]

(Equation 4)

Thus, data access on the memory bank 1 becomes possible. Hereinafter, similarly, the address generator 901
When the head address of the memory bank 2 is added to the output of the memory bank 2, the data can be accessed on the memory bank 2, and when the head address of the memory bank 3 is added, the data can be accessed on the memory bank 3.

FIG. 11 shows the operation timing of the selector 903. In the next one frame period when the selector in the write address generator 804 selects the memory bank 1, the coded block read address generator 805 is read.
Select the memory bank 1 for reading the encoded block data. In the subsequent one frame period, the reference image reading address generator 80
The selector in 6 selects the memory bank 1 for reading the reference image. As described above, the selectors in the write address generator 804, the coded block read address generator 805, and the reference image read address generator 806 access the same bank with a time difference of one frame period.

[0033]

In the above-described memory control, a memory area for storing images of three frames is required for detecting a motion vector. In the NTSC television resolution, the number of pixels in the vertical and horizontal directions is 720 × 480.
When the luminance value of each pixel is expressed by 1 byte, 720 × 480 × 3 is used to store the pixel data of the luminance value.
= 1,036,800 MByte memory capacity is required. Therefore, in the conventional memory controller, an enormous memory capacity is required for performing the interlaced scanning → block format conversion and reading the reference image data, which has been a major obstacle to cost reduction.

In the present invention, with a small memory capacity,
It is an object of the present invention to provide a memory controller capable of executing conversion from interlaced scanning to block format and reading reference image data.

[0035]

According to the present invention, there is provided a memory control device, comprising: an address generating means for generating an address for accessing a data area for one screen in a memory for storing encoded image data; Offset generating means for designating the head position of the data storage area for one screen, an adder for adding the output of the address generating means and the output of the offset generating means, and the output of the adder as the size of the used memory area A remainder computing unit for calculating a remainder when divided by an adder for adding the size of the image data area for one screen to the pre-update offset value; and an output of the adder. From the size of the used memory area.

[0036]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First, a memory control system according to the present invention will be described.

FIG. 3 is a memory map for explaining the storage position in the memory space of the pixel data read / written in the present invention.

In FIG. 3, for the sake of simplicity, the position of the pixel data on the screen and the storage position in the memory space are in a one-to-one correspondence. That is, one word of the frame memory 701 is made to correspond to one pixel, addresses of the number of horizontal pixels H words of the image are allocated in the horizontal direction of the map, and the memory space is expressed two-dimensionally as shown in FIG. doing. Therefore, the positional relationship of the data in the address space shown in FIG. 3 matches the positional relationship of the corresponding pixels on the screen.

In actual hardware, in order to increase the memory bandwidth (increase the amount of data that can be read and written per unit time), the frame memory 7
01, the data bus width of the frame memory 701 is increased.
It is also possible to associate a plurality of pixels with one word. Hereinafter, the memory management in the device of the present invention will be described with reference to FIG. Note that the leading English characters of each item correspond to a to f in FIG.

(A) Input image data input in the interlaced scanning order is written to the sequential frame memory 701. In FIG. 3, since the address space is two-dimensionally developed as H words in the horizontal direction, the input data is written in the memory space in the horizontal direction from the upper left corner to the lower right corner. Here, when data is written to the position of the V / 2 line, data writing for one field is completed, and data for one frame is obtained up to the position of the V line. In FIG. 3, the thin mesh area represents the data of the first frame.

(B) Writing the odd field data of the first frame is completed, and then the even field data is written in B /
The reading of the coded block data 301 is started from the point of time when writing is performed up to two lines. Since the coded block data has an interlaced structure, data is read from the odd field data and the even field data of the first frame in blocks of B pixels in the horizontal direction and B / 2 pixels in the vertical direction, and these are alternately combined in line units. Thus, encoded block data of size B × B can be obtained.

(C) After the writing of the even field data of the first frame is completed, the writing of the odd field data of the second frame is subsequently performed. In FIG. 3, the dark mesh area indicates the data of the second frame.

The time required to read the coded block data for one screen is one frame time as in the case of writing. Therefore, when the even field data of the second frame has been written up to the B / 2 line, reading of the coded block data from the first frame is completed, and then reading of the coded block data 302 from the second frame. Is started. At this time, reference image data 303 is read from the first frame corresponding to the encoded block data 302 at the same time.

When the area shown in FIG. 15 is used as a motion vector search area, the vertical size of the search area is B / 2 + (NB) in FIG. ) / 4 = (B + N) / 4 pixels. Therefore, an area 303 surrounded by a broken line is a motion vector search area (reference image data).

(D) In each field of the first frame, the data written in the area 305 above the upper end of the motion vector search area 304 is not read out again. Therefore, the area 305 becomes an unnecessary area, and another data can be written.

On the other hand, after the writing of the even field data of the second frame is completed, the writing of the odd field of the third frame is subsequently performed. At this time, B / 2 + (N + B) / 4 = (N + 3) of the odd field of the third frame.
B) Up to / 4 line is written in the area 306 following the data area of the second frame, and the remaining part is written from the top of the address space which is an unnecessary area.

The shaded area represents the data of the third frame.

(E) As described in (d), since (N + 3B) / 4 lines of data of the third frame are written immediately after the second frame, the remaining data to be written from the beginning of the address space is written. Odd field data is V /
2- (N + 3B) / 4 lines. Therefore, when the data for B / 2 lines of the even field of the third frame is written subsequently, the total number of lines from the head of the address space is {V / 2− (N + 3B) / 4} + (B /
2) = V / 2− (B + N) / 4 lines, and the odd field data of the first frame remaining on the memory at this time is (B + N) / 4 lines from the bottom of the screen.

On the other hand, the upper end of the motion vector search area 307 for the coded block in the lowest column on the screen is located at (B + N) / 4 line from the lower end of the field image. Therefore, until the reading of the last coded block data of the second frame is completed, it is necessary to hold the data from the bottom of the odd field of the first frame to (B + N) / 4 lines in the memory.

Here, the timing of ending the reading of the coded block data from the second frame is the time when the writing of up to the B / 2 line of the even field of the third frame has been completed in the frame memory. At this time, the odd field data of the first frame remaining on the memory is (B + N) / 4 lines from the bottom of the screen.

As described above, the writing area of the even-numbered field data of the third frame and the reading area of the reference image data from the odd-numbered field of the first frame do not overlap with each other. Until the readout of the first frame is completed, data from the bottom of the odd field of the first frame to (B + N) / 4 lines is held in the memory. "

(F) Following the completion of the reading of the coded block data from the second frame, the reading of the coded block data from the third frame is started, and the reading of the reference pixel data from the second frame in parallel. Is performed.

By repeating the above, H
× {2V + (N + 3B) / 4} pixels of memory space,
Interlaced scanning → block format conversion and reference image data reading can be executed in parallel.

FIG. 1 shows an embodiment of the write address generator 804, the coded block read address generator 805, and the reference image read address generator 806 for realizing the above-mentioned memory control. Here, reference numeral 101 denotes an offset generator that outputs the start address of the image data storage area.

The address generator 102 generates a write address of interlaced scan data in the write address generator 804, generates a read address of the coded block in the coded block read address generator 805, and generates a reference image. The read address generator 806 generates a read address of the reference image data. Numeral 103 denotes a remainder arithmetic unit whose input is H × ｛2
The remainder when dividing by V + (N + 3B) / 4} is output.
In addition, 104 and 105 are adders.

FIG. 2 shows an embodiment of the offset generator in FIG.

Here, 201 is an adder, and 202 is 103
The remainder arithmetic unit having the same function as the above, and 203 is a register.

FIG. 4 shows the address output out in FIG.
5 is a time chart of the offset value and the offset value offset.
4, a time chart of the address output out is shown on the upper side, and a time chart of the offset value offset is shown on the lower side using a common time axis.
4 shows the relationship between the output timing out and the timing of the offset value offset. In the upper part of FIG. 4, the solid arrow indicates the output of the write address generator 804, the thick broken arrow indicates the output of the encoded block data read address generator 805, and the thick dashed arrow indicates the reference image data read address generator 80.
6 represents the output.

Since the image data written to the memory for each line of the image is read in blocks, the read address corresponds to the pixel readout in the horizontal direction of the block as shown in FIG. The generated addresses occur at intervals of H words within a certain range, and this range moves with time. Arrows indicating read addresses in FIG. 4 indicate a range in which an address value is generated. Therefore, in FIG. 4, the read address is represented by a thick line in order to express a transition in this range.

Further, in FIG. 4, the light mesh, dark mesh, and hatched areas correspond to FIG. 3, and represent the types of frame data held in the memory.

The operation of FIG. 1 will be described below with reference to FIG.

The register 203 storing the offset value offset is updated once in one frame period.
At this time, the value of offset is set to 1 by the adder 201.
The number of words in the image data area for the screen is increased by the number of words H × V, and 0 ≦ offset <H
It is rounded to fall within the range of {2V + (N + 3B) / 4}. Therefore, assuming that the initial value of the register 203 is 0, offset is

[0063]

(Equation 5)

Changes as follows.

Here, the timing of writing to the frame memory and reading of the coded block correspond to one field period (1
/ 2 frame period) + B / 2 line input period shift, and the timing of reading the coded block and the reference image data shift by one frame period. Therefore, the update timing of the register 203 in each address generator is shown in the lower part of FIG. As shown, the write and read timings are shifted by the B / 2 line input period.

At this time, the output start timing of the address generator 102 also coincides with the offset value update timing. That is, the address generator 102 is reset at the timing when the register 203 is updated, and subsequently generates an address value required for one frame period.

The output addr of the address generator 102 is added by the adder 104 to the offset value. Here, the output of the adder 104 exceeding the range of the total memory capacity H × {2V + (N + 3B) / 4} is H × {2V + (N +
3B) / 4 is rounded by the remainder arithmetic unit 103 to fall within the range. Output add of address generator 102
r satisfies Equation 3 as in the conventional example. Therefore, the output addr_r of the remainder arithmetic unit is represented by the following equation.

[0068]

(Equation 6)

Referring to the upper part of FIG. 4, a solid line arrow representing a write address does not intersect with a dashed line arrow representing a read address of a motion vector search area. Therefore, it can be seen that in the area where the reference image data is written, the interlaced scan data writing is performed before the reading of the search area data, and it does not occur that the necessary data cannot be read.

In the conventional memory control shown in FIG. 10, the position of the bank for storing the image data is fixed, and the image data is always written in the same position.

On the other hand, in the memory control device of the present invention, if attention is paid to the offset value indicating the head position of the area for storing the image data for one screen, every time the offset value returns to the head of the address space,- H × (N + 3
B) A transition is added by / 4, and the memory bank position is -H
× (N + 3B) / 4. As a result, the memory space can be used efficiently, and the conversion from the interlaced scanning to the block format and the reading of the reference image data can be executed with a smaller memory capacity than before.

[0072]

According to the present invention, the position of the memory bank for storing the frame data is moved with time, so that the memory area for which reading has been completed can be used efficiently. Can be converted to a block format and reference image data can be read.

Here, the number of pixels in one frame is 720 × 4
At 80, when the size of the coding block is B = 16 and the size of the motion vector search area in the vertical direction is N = 48, the memory capacity required to store the pixel data of the luminance component is 720 × ｛2. × 480 + (48 + 3 × 16) / 4｝ =
708,480 bytes. In the conventional example,
Since a memory having a capacity of 036,800 bytes is required, the memory capacity required in the present invention is 70% of the conventional memory capacity.
And the memory can be greatly reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment of an address generator 804, 805, 806 (FIG. 8) in a memory control device of the present invention.

FIG. 2 is a block diagram of an embodiment of an offset generator 101 in FIG.

FIG. 3 is a memory map diagram in the memory control device of the present invention.

FIG. 4 is a time chart of the embodiment in the memory control device of the present invention.

FIG. 5 is a diagram illustrating addresses generated by a read address generator in the embodiment of the memory control device of the present invention.

FIG. 6 is a diagram for explaining notation of a map developed two-dimensionally in the memory map of FIG. 3;

FIG. 7 is a block diagram around an image input unit in a video encoder using motion compensation prediction.

FIG. 8 is a block diagram of a memory controller 702 in FIG. 7;

FIG. 9 shows the address generators 804, 805 and 806 of FIG.
FIG. 4 is a block diagram of a conventional example in FIG.

FIG. 10 is a diagram showing a switching timing of a memory bank in a conventional example.

11 is a diagram showing the control timing of the selector 903 in the address generators 804, 805, 806 in the memory bank switching shown in FIG.

FIG. 12 is a diagram illustrating writing of image data input in the order of interlaced scanning to a memory.

FIG. 13 is a diagram illustrating block reading of image data written in a memory in the order shown in FIG. 12;

FIG. 14 is a diagram illustrating a prediction structure in inter-frame coding of a moving image.

FIG. 15 is a diagram illustrating a motion vector search area in motion vector detection.

[Explanation of symbols]

 Reference Signs List 101 offset generator 102 address generator 103 remainder operation unit 104, 105 adder

Claims (4)

[Claims] 1. An image of a moving image sequence is divided into coding blocks, and a motion vector for generating a prediction block from a motion vector detection area on a reference image is detected for each coding block. In the motion vector detection device in the moving image encoding device that generates a prediction block from a reference image using, and encodes a difference image block between the coded block and the prediction block, the input image data is changed from interlaced scanning to a block format. In an image memory for performing conversion and reading out pixel data for motion vector detection, the position of the image data storage area is moved for each input image by an amount determined by a predetermined calculation from the size of the motion vector detection area and the image size. A memory control device for a motion vector detector. 2. An address generator for an image memory, comprising: an address generator for generating an address for accessing a data area for one screen; and an offset generator for specifying a head position of a data storage area for one screen. Means,
An adder (A) for adding the output of the address generating means and the output of the offset generating means, and the adder (A)
A remainder arithmetic unit (A) for calculating a remainder when the output of the image data is divided by the size of the used memory area. The offset generating means includes a register for holding an offset value, An adder (B) for adding the size to the pre-update offset value stored in the register, and a remainder calculator for calculating a remainder when the output of the adder (B) is divided by the size of the used memory area 2. The memory control device for a motion vector detector according to claim 1, further comprising (B), wherein an output of the remainder arithmetic unit (A) is used as a memory address. 3. The offset generating means according to claim 2, wherein the offset value is updated every one screen period, and the size of the used memory area is H × {2V + (N + 3B) / 4} (H: image data , V: the number of pixels in the image data in the vertical direction, N: the number of pixels in the motion vector detection area in the vertical direction, and B: the number of pixels in the coding block in the vertical direction. 2. The memory control device for a motion vector detector according to claim 1, wherein the memory size is an area size. 4. When the writing of image data input in the order of interlaced scanning to the memory is completed up to V / 2 + B / 2 lines, reading of an encoded block in the input image data is started. And reading the vector detection area pixel data from the encoded block being read out in parallel.
A memory control device for a motion vector detector as described in the above.