JP2009118226A - Memory control device and control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a memory control device and a control method for reducing a bandwidth and the power of data transfer required for image processing. <P>SOLUTION: The memory control device includes: an image processing part for processing an image; a bit compression part for compressing an image supplied from the image processing part; an image memory for reserving the image; a memory schedular for scheduling reading and writing for reserving an output of the bit compression part to the image memory; and a bit restoring part for reading an output of the image memory by the memory schedular and restoring the compressed image. When writing or reading into or from the image memory, by altering an access unit to the image memory in a line unit, it is possible to reduce an access amount to the memory for the reduction of a memory bandwidth and power. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a memory control device and a control method for reducing data transmission bandwidth and power required for image processing.

  In image processing, the number of bits is increasing in the future, such as deep color. Along with this, the amount of data transferred to the external memory is also increasing.

  As a method for transmitting images and sounds as digital signals, there is HDMI (High Definitions Multimedia Interface). It also supports 30-bit, 36-bit, and 48-bit color depths exceeding 24 bits, enabling image and audio transmission at high image quality and high frame rate.

  When decoding a bitstream compressed by the MPEG (Moving Picture Experts Group) method, a wide memory bandwidth is required. Therefore, as a conventional technique for reducing the bandwidth with respect to the external memory, when the MPEG compressed bit stream is decoded and the restored image is stored in the external memory, the data transfer amount is reduced by reducing the resolution. There was a thing (for example, refer to patent documents 1). FIG. 2 shows a configuration of a conventional MPEG-like decoder described in Patent Document 1. In FIG. In FIG. 2, 200 is an input buffer, 201 is VLD, 202 is IQ, 203 is IDCT, 204 is MC, 205 is an output buffer, 206 is an MV processor, 207 is an MV scaling routine, 208 is a motion compensation device, and 209 is an element A processor 210 is an element scaling routine, and 211 is an anchor frame memory. In the MV scaling routine 207 in the MV processor 206, the reference motion vector is scaled and a reference image is taken out. When saving the generated image, the memory bandwidth is reduced by scaling the image with the element scaling routine 210 in the element processor 209.

  On the other hand, there is a system that increases the efficiency of memory bandwidth in a reconfigurable processor (see, for example, Patent Document 2). FIG. 3 shows a configuration of a reconfigurable processor having the conventional external memory and intelligent memory controller described in Patent Document 2. In FIG. 3, 300 is a logical block of a reconfigurable processor, 301 is a calculation function unit, 302 is a data access function unit, 303 is a memory device, 304 is a data prefetch unit, 305 is an intelligent memory controller, 306 is a memory device, 307 Is a block RAM.

The intelligent memory controller 305 and the data prefetch unit 304 slide the data so that unnecessary area data transfer from the external memory is eliminated and only the necessary area can be transferred. Further, by prefetching data, the logical block 300 of the reconfigurable processor is prevented from causing a stall due to data waiting, and an increase in memory bandwidth is prevented.
JP 2000-59795 A (page 9, FIG. 2) JP-T-2007-524905 (page 18, FIG. 9)

  For compression of moving images used in broadcasting or recording media, inter-frame or inter-field compression encoding such as MPEG (Moving Picture Expert Group) is generally used, and past frames and fields can be referred to. is necessary. Even in interlaced and progressive image processing that complements a field image to a frame image, a missing image is complemented by referring to a past field.

  In connection with HDMI or the like, emphasis is placed on high image quality, and the color depth is increased to 30 bits, 36 bits, 48 bits exceeding 24 bits, and the number of bits is increasing.

  The reference to these past fields and the expansion of the color depth increase the amount of read and write accesses to the memory, and increase the memory bandwidth. This requires a high-speed external memory, and there is a problem that increases memory cost and power consumption.

  The present invention solves these problems, and an object of the present invention is to provide a memory control device and a control method for reducing data transmission bandwidth and power required for image processing.

In order to solve the conventional problem, the memory control device of the present invention includes:
An image processing unit for processing an image, a bit compression unit for compressing an image supplied from the image processing unit, an image memory for storing an image, and an output of the bit compression unit for storing in the image memory A memory scheduler that performs reading and writing scheduling, and a bit restoration unit that reads the output of the image memory by the memory scheduler and restores the compressed image, and at the time of writing or reading to the image memory, in units of lines A feature of changing an access unit to the image memory.

  By using the memory control device of this configuration, it is possible to realize a low-cost and low-power system LSI, TV, optical disc recorder, and mobile phone while suppressing image quality deterioration.

  According to the memory control device of the present invention, the memory bandwidth can be reduced while suppressing image degradation by dynamically controlling the memory bandwidth according to the characteristics of the image.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a configuration diagram of a memory control device. The image processing unit can be applied to motion compensation of an image decoder for restoring an image compressed by MPEG or the like, image enlargement / reduction, noise reduction, image interpolation processing for image interpolation, and the like.

  As an example, a case where the image processing unit 120 is a filter that converts an interlaced image into a progressive image will be described.

  The image input to the image processing unit 120 is stored in the image memory 110 via the memory control unit 100. The image input from the image processing unit 120 in units of lines or blocks is stored in the line / block buffer 102 via the feature amount detection unit 101, and the feature amount detection unit 101 detects a person or the like in the image. Extract image features. The extracted feature amount is sent to the format determination unit 107. When the format determination unit 107 includes a person with the bit amount necessary for saving in line or block units, the bit amount is increased. The bit amount is determined so as to reduce the bit amount, and an appropriate bit compression format is selected from the formats stored in the format storage memory 106 according to the bit amount. At the same time, the address information is stored in the format storage memory 106 in correspondence with the bit compression format. The selected bit compression format is sent to the bit compression unit 103, and the bit compression unit 103 compresses the number of bits of the image stored in the line / block buffer 102 in luminance and color difference units according to the compression format. The compressed image is stored in the image memory 110 via the memory scheduler 105 that arbitrates write and read access to the image memory 110.

  The past field is read from the image memory 110 in order to perform interlaced progressive conversion by referring to the past field in the image processing unit 120. The past field image stored in the image memory 110 is sent to the bit restoration unit 104 via the memory scheduler 105. An appropriate bit restoration format is sent from the address of the image to the bit restoration unit 104 from the format storage memory 106, and the bit is restored in units of luminance and color difference. The restored image is sent to the image processing unit 120 as a reference image.

  With this method, it is possible to reduce the amount of memory read and write accesses while suppressing image degradation at a point of interest such as a person, and it is possible to reduce memory bandwidth and power.

(Embodiment 2)
FIG. 4 is a configuration diagram of the memory control device. The image processing unit can be applied to motion compensation of an image decoder for restoring an image compressed by MPEG or the like, image enlargement / reduction, noise reduction, image interpolation processing for image interpolation, and the like.

  As an example, a case where the image processing unit 120 is a filter that converts an interlaced image into a progressive image will be described.

  The image input to the image processing unit 120 is stored in the image memory 110 via the memory control unit 100. The image input from the image processing unit 120 in units of lines or blocks is stored in the line / block buffer 102 via the feature amount detection unit 101, and the feature amount detection unit 101 detects a person or the like in the image. Extract image features. The extracted feature amount is sent to the format determination unit 107. When the format determination unit 107 includes a person with the bit amount necessary for saving in line or block units, the bit amount is increased. The bit amount is determined so as to reduce the bit amount, and configuration information constituting an appropriate bit compression format is selected from the formats stored in the configuration information storage memory 402 according to the bit amount. The selected configuration information is sent to the reconstruction bit compression unit 401. The reconstruction bit compression unit 401 converts the image stored in the line / block buffer 102 into the number of bits in units of luminance and color difference according to the compression format. Compress. For example, the luminance is compressed from 12 bits to 9 bits, the color difference blue is compressed from 12 bits to 7 bits, and the color difference red is compressed from 12 bits to 7 bits, and from 36 bits (4 bytes) to 23 bits per pixel ( 3 bytes). As a result, the amount of data transferred to the memory can be reduced to 75%.

  In the output configuration buffer 403, the partial reconfiguration information is added once to the output of the reconfiguration bit compression unit 401 in units of lines or blocks, and the memory scheduler 105 that arbitrates write and read access to the image memory 110. Are stored in the image memory 110.

  The past field is read from the image memory 110 in order to perform interlaced progressive conversion by referring to the past field in the image processing unit 120. The past field image stored in the image memory 110 is sent to the input buffer 404 via the memory scheduler 105 together with the partial reconstruction information in units of lines or blocks. In the input buffer 404, the partial reconstruction information and the compressed image are separated, and the partial reconstruction information is sent to the reconstruction information decoder 405 so that part of the configuration of the reconstruction bit reconstruction unit 406 can be reconstructed to restore the bits. To. At this time, it is preferable that the partial reconstruction information is prefetched rather than the compressed image data. After the reconstruction is completed, the reconstruction bit restoration unit 406 sends the compressed image of the input buffer 404 to the reconstruction bit restoration unit 406 to restore the bits in units of luminance and color difference. For example, the luminance is restored from 9 bits to 12 bits, the color difference blue is restored from 7 bits to 12 bits, and the color difference red is restored from 7 bits to 12 bits. The restored image is sent to the image processing unit 120 as a reference image.

  With this method, it is possible to reduce the amount of memory read and write accesses while suppressing image degradation at a point of interest such as a person, and it is possible to reduce memory bandwidth and power.

  As shown in FIG. 5, this memory control device is configured as a system LSI 500 to which an MPEG decoder 502, a graphics processing unit 503, and an image filter 504 are applied as an image processing unit, and a stream separation unit 501 and an image output 505 are incorporated. May be.

  Further, as shown in FIG. 6, this memory control device may be configured as a TV 600 to which an antenna 601 is connected and a tuner 602, a display control unit 603, and a monitor 604 are incorporated.

  Further, as shown in FIG. 7, this memory control device is connected to a TV 706, and an MPEG encoder 701, a stream synthesizing unit 702, an input / output selector 703, an optical disk drive 704, and a hard disk drive 705 are incorporated to constitute an optical disk recorder 700. Good.

  Further, as shown in FIG. 8, this memory control device may be configured as a mobile phone 800 by incorporating a memory card IF 801, an antenna 802, a communication modulation / demodulation unit 803, and a liquid crystal panel 804.

  According to the memory control device of the present invention, it is possible to realize a high-quality system LSI with reduced memory bandwidth and power for data transfer necessary for image processing. The present invention can also be applied to applications such as digital video cameras and digital cameras, optical disc players, optical disc recorders, TVs and mobile phones.

1 is a configuration diagram of a memory control device according to Embodiment 1 of the present invention. Configuration diagram of a conventional MPEG-like decoder Configuration diagram of a reconfigurable processor having a conventional external memory and intelligent memory controller Configuration diagram of a memory control device in Embodiment 2 of the present invention Configuration diagram of system LSI in Embodiment 2 of the present invention Configuration of TV in Embodiment 2 of the present invention Configuration diagram of optical disc recorder in Embodiment 2 of the present invention Configuration diagram of mobile phone in Embodiment 2 of the present invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Memory control part 101 Feature quantity detection part 102 Line / block buffer 103 Bit compression part 104 Bit decompression part 105 Memory scheduler 106 Configuration information storage memory 110 Image memory 120 Image processing part 200 Input buffer 201 VLD
202 IQ
203 IDCT
204 MC
205 Output Buffer 206 MV Processor 207 MV Scaling Routine 208 Movement Compensator 209 Element Processor 210 Element Scaling Routine 211 Anchor Frame Memory 300 Reconfigurable Processor Logic Block 301 Computation Functional Unit 302 Data Access Functional Unit 303 Memory Device 304 Data Prefetch Unit 305 Intelligent memory controller 306 Memory device 307 Block RAM
401 Reconfiguration Bit Compression Unit 402 Configuration Information Storage Memory 403 Output Configuration Buffer 404 Input Buffer 405 Reconfiguration Information Decoder 406 Reconfiguration Bit Restoration Unit 500 System LSI
501 Stream separation unit 502 MPEG decoder 503 Graphic processing unit 504 Image filter unit 505 Image output unit 600 TV
601 Antenna 602 Tuner 603 Display control unit 604 Monitor 700 Optical disk recorder 701 MPEG encoder 702 Stream composition unit 703 Input / output selector 704 Optical disk drive 705 Hard disk drive 706 TV
800 Cellular phone 801 Memory card IF
802 Antenna 803 Communication modulation / demodulation unit 804 Liquid crystal panel

Claims (12)

An image processing unit for processing an image;
A bit compression unit for compressing an image supplied from the image processing unit;
An image memory for storing images;
A memory scheduler that schedules reading and writing to store the output of the bit compression unit in the image memory;
A bit restoration unit that reads out the output of the image memory with the memory scheduler and restores the compressed image;
A memory control device, wherein an access unit to the image memory is changed in units of lines or blocks when writing to or reading from the image memory. The memory control device according to claim 1, wherein the feature amount detection unit performs control so as to increase a bit amount at a center line of an image. The memory control device according to claim 1, wherein the feature amount detection unit detects specific information. The memory control device according to claim 3, wherein the feature amount detection unit detects a person. The memory control device according to claim 1, wherein the bit compression unit and the bit restoration unit are reconfigurable circuits. 6. The memory control device according to claim 5, wherein partial reconfiguration information designating a memory format is stored together with image data in the image memory, and the configuration of the bit restoration unit is reconfigured based on the read partial reconfiguration information. 7. The memory control device according to claim 6, wherein the partial reconfiguration information is pre-read and the configuration is changed. After the image is bit-compressed and scheduled for reading and writing to the image memory, the result is stored in the image memory, the stored compressed image is bit-restored as the output of the image memory, and the image memory A memory control method, wherein an access unit to the image memory is changed in units of lines or blocks in writing or reading. An image processing unit for processing an image;
A bit compression unit for compressing an image supplied from the image processing unit;
An image memory for storing images, and a memory scheduler for scheduling reading and writing to store the output of the bit compression unit in the image memory;
A bit restoration unit that reads out the output of the image memory with the memory scheduler and restores the compressed image;
A system LSI, wherein an access unit to the image memory is changed in units of lines or blocks when writing to or reading from the image memory. An image processing unit for processing an image;
A bit compression unit for compressing an image supplied from the image processing unit;
An image memory for storing images, and a memory scheduler for scheduling reading and writing to store the output of the bit compression unit in the image memory;
A bit restoration unit that reads out the output of the image memory with the memory scheduler and restores the compressed image;
A TV, wherein an access unit to the image memory is changed in units of lines or blocks when writing to or reading from the image memory. An image processing unit for processing an image;
A bit compression unit for compressing an image supplied from the image processing unit;
An image memory for storing images, and a memory scheduler for scheduling reading and writing to store the output of the bit compression unit in the image memory;
A bit restoration unit that reads out the output of the image memory with the memory scheduler and restores the compressed image;
An optical disc player and a recorder, wherein an access unit to the image memory is changed in units of lines or blocks when writing to or reading from the image memory. An image processing unit for processing an image;
A bit compression unit for compressing an image supplied from the image processing unit;
An image memory for storing images, and a memory scheduler for scheduling reading and writing to store the output of the bit compression unit in the image memory;
A bit restoration unit that reads out the output of the image memory with the memory scheduler and restores the compressed image;
A cellular phone characterized in that the unit of access to the image memory is changed in units of lines or blocks in writing or reading out of the image memory.

Images