JP2011061603A - Image processing device, and method for processing image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing device which can reduce the number of times of access to an image memory, and to provide a method for processing an image. <P>SOLUTION: A configuration in which an image data processing section 33 that has been mounted in an image processing unit 10 is mounted in an image memory controller 30 and additionally, an address conversion section 32 and a control section 31 are mounted is used. Moreover, by defining a virtual readout address area of an image memory 20, a memory usage can be reduced and the number of times of access to a memory can be reduced. Communication speed can be improved. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing method.

  As the resolution of the input image of the image processing apparatus increases, there are problems such as insufficient capacity of the image memory and a slow image transfer speed between the image memory and the arithmetic processing unit.

In the process of obtaining the desired image processing result, conversion processing such as enlargement or reduction of an image or a filter by convolution is often performed. During the processing, a large amount of data transfer is required between the enlargement / reduction filter processing unit that performs conversion and the memory. More specifically, when the image processing unit performs image processing,
(1) Read the original image data from the image memory,
(2) After performing the data conversion process in the enlargement / reduction filter unit, the converted data is written in the image memory,
(3) Further, since the converted data must be read from the image memory and the subsequent image calculation process is executed, the capacity of the image memory and the number of accesses are inevitably increased.

  As a conventional technique for solving the above problem, there is a method of reducing the amount of image data by thinning out by address conversion. If the amount of image data is small, the capacity of the image memory is reduced and the amount of data to be transferred is also reduced (see, for example, Patent Document 1).

JP-A-4-166922

  However, in the invention of Patent Document 1, since the image data is simply thinned out, for example, when the image data is reduced and displayed, the image quality is deteriorated as compared with the image subjected to the filter processing or the like. Further, this method cannot cope with enlargement of image data.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus and an image processing method capable of reducing the number of accesses to an image memory.

  In order to achieve the above object, the image processing apparatus of the present invention separates an image memory in which image data is stored and a real address area of the image memory when reading the image data written in the image memory. Means for accessing the image memory using a provided virtual read address; address conversion means for converting the virtual read address to an original image data address; and reading from the image memory by the original image data address Image data processing means for executing a predetermined calculation on the original image data thus generated and generating and outputting image data different from the image data.

  According to the image processing apparatus of the present invention, the processed image data can be read out by accessing the image memory a smaller number of times compared to the prior art. By defining the virtual read address area in the image processing unit, it is possible to reduce the used capacity of the image memory and increase the speed of image transfer.

1 is a block diagram showing components of an image processing apparatus according to a first embodiment of the present invention. The address map which shows operation | movement of the virtual read address area defined in the memory controller which concerns on the said embodiment. FIG. 5 is a diagram showing out-of-region pixel extension according to the embodiment. The figure which shows the operation | movement which reads data per block which concerns on the said embodiment. The figure which shows the specific example which reads data per block which concerns on the said embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to the first embodiment of the present invention.

  As shown in FIG. 1, the image processing apparatus includes an image processing unit 10, an image memory 20 that stores image data, and an image memory controller 30 that connects the image processing unit (CPU) 10 and the image memory 20. Is done.

  Further, the image processing unit 10 has an internal storage unit 11 (cache memory) that stores image data after image data processing. The image memory controller 30 includes a control unit 31, an address conversion unit 32, and an image data processing unit 33. Note that an external storage device may be attached instead of the internal storage unit 11. The control unit 31, the address conversion unit 32, and the image data processing unit 33 may be configured separately.

  The image processing unit 10 includes an image processing processor. For example, the image processing unit 10 performs processing such as object extraction using a background difference method, three-dimensional measurement using a stereo method, and image compression. In order to execute these processes, various commands are issued to the image memory controller 30 to access the image memory 20. At this time, the image processing unit 10 outputs the virtual address after the image processing to the internal storage unit 11, and immediately outputs the virtual address to the image memory controller 30 if the image processed image data cannot be read.

  In response to the read command from the image processing unit 10, the control unit 31 outputs the virtual address after the image processing to the address conversion unit 32. Further, the control unit 31 notifies the image data processing unit 33 of an image processing command.

  The address conversion unit 32 converts the virtual address input from the control unit 31 into a real address and outputs the real address to the control unit 31. The control unit 31 reads actual image data from the image memory 20 using the actual address.

  The image data processing unit 33 includes an enlargement / reduction processing unit, a filter processing unit, a peripheral pixel expansion processing unit, and the like. The image data processing unit 33 receives an image processing command from the control unit 31 and executes enlargement / reduction processing, filter processing, peripheral pixel expansion processing, or the like on the actual image data read from the image memory 20. The result is output to the image processing unit 10.

  FIG. 2 shows the address mapping of the image memory 20. That is, the image memory 20 is a mounted memory area, and has a normal read / write area 20a in which actual image data is stored. In addition, the image memory 20 is a virtual space that is a 1/2 times reduced image reading area 20b, a 2 times enlarged image reading area 20c, a filtered image reading area 20d, a block unit reading area 20e, and an out-of-region image extended reading. The area 20f is configured to have a bag. These 1 / 2-fold reduced image readout area 20b, 2-fold enlarged image readout area 20c, filtered image readout area 20d, block unit readout area 20e, and out-of-region image extension readout area 20f are formed in the internal storage unit 11. The storage area (cache target area) is a virtual address reading area by the image processing unit 10.

  Next, the operation of this embodiment will be described.

  A camera input unit (not shown) captures image data by a DMA (Direct Memory Access) method, and the image data is transferred to the normal read / write area 20a of the image memory 20 via the image memory controller 30 in a normal operation. Write.

  Here, for example, when the image processing unit 10 wants to read 1/2 times the image data, it generates a virtual address (for example, FF1000) indicating the 1/2 times reduced image reading area 20b of the image memory 20, A read command is issued to the image memory controller 30 (1). Note that when the virtual address is read from the internal storage unit 11, a read operation to the image memory controller 30 is prohibited. In this case, the conversion from the virtual address to the real address can be performed either automatically by programmatic conversion or by conversion using an address conversion table.

  The control unit 31 in the image memory controller 30 analyzes the 1/2 reduced image processing command from the image processing unit 10 and outputs the command to the image data processing unit 33 (2.1). In addition, the control unit 31 outputs the virtual address (for example, FF1000) received from the image processing unit 10 to the address conversion unit 32 (2.2).

  The address conversion unit 32 converts the virtual address for the 1 / 2-fold reduced image reading area 20b into the real address of the normal reading / writing area 20a (2.3). For example, a virtual address is converted to a real address using an address conversion table (not shown), but the present invention is not limited to this. Then, the control unit 31 outputs the real address converted by the address conversion unit 32 to the image memory 20 (2.4). As a result, the original image data is read from the normal read / write area 20a of the image memory 20. The read original image data is sent to the image data processing unit 33 (3).

  An enlargement / reduction processing unit (not shown) of the image data processing unit 33 creates 1/2 reduced image data from the original image data. Then, the image data reduced by 1/2 is sent from the image data processing unit 33 to the image processing unit 10 (4). The image processing unit 10 stores the image data reduced by ½ times at a real address corresponding to the virtual address of the internal storage unit 11 and uses it for subsequent image processing. In the case of double expansion image processing, double expansion image data can be obtained on the same principle as described above except that FF2000 is used as a virtual address.

  At this time, since the original image data in the image memory 20 is not changed, when the original image data is required, it may be read from the normal read / write area 20a. Note that the ½ times reduced image reading area 20 b is not limited to one, and a plurality of reduced (enlarged) image reading areas may be provided in the image memory 20. In the image enlargement / reduction processing, a plurality of reduced (enlarged) image readout areas can be assigned for each method such as the latest sample interpolation, linear interpolation, cubic convolution interpolation, or the like. In the enlargement process, an image restoration method based on super-resolution can be used.

  Next, the image reading of the filter process will be described. FIG. 3 is a diagram showing a rectangular area for filter processing.

  In the filter process, a convolution calculation process is performed on the rectangular area G2 in the image area G1 by a determinant having filter characteristics. When the filtering process is performed on the edge portion of the image area G1 (hereinafter referred to as “outside image area G3”), for example, pixels outside the image area G31 are used as shown in FIG. In this case, as the pixel value to be expanded, it is only necessary to select whether to use an edge value or a constant suitable for filtering.

  When the image processing unit 10 wants to read out the image data of the filter processing, the image processing unit 10 generates a virtual address (for example, FF6000) indicating the filtered image reading area 20d of the image memory 20, and reads out the image memory controller 30 from the read command. (1). Similarly, when the virtual address is read from the internal storage unit 11, the read operation to the image memory controller 30 is prohibited.

  The control unit 31 in the image memory controller 30 analyzes the filter processing command from the image processing unit 10 and outputs the command to the image data processing unit 33 (2.1). In addition, the control unit 31 outputs the virtual address (for example, FF6000) received from the image processing unit 10 to the address conversion unit 32 (2.2).

  The address conversion unit 32 converts the virtual address for the filtered image reading area 20d into the real address of the normal reading / writing area 20a (2.3). Then, the control unit 31 outputs the real address converted by the address conversion unit 32 to the image memory 20 (2.4). As a result, the original image data is read from the normal read / write area 20a of the image memory 20. The read original image data is sent to the image data processing unit 33 (3).

  A filter processing unit (not shown) of the image data processing unit 33 creates filtered image data from the original image data. As shown in FIG. 4, in the case of performing expansion outside the image area G31, a peripheral pixel expansion processing unit (not shown) operates together. The filtered image data is sent from the image data processing unit 33 to the image processing unit 10 (4). The image processing unit 10 stores the filtered image data at a real address corresponding to the virtual address in the internal storage unit 11 and uses it for subsequent image processing.

  Next, the block-unit image reading process will be described. FIG. 4 is a diagram illustrating image reading in units of blocks.

  When image data for each designated block area is necessary, for example, as shown in FIG. 4, the image data is stored on the image memory 20 at an address in the direction of the arrow. For this reason, when the normal read / write area 20a is accessed to read data in the block area, the read address becomes discontinuous, unnecessary data is read to the internal storage unit 11, and the cache function is effective. Does not work. When the image processing unit 10 fetches the unnecessary data, the image processing unit 10 deletes the data, reads the block data necessary for the next processing from the image memory 20, and repeats this. In this embodiment, in order to prevent this, an area for accessing the block area with continuous addresses is provided. Similarly, the control unit 31 of the image memory controller 30 sets the address conversion unit 32 so that the contents of the image memory 20 can be read at successive addresses in response to a block unit read request from the image processing unit 10.

  That is, the image processing unit 10 generates a virtual address (for example, FF8000) indicating the block unit reading area 20e of the image memory 20 and reads it to the image memory controller 30 when reading the image data in block units. Command (1). Similarly, when the virtual address is read from the internal storage unit 11, the read operation to the image memory controller 30 is prohibited.

  The control unit 31 in the image memory controller 30 analyzes the block unit readout command from the image processing unit 10 and outputs the command to the image data processing unit 33 (2.1). In addition, the control unit 31 outputs the virtual address (for example, FF8000) received from the image processing unit 10 to the address conversion unit 32 (2.2).

  The address conversion unit 32 converts the virtual address for the block unit read area 20e into an address indicating a block area in the actual image data in the normal read / write area 20a (2.3). In this case, an address is generated according to the size (block length) of the block area. Then, the control unit 31 supplies the address indicating the block area converted by the address conversion unit 32 to the image memory 20 (2.4), and the original image data in the block area from the normal read / write area 20a of the image memory 20 Is read (3). The read original image data of the block area is sent to the image processing unit 10 via the image data processing unit 33 (4). The image processing unit 10 stores the image data of the block area at a real address corresponding to the virtual address of the internal storage unit 11 and uses it for subsequent image processing.

  With reference to FIG. 5, address conversion for block unit reading will be specifically described.

5, W is the width of the entire real image, H is the height of the entire real image, a is the height of the block 51 to be read, b is the width of the block 51 to be read, and (X, Y) is the upper left corner of the block 51 to be read. Coordinates and dx indicate the X coordinate in the block 51 to be read, and dy indicates the Y coordinate in the block 51 to be read.

  Here, read_address is an address read from the image processing unit 10 (input of address conversion), mem_address is an address obtained from the address conversion unit 32 (output of address conversion), and block_address is an address of a block assigned in order from the upper left. . At this time, the output address mem_address after address conversion is obtained from the input address read_address.

First, the coordinates of the address in the block to be read are dx = read_address% b (1)
dy = (read_address% (a * b) / b (2)
It appears in. The number of blocks to be read is
block_address = read_address / (a * b) (3)
The upper left address of each is X = (block_address% (W / b)) * b (4)
Y = (block_address / (W / b)) * a (5)
It becomes. An output address is obtained as an input address through the above equations (1) to (5).

mem_address = (Y + dy) * W + X + dx (6)
An example is given by substituting specific numerical values. At this time, W = 32, H = 24, a = 8, and b = 8 are set. The numerical values of these image parameters are stored in a setting register in the control unit 31. Here, assuming that the read address read_address = 339, substituting into the above (1) to (5),
dx = 339% 8 = 3
dy = (339% (8 * 8)) / 8 = 2
block_address = 339 / (8 * 8) = 5
X = (5% (32/8)) * 8 = 8
Y = (5 / (32/8)) * 8 = 8
By substituting these values into (6), mem_address = (8 + 2) * 32 + 8 + 3 = 331
Is converted.

  As described above, according to the present embodiment described in detail, an enlarged or reduced image can be obtained by one reading between the image memory and the image processing unit (CPU). Similarly, an image obtained by performing a convolution filter operation with one reading is obtained. In addition, since the rectangular area of the image can be read with a continuous address, the cache utilization rate of the internal storage unit 11 of the image processing unit 10 is improved. Furthermore, the reference image out-of-region expansion process is not required when performing template matching for calculating the similarity between the reference image and the template image. Therefore, the capacity of the original image memory is not increased more than necessary.

  Note that the types and magnifications of the address areas that are virtually defined can be arbitrarily combined by the required data processing.

  In the present invention, the above-described embodiment can be appropriately changed in design without departing from the gist of the invention, and the embodiments may be appropriately combined as necessary.

10: Image processing unit 11: Internal storage unit (cache memory of image processing unit)
20 ... Image memory 30 ... Image memory controller 31 ... Control unit 32 ... Address conversion unit 33 ... Image data processing unit 50 ... Block 51 ... Block to be read

Claims (7)

An image memory for storing image data;
Means for accessing the image memory using a virtual read address provided separately from the real address area of the image memory when reading the image data written in the image memory;
Address conversion means for converting the virtual read address into an original image data address;
Image data processing means for executing a predetermined operation on the original image data read from the image memory by the original image data address, and generating and outputting image data different from the image data;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the image data processing unit is a data process to be enlarged or reduced.   2. The image processing apparatus according to claim 1, wherein the image data processing means is a convolution filter calculation process.   The image processing apparatus according to claim 1, wherein the image data processing means is image processing for performing pixel expansion.   The image processing apparatus according to claim 1, wherein the image data processing means includes two or more processing means according to claims 2 to 4. The image processing apparatus according to claim 1, wherein the address conversion unit converts the rectangular area so that the rectangular area can be read with continuous addresses. When reading the image data stored in the image memory, the image memory is accessed using a virtual read address provided separately from the real address area of the image memory,
Converting the virtual read address into an original image data address;
Performing a predetermined operation on the original image data read from the image memory by the converted original image data address, and generating and outputting image data different from the image data;
An image processing method characterized by the above.

Images