JP2009044654A - Affine transformation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device capable of performing affine transformation at high speed. <P>SOLUTION: A generating device 50 includes: a first input unit 61 which acquires an original scanning line data from a first data for outputting a first image; and an interpolation calculation unit 65 which generates pixel data for outputting a plurality of pixels of a second image. The pixel data includes line segment data of scanning lines of the second image. The generating device 50 includes: a first output unit 62 which stores block data containing each line segment data in a memory 59; a second input unit 71 which acquires block data containing line segment data for outputting a third scanning line of the second image from the memory 59; and a second output unit 72 which stores scanning line data for outputting a plurality of pixels included in the third scanning line in the memory 59 as the second data. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an apparatus for generating data for outputting an affine transformed image.

  Affine transformation is known as one of transformations from a plane figure (including projection of a solid figure) to a plane figure. The affine transformation is a transformation in which parallel lines are converted into parallel lines, and is used for rotating a figure by a predetermined angle or performing skew correction.

In the affine transformation, image data (including luminance data) generated by the xy coordinate system is converted into image data in a wz coordinate system obtained by rotating the xy coordinate system by θ. The rotating coordinate system is represented by the following formula (A). R (θ) is an affine transformation matrix.
(W, z) = R (θ) (x, y) (A)

The image data is RGB luminance data stored on the line (scanning line), and is stored in the memory by the following equation (B) to form the Zth line data in the rotating coordinate system. It is difficult to improve the processing speed because it is necessary to access to each address.
x = R ^ (-1) (θ) (w, Z)
y = R ^ (-1) (θ) (w, Z) (B)

In Patent Document 1, image data is read from a large-capacity low-speed memory in units of blocks and stored in a small-capacity high-speed memory. Further reading is performed according to the address generated by the address generation unit, and skew correction is performed by the calculation unit that performs the interpolation calculation. The address generation unit generates addresses sequentially by approximating the decimal part with a predetermined accuracy in order to read out the block in an oblique direction from the memory at an angle according to the inclination of the input image, and also according to the accuracy. By increasing / decreasing by a predetermined amount in a given period, the accumulated error of the address due to approximation is suppressed. On the other hand, from the small-capacity high-speed memory, the coordinates of four points of image data adjacent to the corrected pixel are calculated and obtained using an affine inverse matrix.
JP 2005-352703 A

  In the technique described in Patent Document 1, reading is performed in block units from a large-capacity low-speed memory to a small-capacity high-speed memory. However, from the small-capacity high-speed memory, it is necessary to read out data necessary for correction for each pixel to be corrected, and random access is required. Therefore, the processing time for inputting and outputting data becomes a critical path and / or bottleneck of the processing time for converting image data, and it is difficult to further reduce the time required for image processing.

  In a copy machine or an application for editing an image, the frequency of processing for correcting a minute inclination of an image (skew correction processing) is high, and it is important that this processing can be performed at high speed. Even if the inclination of the image is 2 degrees or less or 1 degree or less, it is often noticeable that the straight line portion is displayed obliquely, particularly in a digitized image. Further, by repeatedly displaying an image with a small inclination corrected, the user can select an image with the inclination corrected appropriately.

  One embodiment of the present invention is an apparatus that generates, from first data for outputting a first image, second data for outputting a second image in which the first image is affine-transformed. . This apparatus outputs a plurality of pixels included in a first original scan line and a second original scan line adjacent to each other in a first image from a first recording medium storing first data. A first input unit that obtains original scan line data; and an interpolation calculation unit that generates pixel data for outputting a plurality of pixels of the second image from the original scan line data. The pixel data includes line segment data for outputting a plurality of pixels included in the line segments of the plurality of scanning lines of the second image. The apparatus further includes a first output unit that stores block data including each line segment data as an input / output unit in the second recording medium, and a third output of the second image from the second recording medium. A second input unit for acquiring a plurality of block data including a plurality of line segment data for outputting a scanning line, and a second scanning line data for outputting a plurality of pixels included in the third scanning line. And a second output unit for storing the data in a third recording medium.

  A plurality of pixels (pixels after affine transformation) generated (interpolated) using pixels included in the first scanning line and the second scanning line of the first image are the first scanning line and the second scanning line. Between the scanning lines of the second image and the pixels included in a part (line segment) of the plurality of scanning lines of the second image. Affine transformation is a process of converting parallel lines into parallel lines. Basically, these line segments do not overlap. If the angle to be converted is within an appropriate range, one line segment is scanned in the scanning direction ( A plurality of pixels adjacent in the main scanning direction) are included. That is, the pixel data generated from the original scanning line data includes a plurality of line segment data, and each line segment data includes data for outputting a plurality of pixels. Accordingly, data (line segment data) for outputting a plurality of pixels in units of line segments is temporarily stored in a recording medium such as a memory, and the line segment data is read out in the order of the scanning lines of the second image to obtain the second data. By generating the scanning line data of the image, it is possible to omit the process of randomly reading out the data for outputting the pixels for each pixel. For this reason, the data input / output time in image processing including affine transformation can be shortened.

  The scanning direction is not limited to the scanning direction for outputting an image, and may be a direction for inputting image data in this apparatus. By using block data including each line segment data as an input / output unit, one line segment data may be included in one block data and stored, and one line segment data may be divided into a plurality of block data. It may be stored.

  The first output unit preferably stores the block data in the second recording medium with a fixed length. Data input / output time may be further reduced. The first output unit preferably stores the block data in the second recording medium with a burst length. Data input / output time may be further reduced. Even if the line segment data has a variable length due to a process such as compression, it can be input / output at a fixed length by a process such as adding padding data.

  The first output unit preferably stores the block data in the second recording medium while changing the address so that the block data is read in the order of the scanning lines of the second image. The second input unit can sequentially read the block data, and the second output unit can output the scan line data of the second image.

  The second input unit can acquire block data from the second recording medium by changing the address so that the block data is read in the order of the scanning lines of the second image. The first output unit can output block data sequentially.

  The interpolation operation unit preferably includes a first operation circuit that generates data for outputting a plurality of pixels included in each line segment data in parallel by interpolation in the sub-scanning direction. Parallel processing can reduce the time required for arithmetic processing. It is desirable that the interpolation operation unit further includes a second operation circuit that generates in parallel data for outputting a plurality of pixels included in each line segment data by interpolation in the scanning direction.

  The first output unit stores block data including the line segment data output from the first arithmetic circuit in the second recording medium, and the second arithmetic circuit acquires the line acquired by the second input unit. It is possible to correct the segment data in the scanning direction and generate corrected line segment data. In this apparatus, since it is necessary to input / output block data to / from the second recording medium, the interpolation calculation may be executed in two stages. When the circuit is mounted on a reconfigurable device, the calculation is performed. Reduce the amount of hardware resources required.

  That is, it is desirable that the apparatus includes a reconfigurable unit that can reconfigure a circuit and a control unit that configures a circuit in the reconfigurable unit. The control unit constitutes a first stage circuit including a first input unit, a first arithmetic circuit, and a first output unit in the reconfigurable unit, and the second input unit, And a second stage circuit including a second output unit. The second arithmetic circuit may be included in the first-stage circuit or the second-stage circuit depending on the amount of hardware resources of the reconfigurable unit that can be used for this processing.

  Another aspect of the present invention is a method for controlling an apparatus having a reconfigurable unit capable of reconfiguring a circuit and a control unit configuring the circuit in the reconfigurable unit by the control unit. The method includes generating second data for outputting a second image obtained by affine transformation of the first image from the first data for outputting the first image. Further, generating includes configuring a first stage circuit in the reconfigurable unit and configuring a second stage circuit in the reconfigurable unit.

  The first-stage circuit outputs a plurality of pixels included in the first original scan line and the second original scan line adjacent to each other in the first image from the first recording medium storing the first data. A first input unit that obtains original scanning line data to perform, and an interpolation calculation unit that generates pixel data for outputting a plurality of pixels of the second image from the original scanning line data. The pixel data includes line segment data for outputting a plurality of pixels included in the line segments of the plurality of scanning lines of the second image. Further, the first stage circuit includes a first output unit that stores block data including the respective line segment data in the second recording medium as an input / output unit.

  The second stage circuit includes a second input unit that acquires a plurality of block data including a plurality of line segment data for outputting the third scanning line of the second image from the second recording medium; A second output unit that stores scan line data for outputting a plurality of pixels included in the third scan line in a third recording medium;

One of the further aspects of the present invention generates second data for outputting a second image in which the first image is affine-transformed from the first data for outputting the first image. A method comprising the following steps.
1. Original scan line data for outputting a plurality of pixels included in the first original scan line and the second original scan line adjacent to each other in the first image from the first recording medium storing the first data. To get.
2. Generating pixel data for outputting a plurality of pixels of the second image from the original scanning line data. The pixel data includes line segment data for outputting a plurality of pixels included in the line segments of the plurality of scanning lines of the second image.
3. Block data including each line segment data is stored in the second recording medium as an input / output unit.
4). Obtaining a plurality of block data including a plurality of line segment data for outputting the third scanning line of the second image from the second recording medium;
5). Scan line data for outputting a plurality of pixels included in the third scan line is stored in the third recording medium.

  In the second recording medium, it is desirable to store block data with a fixed length, and it is more desirable to store with block length.

  The second recording medium may store the block data by changing the address so that the block data is read in the order of the scanning lines of the second image. The block data may be acquired from the second recording medium by changing the address so as to read in the order of the scanning lines of the second image.

  FIG. 1 is a basic configuration block diagram of an apparatus for generating image data according to an embodiment of the present invention. This image processing apparatus or generation apparatus 50 is realized by using a device (processing apparatus) 1 capable of reconfiguring a circuit. The function as the generation device 50 is configured in the reconfigurable area 10 of the processing device 1 by being divided into a first-stage circuit 51 and a second-stage circuit 52. The device 50 generates second data D2 for outputting a second image G2 obtained by affine transformation of the first image G1 from the first data D1 for outputting the first image G1. Device. The device 50 includes a memory 59 as a recording medium, and the memory 59 stores first data D1, second data D2, and intermediate stage data MD. The first-stage circuit 51 configured in the reconfigurable area 10 includes a first input unit 61 that reads data from the memory 59, a first interpolation calculation unit 63 that performs interpolation calculation, and a first interpolation calculation unit. The block data generation unit 66 that divides the data generated by (1) into a plurality of blocks (a block), a first output unit 62 that writes these blocks to a predetermined address of the memory 59, and first-stage control information generation Unit 69.

  The first input unit 61 uses the first data D1 in the memory 59 and the adjacent first original scan line and second original scan line (hereinafter referred to as the first line L1 and the first line) of the first image G1. The original scanning line data E1 for outputting a plurality of pixels included in the second line L2) is acquired. The first interpolation calculation unit 63 is a unit for generating pixel data P1 for outputting a plurality of pixels of the second image G2 from the original scanning line data E1. In particular, the first interpolation calculation unit 63 of this example performs interpolation in the sub-scanning direction from the data of the first and second lines included in the original scanning line data E1, and obtains the pixel data P1 of the second image G2. Generate. As will be described later, the pixel data P1 generated by interpolating the data of the first and second lines of the first image G1 is included in the line segments LS of the plurality of scanning lines of the second image G2. Line segment data LSD for outputting a plurality of pixels is included. The first output unit 62 stores the block data including the respective line segment data LSD in the memory 59 as a part of the intermediate stage data MD in the input / output unit.

  The second-stage circuit 52 configured in the reconfigurable area 10 includes a second input unit 71 that reads line segment data LSD from the memory 59, a second interpolation operation unit 73, and the read line segment data. A line data generation unit 76 for generating scanning line data to output the second image G2 from the LSD, a second output unit 72 for storing the second data D2 in the memory 59, and control information generation at the second stage A unit 79.

  The second input unit 71 sequentially acquires a plurality of block data including a plurality of line segment data LSD for outputting the third scanning line L3 of the second image G2 from the memory 59. The second interpolation calculation unit 73 performs interpolation in the scanning direction to correct or regenerate the line segment data LSD. The second output unit 72 stores the scanning line data for outputting a plurality of pixels included in the third scanning line L3 in the memory 59 as a part of the second data D2.

  The control information generation units 69 and 79 appropriately determine a block size for input / output to / from the memory 59 according to a rotation angle given by a user or an application program, and provide information for generating or managing input / output addresses. Provide to each unit.

  Referring to FIGS. 2 to 4, in this generation apparatus 50, the original image (first image) indicated by coordinates xy is affine transformed to generate a second image indicated by coordinates wz. The basic algorithm to do is explained. The generating device 50 is intended to perform high-speed micro-angle rotation, for example, affine transformation (micro-angle affine transformation) with a rotation angle θ within about ± 2 degrees. For example, when an original image (first image) G1 of A4 size (210 mm × 297 mm) is read by a scanner with a resolution of 300 DPI, image data (first data) D1 having 2520 × 3560 pixels (9M pixels in total) is obtained. can get. The luminance data for displaying each pixel is 8 bits. Here, the second image G2 is rotated by a certain angle θ with the short side (X direction) of the first image G1 as the scanning direction, and the W direction is the scanning direction (scanning direction, Z direction is the sub-scanning direction). Get.

  As shown in FIG. 2, by interpolating data of a plurality of pixels included in two scanning lines (lines) OL1 and OL2 that are adjacent to the first image G1 in the sub-scanning direction Y and extend in the scanning direction X, A pixel of the second image G2 is generated. In this case, a plurality of scanning lines (hereinafter referred to as lines or new lines) L (n) between the two scanning lines (hereinafter referred to as original lines) OL1 and OL2 of the first image G1. To L (n + i) line segment LS (n (1)) to line segment LS (n + i (i + 1)) (i and n are integers) are generated. In FIG. 2, line segments LS (n (1)), LS (n + 1 (2)) and LS (n + 2 (3)) of three lines Ln to Ln + 2 are included between the original lines OL1 and OL2. Is schematically shown. In the first data D1 of 300 DPI, there are 44 scanning lines of the second image G2 between the adjacent original scanning lines OL1 and OL2 of the first image G1 when the rotation angle θ is 1 degree. When the rotation angle θ is 2 degrees, 88 scanning line segments LS are included.

  When the rotation angle θ is 2 degrees, the generation device 50 reads original scanning line data E1 including luminance data of pixels included in the original lines OL1 and OL2 of the image data D1 of the first image G1. Next, the line segments LS of the 88 converted scanning lines are calculated, and each line segment data LSD is stored in the memory 59 as block data having a fixed length of 8 words. That is, the luminance data of 29 or 28 pixels is included in one line segment LS, and one line segment data LSD is 232 bit or 224 bit data. For this reason, the generation device 50 pads a portion less than 8 words (256 bits) and stores it in the memory 59 as block data having a fixed length of 8 words. The block data of 8 words is one of the burst units of the memory 59 and can be input / output at high speed.

  As shown in FIG. 3, the generation device 50 continuously reads the original scanning line data E1 in the sub-scanning direction Y from the image data (data file) D1 in the memory 59, and scans the main scanning direction and the scanning line direction. ) Interpolate X to generate line segment data, convert it into block data, and store it in the memory 59 as intermediate data (data file) MD. Two lines of original image data E1 may be read from the image data D1. Also, a line buffer is prepared in the generation device 50, the original image data E1 for one line is read, and the original image for two lines is substantially combined with the original image data for the previous line remaining in the line buffer. Data may be acquired. The generation device 50 repeats this process for one image data D1 of the first image G1, generates line segment data included in the second image G2, and stores it in the intermediate data file MD. The order in which the line segment data LSD is generated and stored in the intermediate data file MD is, for example, LSD (n (1)), LSD (n + 1 (2)), LSD (n + 2 (3)), LSD (n + 1 (1) ), LSD (n + 2 (2)), LSD (n + 3 (3)).

  As illustrated in FIG. 4, the generation device 50 reads block data including line segment data from the intermediate data file MD in the memory 59 according to the scanning line L3 of the second image G2. For example, the order of reading from the intermediate data file MD is LSD (n (1)), LSD (n (2)), LSD (n (3)), LSD (n + 1 (1)), LSD (n + 1 ( 2)), LSD (n + 1 (3)). The generation device 50 further generates one scanning line data from the read line segment data LSD, stores it in the image data file D2 of the memory 59, and outputs image data D2 for outputting the second image G2. Is generated.

  As shown in FIG. 5, the line segment data LSDn (1) is added to the intermediate data file MD as pad data pd (n (1)) having a fixed-length burst size (8 words) by adding padding data pd. Stored. The same applies to the other line segment data LS. Therefore, input / output of individual block data BD to / from the memory 59 is performed at high speed. At the same time, the desired block data BD can be randomly stored at an appropriate address in the memory 59 by designating the input / output address, and the desired block data BD can be randomly acquired. This input / output is not a random access to the memory 59 but a burst access in units of blocks, and the input / output to the memory 59 is performed at the highest speed.

  FIGS. 6A and 6B show two examples of storing block data BD in the intermediate data file MD. FIG. 6A shows that the block data BD can be read sequentially from the intermediate data file MD when generating the scanning line data of the second image G2. In FIG. 6B, the block data BD is sequentially stored in the intermediate data file MD in the order in which the line segment data LSD is generated from the scanning line data of the first image G1.

  When an original image (first image) G1 of A4 size (210 mm × 297 mm) is read by a scanner with a resolution of 300 DPI, if the rotation angle θ is ± 2 degrees, as described above, one line segment data LSD is Since it is 232 bits or 224 bits, it is desirable to pack it into fixed length block data BD having a burst size of 8 words. If the rotation angle θ is ± 1 degree, the data amount of one line segment data LSD is doubled, so it is desirable to pack 16 words into fixed-length block data BD that is burst-sized. The use efficiency and the access efficiency of the memory 59 are high. The line segment data LSD may be stored using two 8-word fixed-length block data BD instead of the 16-word fixed-length block data BD.

  When the rotation angle θ is between ± 2 degrees and ± 1 degree, one line segment data LDS may be packed into 16-word fixed-length block data BD, and divided into two 8-word fixed-length block data BD. You may pack. When the rotation angle θ is between ± 1 degree and 0 degree, it is desirable that one line segment data LDS is divided into two or more 16-word fixed length block data BD and packed. When the rotation angle θ exceeds ± 2 degrees, one line segment data LSD can be packed into 8-word fixed-length block data BD. However, as the rotation angle θ increases, the padding data dp increases and the use efficiency of the block data BD decreases. In this specification, the description is based on the fact that one block data BD does not include a plurality of line segment data LSD. That is, one line segment data LSD is included in one or a plurality of (usually two) block data BDs associated therewith. Although it is not excluded to pack a plurality of line segment data LSD into one block data BD, the number of times of accessing the memory to input / output one line segment data LSD tends to increase. This method and apparatus is suitable for image processing with a small rotation angle θ, for example, ± 2 degrees or less.

  FIG. 7 is a flowchart showing processing in the generation device 50. First, in step 81, the first stage circuit 51 is reconfigured in the reconfigurable area (reconfigurable unit) 10 of the reconfigurable processing device 1. The first stage circuit 51 includes a first input unit 61, a first interpolation operation unit 63, a block data generation unit 66, a first output unit 62, and a control information generation unit 69. In this example, the process of generating pixel data for outputting a plurality of pixels of the second image G2 from the scanning line data of the first image G1 is executed in a first stage and a second stage. The Therefore, the interpolation calculation unit for generating the pixel data is reconfigured by being divided into the first interpolation calculation unit 63 and the second interpolation calculation unit 73.

  In step 81, the first input unit 61 acquires the original scanning line data E 1 from the first data file D 1 in the memory 59. In step 82, the first interpolation calculation unit 63 generates scan line segment data LSD for outputting the second image G2 from the original scan line pixel data included in the original scan line data E1. .

  8 to 10 show an outline of the first interpolation processing performed in step 82. FIG. As shown in FIG. 8, the first interpolation processing is performed by using two pixel data (luminance data) P (n−1, i) and P (n, i) of the first image G1 adjacent in the sub-scanning direction Y. Are weighted and interpolated to generate pixel data P ′ (n, i) of the second image G2. The interpolation formula is the formula (1) shown in FIG. The parameter Weight is determined by the rotation angle θ. Further, as shown in FIG. 10, when the turning angle θ is defined around the leftmost pixel of the original scanning line OL, the block leading offset a is zero for the leftmost line segment LS. You may turn around the pixel at the center of the original scanning line OL, or turn around the pixel at the right end. In this case, the offset a is zero for the pixel that is the center of rotation.

  In this example, the leftmost line segment LS (1) is configured by four pixels P ′ (n, 0) to P ′ (n, 3). Therefore, the line segment data LSD includes four pixel data, which are calculated in parallel using the arithmetic elements of the reconfigurable area 10. The control information generation unit 69 determines the number of line segments of the second image G2 generated between the original scan lines and the number of pixels included in one line segment according to conditions such as the rotation angle θ, resolution, and image size. Provides information necessary for interpolation calculation and blocking, such as (number of dots). In step 83, if the block data BD having a predetermined length can be configured by only the pixel data, the block data generating unit 66 collectively sets the pixel data P ′ of the number of pixels to be processed as the block data BD. When pixel data alone is insufficient, padding data is added to generate fixed-length (burst size) block data BD. In step 83, the first output unit 62 further stores the block data BD in the predetermined order in the intermediate data file MD of the memory 59 according to an instruction (address) from the control information generation unit 69. To do. An example of the order of storing is the method described in FIGS. 6 (a) and 6 (b). The first output unit 62 accesses the memory 59 in a burst mode and outputs data at a high speed.

  In step 84, steps 81 to 83 are repeated until the original scanning line data included in the image data D1 of the first image G1 is completed. When the weighted interpolation processing using the original scanning line data included in the image data D1 is completed, the second-stage circuit 52 is reconfigured in the reconfigurable area 10 of the reconfigurable processing device 1 in step 85. To do. The reconfigurable device 1 on which the generation device 50 is mounted (mounted) is a dynamic reconfigurable device, and the circuit configuration of the reconfigurable area 10 is changed from the first stage circuit 51 to the second stage circuit 52. Can be changed in one clock (cycle). The second stage circuit 52 includes a second input unit 71, a second interpolation operation unit 73, a line data generation unit 76, a second output unit 72, and a control information generation unit 79.

  In step 85, the second input unit 71 converts the block data BD including the line segment data LSD from the intermediate data file MD of the memory 59 to the scanning line after rotation based on the address generated by the control information generation unit 79. Read in the order they are included. The second input unit 71 accesses the memory 59 in the burst mode and reads (acquires) the block data BD at a high speed. If the pad data pd is included in the block data BD, the second input unit 71 extracts the line segment data LSD from the block data BD or removes the padding data pd, and sends it to the second interpolation calculation unit 73. Supply. In step 87, the second interpolation calculation unit 73 performs an interpolation calculation (second interpolation) in the scanning direction (scanning line direction) W.

  11 and 12 show an outline of the first interpolation processing performed in step 87. FIG. As shown in FIG. 11, the second interpolation process weights and interpolates pixel data P′n (i) and P′n (i + 1) adjacent to each other in the scanning direction W after the rotation to obtain a second image. Pixel data P ″ n (i) for outputting G2 is generated. The interpolation formula is the formula (2) shown in FIG. When 300 DPI and the rotation angle θ is 2 degrees, the difference between the pixel position in the X direction of the original scanning line and the pixel position in the W direction of the rotated scanning line is less than one pixel per line (less than 0.1%) So it is almost negligible. Therefore, this second interpolation process may be omitted. When omitted, the second image G2 displays the brightness at a position swollen by about 0.1% in the scanning direction W, so that the second image G2 is an image contracted by about 0.1% in the scanning direction W. Become. In the second interpolation process, luminance data (pixel data) is interpolated in accordance with the pixel position in the scanning direction W, and expansion and contraction of the rotated second image G2 are suppressed.

  In step 88, the line data generation unit 76 combines the pixel data P ″ subjected to the second interpolation processing in the order of the scanning direction W, and generates rotated scanning line data (line data). Further, in step 88, the second output unit 72 stores the scan line data in the second data file D2 of the memory 59 in order. In step 89, steps 86 to 88 are repeated until the block data BD stored in the intermediate data file MD is completed. In this way, the generation device 50 generates image data D2 for outputting the second image G2 obtained by rotating the first image G1 by the angle θ. In this example, the common memory 59 is used as a recording medium for storing image data or the like, but different recording media may be used. For example, a RAM element included in the reconfigurable area 10 may be a storage destination of the intermediate data file MD.

  FIG. 13A shows an example of a reconfigurable processing apparatus (reconfigurable device) 1. This device 1 is a semiconductor integrated circuit device called DAPDNA developed by the applicant of the present application. The device 1 includes a RISC core module 2 called DAP and a dynamic reconfigurable data flow accelerator 3 called DNA. In addition to DAP2 and DNA3, the device 1 has a DNA4 direct input / output interface 4, a PCI interface 5, an SDRAM interface 6, a DMA controller 7, and other peripheral devices 8, and a high speed for connecting them. Switching bus (internal bus) 9. The PCI interface 5 and / or the SDRAM interface 6 can be connected to a large-capacity recording medium such as a RAM disk 59b, a flash memory 59a, etc., and can store image data.

  The DAP2 includes a debug interface 42a, a RISC core 42b, an instruction cache 42c, and a data cache 42d. The DNA 3 is used to reconfigure the PE matrix 10 by changing the function and / or connection of the PE matrix 10 in which 376 PEs (PEs, processing elements) are two-dimensionally arranged and the PEs included in the PE matrix 10. And a configuration memory 19 in which configuration data 18 is stored. The PE matrix 10 corresponds to a unit that can reconfigure a circuit, that is, a reconfigurable area. Further, the configuration of the PE matrix 10 is controlled by the DAP 2, and the DAP 2 corresponds to a control unit.

  The configuration memory 19 has a plurality of banks. For example, as shown in FIG. 13B, in the PE matrix 10, a first function (data flow, circuit design) 17a is configured by configuration data 18 stored in the foreground bank. Further, the second function 17b and the third function 17c are configured by configuration data stored in different background banks, respectively. By switching the bank of the memory 19, the PE matrix 10 is reconfigured with the second function 17b or the third function 17c instead of the first function 17a. The reconstruction of the PE matrix 10 is dynamically performed in one cycle (clock cycle), for example. Thus, the PE matrix 10 is a reconfiguration unit including a plurality of elements for configuring a circuit and internal wirings for connecting these elements. By changing the connection of the elements by the internal wirings, the PE matrix 10 10 can be reconfigured.

  FIG. 13C is an example in which a circuit is reconfigured in the PE matrix 10. A plurality of functions (subfunctions) obtained by time-division of an application such as an MPEG decoder are reconfigured in the PE matrix 10 by time-division, and the functions of the MPEG decoder are provided by a dedicated circuit (dedicated hardware). By using such a device, an application requiring a large amount of hardware resources can be executed with a small amount of hardware resources using the device 1 which is a reconfigurable data processing apparatus.

  FIG. 13 (d) is another example of configuring a circuit in the PE matrix 10. In order to execute applications having different playback methods, the PE matrix 10 can be reconfigured so that a plurality of functions are realized. Through such use, many applications can be executed using the common hardware (device) 1. Since this device 1 can be implemented by switching a large number of functions at the data flow level (data path level, hardware level) instead of the program level (instruction level), processing can be performed at a speed comparable to that of dedicated hardware. Can do.

  FIG. 14 shows the arrangement of the PE matrix 10 in an enlarged manner. The processing elements PE as a whole are arranged so as to form a 16 × 24 matrix. Some PEs occupy the space of two PEs, and 376 PEs are arranged in the PE matrix 10 as a whole. These PEs are further divided into six groups each consisting of 8 × 8 PEs. These groups are referred to as segments S, and segments S0 to S5 are arranged in order from the upper left to the lower right of the PE matrix 10. The PEs included in each of the segments S0 to S5 are connected by an intra segment connection that can transmit and receive data within a delay of one cycle. Also, among the segments S0 to S5, adjacent segments are connected by an intersegment connection via a delay element described later.

  Among PEs shown in FIG. 14, PEs beginning with “EX” are called EXE elements, and are elements for operations including arithmetic operations, logical operations, and 2-input comparison functions. “EXC” has a CMPSB instruction, “EXF” has a FF1 instruction, “EXM” has a multiplication instruction, “EXR” has a BREV instruction, and “EXS” has a BSWAP instruction It also includes a calculation function unique to each type.

  PEs beginning with “DL” are delay elements, and can each set a delay of 1-8 clocks. “DLE” is for data delay within a segment, “DLV” is for data transmission / reception between vertical segments, “DLH” is for data transmission / reception between horizontal segments, and “DLX” is It is an element for data transmission / reception between vertical and horizontal segments.

  The PE displayed as “RAM” is an element (memory element) used as an internal memory of DNA. The PE displayed as “LDB” is a DNA internal buffer for data input, and is used to configure an input interface or an input unit. The PE displayed as “STB” is a DNA internal buffer for data output, and is used to configure an output interface or an output unit. The PE displayed as “C16E” is an address generation element for the DNA internal buffer. The PE displayed as “C32E” is an address generation element for the external memory space. The PE displayed as “LDX” is an element for data input from the DNA direct I / O. The PE displayed as “STX” is an element for outputting data to the DNA direct I / O. In the PE matrix 10, LDB and LDX can be used as input interfaces for inputting data from outside, and STB and STX can be used as output interfaces for outputting data to the outside.

  FIG. 15 is a block diagram showing a schematic configuration of an EXM element (“EXM”) as an example of PE. The EXM element includes an ALU 11a, a MUL (16 × 16) 11b, an FF 11c, and the like. This EXM element is configured to execute an arithmetic operation, a logical operation, a two-input comparison function, a multiplication instruction, or a compound instruction by using configuration data 18 stored in the configuration memory 19 of DNA3. Can be configured. In addition, since a plurality of FFs 11c are built in, it is possible to control the latency from data input to output to the element PE, and in a configuration where the number of delay elements (DLE) is insufficient, the function as a delay element. It is also possible to set

  FIG. 16 is a block diagram showing a schematic configuration of a RAM element (“RAM”) as another example of PE. This RAM element is a data storage memory element, which is a 16 KB (32 bits × 4096 words) RAM module 12a, an address input address register (FF) 12b, a latch 12c, and a data input write data register (FF). ) 12d, a latch 12e, and a read data register (FF) 12f for data output. The read / write control of the RAM module 12a is performed by the token value input together with the address data and / or read data. From the address input to the read data output, it is possible in about 3 clock cycles like the EXE element, and data input / output is possible with the same latency as other types of PE included in the PE matrix 10. is there. This RAM element has a 32-bit mode, a dual-port 32-bit mode, a FIFO mode, a 16-bit mode, an 8-bit mode, and an FSM (feedback state mode) based on configuration data 18 stored in the configuration memory 19 of DNA3. Can be configured to input and / or output data.

  The EXE element and the counter element C16E and / or C32E can be used to generate the access address of the RAM element, and can be input to the RAM element through the routing matrix (matrix bus) of the PE matrix 10. Therefore, input / output to / from the RAM element can be controlled by a circuit reconfigured in the PE matrix 10.

  The PE matrix 10 includes a plurality of PEs and a routing matrix (wiring group) 20 for connecting them. The routing matrix 20 is used to connect the first level wiring group (first level routing matrix, intraconnect) 21 for connecting the PEs in the segment S and the adjacent segment S via the delay element. Second level wiring group (second level routing matrix, interconnect). Connection of PEs by the routing matrix 20 can be controlled by the configuration data 18. Therefore, the PE matrix 10 has different circuits (data paths) by changing the function of each of the plurality of PEs and / or changing at least part of the connection of the routing matrix 20 according to the configuration data 18. , Data flow) can be reconfigured.

  FIG. 17 shows an example of the configuration of the first-level wiring group 21 for connecting the PEs inside the segment S. The first level routing matrix 21 includes 128 vertical buses 23 and 64 horizontal buses 24 for connecting 8 × 8 PEs included in the segment S0. The vertical buses 23 are divided into 16 groups, and two V-buses 23x and 23y each including eight buses are paired and arranged along the vertical column of PEs on both sides of the column. Has been. The horizontal buses 24 are divided into eight groups, and H-buses 24 each including eight buses are arranged along the horizontal rows (lines) of the PEs. The V-buses 23x and 23y are provided with 8-1 bus selectors (multiplexers, MUX) 25 corresponding to the respective PEs, and data can be input to the respective PEs.

  The H-bus 24 is provided with 8-1 bus selectors (multiplexers, MUX) 26 corresponding to the intersections of the H-bus 24 and the V-buses 23x and 23y. Therefore, one data set can be output from one H-bus 24 to one V-bus 23 x or 23 y intersecting with the H-bus 24. The reverse is also possible. Each bus included in the H-bus 24 is connected to the output of the PEs of that line. Therefore, PEs included in the segment S can be connected through the V-buses 23x and 23y and the H-bus 24. Data can be connected within the range that can be connected by the first level bus 21 including the V-buses 23x and 23y and the H-bus 24, that is, within one cycle (one clock) between the PEs in the segments S0 to S5. Can send and receive. Therefore, in terms of timing, for example, all PEs included in the segment S0 are equivalent. For this reason, in order to configure a circuit within the same segment, no matter which PE is selected and assigned a function, there is no need to consider the timing. In terms of timing, PEs in the segment are used. The predetermined circuit can be freely arranged and wired.

  By the second level routing matrix, adjacent segments, for example, connection elements DLH included in the segments S1 and S2, respectively, are connected. Each DLH is connected to a first level routing matrix 21 within each segment. Accordingly, PEs included in different segments can be connected via the second level routing matrix. The connection delay element DLH functions as an interface of a bus included in the first level routing matrix 21. Therefore, the buses included in the first level routing matrix 21 can be used independently for each segment. On the other hand, when it is necessary to input / output data between segments, it is necessary to input / output data via a plurality of FFs included in the connection delay element DLH, and there is a delay of two cycles or more in synchronization with the clock. Newly join.

  In this way, when PEs are connected using only the first level routing matrix 21, it is guaranteed that PEs are connected within a range of one cycle (first delay), and timing verification is unnecessary. It is. On the other hand, when connecting PEs via the second level routing matrix, a delay of two cycles or more is added. The delay when connecting via the second level routing matrix depends on the setting of the delay element DLH. For example, by controlling the delay amount of DLH, it is possible to synchronize a signal that uses the second routing matrix twice and a signal that is used once. The same applies when adjacent via the other delay elements DLV and DLX for connection.

  FIG. 18 shows an example of the first stage circuit 51 configured in the PE matrix 10 which is a reconstruction area. In this example, the first input unit 61 uses the input buffer LDB to acquire the original scanning line data E1 in units of lines, and further uses the RAM element as a line buffer to transfer data to the first interpolation operation unit 63. It is configured to supply data for a line. In the first interpolation arithmetic unit 63, an arithmetic circuit including multiplication and addition is configured using EX elements. In the first interpolation operation unit 63, a circuit is configured such that processing for generating data of a plurality of pixels of the line segment data LSD included in the block data BD is performed in the pipeline. Therefore, following the reading of the original scanning line data E1, the line segment data LSD is continuously output in a pipeline manner, for example, every cycle.

  In the block data generation unit 66, a circuit including a selector is configured using EX elements. In the first output unit 62, a circuit is configured using the output buffer STB. In the block data generation unit 66 and the first output unit 62, circuits for handling two systems of block data BD are prepared in parallel. This is because it takes time to add the padding data pd when the line segment data SLD is changed to the block data BD, so that the pipeline may break down. However, the block data BD can be stored in the memory 59 without waiting time by burst writing the block data BD to the memory 59 from the first output unit 62.

  In the control information generation unit 69, a counter for counting coordinates using the EX element is configured, and a constant table is configured using the RAM element. The constant data includes line segment information determined by the resolution and the rotation angle θ, various parameters for interpolation calculation, information for blocking the line segment data, and the like.

  FIG. 19 schematically shows how data is transferred at main points of the circuit 51 in the first stage. The first input unit 61 continuously obtains original scanning line data from the first image data D1 (point A), and outputs data LD1 and LD2 for two lines using a buffer (point B). In the block data generation unit 66, the padding data pd is added to the line segment data LSD continuously output from the first interpolation operation unit 63, and the block data BD is output (point C). The first output unit 62 burst writes the block data BD to a predetermined address in the memory 59.

  FIG. 20 shows an example of the second stage circuit 52 configured in the PE matrix 10. In this example, the second input unit 71 uses the input buffer LDB to acquire block data BD in the order of the scanning lines, and further uses the DEL element to obtain data of two pixels adjacent in the scanning direction W as the second. The interpolation calculation unit 73 is supplied. In the second interpolation operation unit 73, an operation circuit including multiplication and addition is configured using EX elements. In the second interpolation operation unit 73, a circuit is configured such that processing for generating data of a plurality of pixels of the line segment data LSD included in the block data BD is performed in a pipeline manner. Therefore, the line segment data SLD subjected to the second interpolation processing is continuously output in a pipeline manner, for example, every cycle.

  In the line data generation unit 76, a circuit including a selector is configured using the EX element, and the padding data pd included in the block data BD can be removed by gating. The DEL element is for waiting for control information. In the second output unit 72, a circuit is configured using the output buffer STB.

  In the control information generating unit 79, a counter for counting coordinates using the EX element is configured, and a constant table is configured using the RAM element. Similar to the first-stage circuit 51, the constant data includes information about line segments determined by the resolution and the rotation angle θ, various parameters for interpolation calculation, and line segment data LSD from the block data BD. Information etc. are included.

  FIG. 21 schematically shows how data is transferred at main points of the circuit 52 in the second stage. The second input unit 71 outputs block data BD acquired from a predetermined address in the memory 59 and two systems of block data BD obtained by shifting the block data BD by one or more clocks (point B). The line data generation unit 76 removes the padding data pd from the block data output from the second interpolation operation unit 73 and outputs only the line segment data LSD. The second output unit 72 stores the line segment data LSD in order in the memory 59, and generates the second image data D2.

  As shown in FIG. 22, when the first image G1 is simply rotated, it looks bad. Therefore, the line data generation unit 76 has a function as a rotated image adjustment unit. That is, the regions R1 to R4 in FIG. 22 look bad in the display after rotation and are considered unnecessary. For this reason, the line data generation unit 76 does not output (delete) the data in the regions R1 to R4 using the gating function for cutting the padding data Pd. This process is performed based on the control information output by the control information generating unit 79.

  FIG. 23 is a basic configuration block diagram illustrating another example of the generation device 50. This generation apparatus 50 is also realized by using a processing apparatus (reconfigurable device) 1 capable of reconfiguring a circuit. The first-stage circuit 51 configured in the PE matrix 10 which is a reconfigurable area includes a first input unit 61, an interpolation calculation unit 65, a first output unit 62, and a control information generation unit 69. Including. The interpolation calculation unit 65 includes a first interpolation calculation unit 63, a second interpolation calculation unit 73, and a singular point processing unit 64. As described above, the first interpolation calculation unit 63 and the second interpolation calculation unit 73 may perform interpolation processing in the scanning direction for each line segment LS. Further, luminance data (pixel data) of a pixel after rotation belonging to a rectangular region surrounded by four pixels that are two-dimensionally adjacent can be obtained by interpolation using a distance of four pixels as a weighting parameter. This rectangular area may include not only one rotated pixel but also 0 to 2 rotated pixels. The case where 0 or 2 are included is called a singular point. When only one pixel after rotation is included, it can be handled only by the processing described above. However, when there are 0 or 2 pixels after rotation, singularity processing is required.

  The second stage circuit 52 includes a second input unit 71, a line data generation unit 76, a second output unit 72, and a control information generation unit 79. Since the interpolation operation for generating the rotated pixel data is all performed by the first-stage circuit 51, the second-stage circuit 52 reads the block data BD in the order of the scanning direction W after the rotation, It will only be output as data. In this example, the second stage circuit 52 includes an address generation function for reading the block data BD in the scanning direction W, thereby reducing the hardware load of the first stage circuit 51. That is, the first output unit 62 stores the block data BD in the intermediate data file MD in the memory 59 in the order of the scanning direction X before the rotation. The second input unit 71 performs address generation, acquires block data BD in a jump, and acquires block data BD from the memory 59 in the order of the scanning direction W after rotation.

  As described above, these generation devices 50 need only perform burst access and serial access to the memory 59 in order to generate an affine-transformed image. There is no need to randomly access the memory 59 in order to acquire individual pixel data. Therefore, it is possible to suppress the processing time for inputting / outputting data from becoming a critical path and / or bottleneck of the processing time for converting image data. For example, as shown in FIG. 19, it is not necessary to stop the interpolation calculation process in order to write the block data BD to the memory 59, and input / output to the memory is prevented from becoming a critical path. Therefore, it is possible to shorten the time for image processing that requires affine transformation.

  The image processing method disclosed above is suitable for image processing including minute rotations of ± several degrees or less in the case of an image having a resolution of about 300 DPI in terms of memory utilization efficiency and the like. Image processing with such a small rotation is often used for skew angle correction and the like, and it is effective to increase the image processing speed. The image processing disclosed above can also be applied to image processing involving a large rotation of ± several degrees or more. It is also possible to rotate the image by a desired angle by repeating the minute rotation.

  In addition, the above generation apparatus is divided into a first-stage circuit for writing block data to the memory and a second-stage circuit for reading block data from the memory, and is a reconfigurable device, in particular a dynamic reconfigurable device. It is realized using. However, the first-stage circuit and the second-stage circuit may be configured at the same time, or these circuits may be fixedly mounted using FPGA or ASIC. Further, input / output of block data to / from the memory can be performed in parallel at an appropriate processing interval by using a double port memory or the like.

The block diagram of the production | generation apparatus of embodiment of this invention. The figure which shows that a line segment is produced | generated by performing an affine transformation. The figure which shows producing | generating the data of the line segment after conversion by interpolating the data of the scanning line before conversion. The figure which shows producing | generating the data of the scanning line after conversion combining the data of the line segment after conversion. The figure which shows converting the data of a line segment into block data. FIGS. 6A and 6B are diagrams showing different examples of storing block data in a memory. The flowchart which shows the process of a production | generation apparatus. The figure which shows a mode that the data of a line segment are produced | generated by interpolating in a subscanning direction. The figure which shows the interpolation type | formula of a subscanning direction. The figure which shows a mode that the data of the line segment of the left end when it rotates centering on the left end of a scanning line are produced | generated. The figure which shows a mode that the data of a line segment are produced | generated by further interpolating in a scanning direction. The figure which shows the interpolation formula of a scanning direction. FIG. 13A shows a schematic configuration of an example of a reconfigurable device, FIG. 13B shows an overview of a PE matrix (reconfigurable region), and FIGS. 13C and 13D ) Shows how the PE matrix is dynamically reconfigured. The figure which shows the type of PE arrange | positioned at PE matrix. The block diagram which shows the structure of one type of EXM of PE. The block diagram which shows the structure of one type of RAM of PE. The figure which shows the wiring (intra segment wiring) in a segment. The block diagram which shows an example of the circuit of a 1st step. The figure which shows a mode that data are transferred in the circuit of a 1st step. The block diagram which shows an example of the circuit of a 2nd step. The figure which shows a mode that data are transferred in the circuit of a 2nd step. The figure which shows deleting the part which protruded when the image was rotated. The block diagram which shows the example from which the production | generation apparatus of embodiment of this invention differs.

Explanation of symbols

1 Dynamic Reconfigurable Device 10 PE Matrix (Reconfiguration Unit)
50 generation device (image processing device)
51 First Stage Circuit 52 Second Stage Circuit

Claims (15)

An apparatus for generating, from first data for outputting a first image, second data for outputting a second image obtained by affine transformation of the first image,
An original scan for outputting a plurality of pixels included in the first original scan line and the second original scan line adjacent to each other in the first image from the first recording medium storing the first data. A first input unit for obtaining line data;
An interpolation calculation unit that generates pixel data for outputting a plurality of pixels of the second image from the original scanning line data;
The pixel data includes line segment data for outputting a plurality of pixels included in a plurality of scanning line segments of the second image,
A first output unit that stores block data including each line segment data as an input / output unit in a second recording medium;
A second input unit for acquiring a plurality of block data including a plurality of line segment data for outputting a third scanning line of the second image from the second recording medium;
And a second output unit that stores scan line data for outputting a plurality of pixels included in the third scan line as the second data in a third recording medium.   2. The apparatus according to claim 1, wherein the first output unit stores the block data in the second recording medium with a fixed length.   3. The apparatus according to claim 2, wherein the first output unit stores the block data in the second recording medium with a burst length.   The first output unit according to claim 1, wherein the first output unit changes the address so that the block data is read in the order of the scanning lines of the second image, and stores the block data in the second recording medium. apparatus.   2. The apparatus according to claim 1, wherein the second input unit obtains the block data from the second recording medium by changing an address so that the block data is read in the order of scanning lines of the second image.   2. The apparatus according to claim 1, wherein the interpolation calculation unit includes a first calculation circuit that generates in parallel data for outputting a plurality of pixels included in each line segment data by interpolation in the sub-scanning direction.   7. The apparatus according to claim 6, wherein the interpolation calculation unit further includes a second calculation circuit that generates in parallel data for outputting a plurality of pixels included in each line segment data by interpolation in the scanning direction. . The first output unit according to claim 7, wherein block data including line segment data output from the first arithmetic circuit is stored in the second recording medium,
The second arithmetic circuit corrects line segment data acquired by the second input unit in a scanning direction to generate corrected line segment data. In Claim 6, it further has a reconfigurable unit that can reconfigure a circuit, and a control unit that configures a circuit in the reconfigurable unit,
The control unit constitutes a first stage circuit including the first input unit, the first arithmetic circuit, and the first output unit in the reconfigurable unit, and as the second stage, An apparatus comprising a second stage circuit including a second input unit and the second output unit. A method of controlling a device having a reconfigurable unit capable of reconfiguring a circuit and a control unit configuring a circuit in the reconfigurable unit by the control unit,
Generating second data for outputting a second image in which the first image is affine-transformed from the first data for outputting the first image;
Said generating is
Configuring a first stage circuit in the reconfigurable unit;
Configuring a second stage circuit in the reconfigurable unit;
The first stage circuit comprises:
An original scan for outputting a plurality of pixels included in the first original scan line and the second original scan line adjacent to each other in the first image from the first recording medium storing the first data. A first input unit for obtaining line data;
An interpolation calculation unit that generates pixel data for outputting a plurality of pixels of the second image from the original scanning line data,
The pixel data includes line segment data for outputting a plurality of pixels included in a plurality of scanning line segments of the second image,
Further, the first stage circuit includes a first output unit that stores block data including line segment data as input / output units in a second recording medium,
The second stage circuit comprises:
A second input unit for acquiring a plurality of block data including a plurality of line segment data for outputting a third scanning line of the second image from the second recording medium;
And a second output unit that stores scan line data for outputting a plurality of pixels included in the third scan line in the third recording medium as the second data. A method of generating, from first data for outputting a first image, second data for outputting a second image obtained by affine transformation of the first image,
An original scan for outputting a plurality of pixels included in the first original scan line and the second original scan line adjacent to each other in the first image from the first recording medium storing the first data. Getting line data,
Generating pixel data for outputting a plurality of pixels of the second image from the original scanning line data;
The pixel data includes line segment data for outputting a plurality of pixels included in a plurality of scanning line segments of the second image,
Furthermore, storing block data including each line segment data in the second recording medium as an input / output unit;
Obtaining a plurality of block data including a plurality of line segment data for outputting a third scanning line of the second image from the second recording medium;
Storing scan line data for outputting a plurality of pixels included in the third scan line as the second data in a third recording medium.   12. The method according to claim 11, wherein the block data is stored at a fixed length in the second recording medium.   13. The method according to claim 12, wherein the block data is stored in a burst length on the second recording medium.   12. The method according to claim 11, wherein the block data is stored in the second recording medium while changing the address so that the block data is read in the order of the scanning lines of the second image.   12. The method according to claim 11, wherein the block data is acquired from the second recording medium by changing an address so as to read in the order of scanning lines of the second image.

Images