JP4933462B2 - Image processing method - Google Patents

Description

  The present invention relates to an image processing method such as image distortion correction.

  For example, methods disclosed in Patent Documents 1 and 2 have been proposed as image processing methods for performing processing such as rotation and distortion correction on an input image.

  The image processing method described in Patent Literature 1 includes a unit that encodes each block by dividing m-line image data read in line units into blocks for every n pixels in the main scanning direction, and (m pixels) × ( The image data is rotated by means for rotating the image data of n pixels) in units of 90 °.

The image processing method described in Patent Document 2 aims to perform rotation processing of image data only by utilizing the line data holding memory and interpolation processing without performing complicated calculations, and by taking in line scanned image data and performing image processing. In the image processing in which rotation is performed, the position where the line connecting the sampling points before rotation in the sub-scanning direction and the line in the main scanning direction after rotation intersect is first interpolated from the image data in the sub-scanning direction before rotation. Sub-interpolation in the main scanning direction after rotation is determined at the position where the sub-scan interpolation image data obtained by processing is arranged, and the position where the rotated main scanning line intersects the line connecting the rotated sampling points in the sub-scanning direction The image data is obtained from the image data by the second interpolation calculation, and the rotated image data is arranged.
JP-A-10-93805 JP 2006-295379 A

  However, the image processing methods described in Patent Documents 1 and 2 can perform processing using a small-capacity memory and reduce the system cost. However, since a special circuit to be accessed is required, affine transformation, etc. The complicated calculation is required. Furthermore, when a plurality of pixels are simultaneously processed by a processor that performs parallel processing, processing cannot be performed unless all the image data necessary for parallel processing of one line data is held in a memory for image processing. Depending on the case, there is a problem that the processing speed cannot be improved.

For example, in the case of correcting an image having distortion as shown in FIG. 12 , when the moving distance of the pixel is short (the center of the image in the case of FIG. 12 ) and when it is far (in the case of FIG. 12 , the left and right edges of the image). There was a problem that the contents of processing had to be changed.

  The present invention aims to solve such problems.

  According to the first aspect of the present invention, a GP command for performing processing for each single pixel of the input image data and a PE command for performing processing for a plurality of pixels at the same time are selected. In the image processing method to be executed, when predetermined processing is performed on the image data, the GP command and the PE command are selected based on conversion position information that is a parameter that moves from the origin to the moving point. An image processing method is characterized in that any one of the processing times is switched to an instruction having a short processing time.

  The invention described in claim 2 is characterized in that, in the invention described in claim 1, the GP instruction and the PE instruction process the image data by different algorithms each having the same processing result. To do.

The invention described in claim 3 is the invention described in claim 1 or 2, wherein the GP command and the PE command are switched for each of a plurality of pixels that can be processed at once by the PE command of the image data. It is characterized by this.

The invention described in claim 4 is the invention described in any one of claims 1 to 3 , wherein the conversion position information is calculated for each of a plurality of pixels that can be processed at once by the PE instruction. It is characterized by.

The invention described in claim 5 is characterized in that, in the invention described in claim 4 , the conversion position information is calculated and held in advance.

  According to the first aspect of the present invention, processing is performed by switching to a command suitable for processing with a shorter processing time out of either the GP command or the PE command based on the conversion position information of the input pixel data. The processing speed, particularly the processing speed for distortion correction and image rotation can be improved.

  According to the second aspect of the present invention, even when two different algorithms are used, the same processing result can be obtained as a result, so that an appropriate processing can be selected.

According to the third aspect of the invention, since the GP instruction and the PE instruction are switched for each of a plurality of pixels that can be processed at once by the PE instruction, the frequency of determination processing of the PE instruction and the GP instruction is reduced, and the processing speed is reduced. Can be improved. Further, it is possible to select an appropriate instruction suitable for the processing for each of a plurality of pixels that can be processed at once by the PE instruction.

According to the invention described in claim 4 , since the conversion position information is calculated for each of the plurality of pixels that can be processed at once by the PE command, the more efficient one for each of the plurality of pixels that can be processed by the PE command at once. The image processing speed can be improved by selecting and executing the command.

According to the fifth aspect of the present invention, since the conversion position information is calculated and held in advance, the processing time and the number of steps at the time of processing of a plurality of pixels that can be processed at a time by a pixel unit or a PE command are reduced Image processing speed can be improved.

[First Embodiment]
Hereinafter, a first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is an explanatory diagram illustrating a configuration of an image on which distortion correction processing is performed as image processing in the present embodiment. FIG. 2 is an explanatory diagram showing SIMD units. FIG. 3 is an explanatory diagram showing the relationship between the image shown in FIG. 1 and SIMD units. FIG. 4 is a block diagram of a SIMD type microprocessor as an image processing apparatus according to the first embodiment of the present invention. FIG. 5 is an enlarged view of a part of the image shown in FIG. FIG. 6 is an explanatory diagram when processing is executed in SIMD units. FIG. 7 is a flowchart showing a selection operation between the GP instruction and the PE instruction when the conversion position information is calculated in advance in SIMD units and stored in a memory or the like. FIG. 8 is a flowchart showing a selection operation between the GP instruction and the PE instruction when the conversion position information is calculated for each SIMD unit when determining the switching between the PE instruction and the GP instruction.

  FIG. 1 shows an image of 2112 pixels in the main scanning direction and N lines in the sub-scanning direction. 2112 pixels in the main scanning direction means 352 pixels × 6, which indicates six SIMD units.

  The SIMD unit means the number of pixels (the number of data) that can be processed at a time when a PE instruction is executed in a SIMD type microprocessor, which will be described later. In the SIMD type microprocessor used in this embodiment, 352 pixels are set once as shown in FIG. Can be processed.

  Therefore, as shown in FIG. 3, by processing one line in SIMD units, the number of processes can be reduced as compared with processing in one pixel unit, and more pixels are processed in a certain time. be able to.

  FIG. 4 shows a SIMD type microprocessor used for the distortion correction processing of this embodiment. The SIMD type microprocessor 2 includes a global processor 3 and a processor element block 4.

  As shown in FIG. 4, the global processor 3 includes a program RAM (Program-RAM) for storing programs executed by the global processor 3 and a data RAM (Data-RAM) for storing operation data. Furthermore, a program counter (PC) that holds the address of the program, G0 to G15 registers that are general-purpose registers for storing data for arithmetic processing, and a stack pointer that holds the address of the save destination data RAM when the registers are saved and restored SP), a link register (LS) that holds the address of the caller at the time of a subroutine call, and LI and LN registers that hold the branch source address at the time of IRQ (interrupt) and NMI (non-maskable interrupt), and the processor status A processor status register (P) is built in. A global processor instruction (GP instruction) is executed using these registers and an instruction decoder, ALU, memory control circuit, interrupt control circuit, external I / O control circuit, and GP operation control circuit (not shown). When a processor element instruction (PE instruction) is executed, an instruction decoder, a register file control circuit (not shown), and a PE operation control circuit are used to control a register file array 7 and an operation array 8 described later. That is, the GP instruction is an instruction executed using an arithmetic circuit or the like in the global processor 3, and the PE instruction is an instruction executed using the processor element block 4.

  The processor element block 4 includes a register file array 7 and an operation array 8.

  The register file array 7 includes register files 71 corresponding to the number of processor elements. The register file 71 includes 32 16-bit registers, and the registers are called R0 to R31 for each PE. Each register has a port for the arithmetic array 8 and can be accessed from the arithmetic array 8 by a 16-bit read / write bus (hereinafter referred to as a register bus). The number of registers shown in the figure is six for each PE due to space limitations.

  The arithmetic array 8 is composed of arithmetic units 81 corresponding to the number of processor elements. The calculation unit 81 includes a 16-bit 7-to-1 multiplexer (7-to-1 MUX) 81a for connection with the register file 71, and the PE register buses are 1, 2, and 3 apart to the left in the PE direction, and 1 to the right. It is possible to connect to the register buses of the PEs 2 and 3 apart from each other and the register bus of the own PE, and select from seven register buses as the operation target. Selection control is performed by the global processor 3.

  A shifter 81b is provided after the 7to1 multiplexer 81a to perform bit shift and bit expansion of data read from the register file 71. Shift control is performed by the global processor 3.

  At the subsequent stage of the shifter 81b, an upper 16-bit ALU 81f, an upper 16-bit A register 81g, an upper F register 81h, a lower 16-bit ALU 81c, an upper 16-bit A register 81d, and an upper F register 81e are provided. . In the operation by the PE instruction, basically, the data read from the register file 71 is input as one side of the high-order 16-bit ALU 81f if it is high-order, for example, and the content of the high-order 16-bit A register 81g is input to the other side. The result is stored in the upper A register 81g. Therefore, an operation is performed between the upper A register 81g and the R0 to R31 registers. The same applies to the lower 16-bit ALU 81c.

  The high-order 16-bit ALU 81f and the low-order 16-bit ALU 81c can independently perform 16-bit operations, and the high-order 16-bit ALU 81f and the low-order 16-bit ALU 81c operate in conjunction with each other, so that a total of 32 bits Arithmetic is also possible. Each operation is controlled by the global processor 3. Since the upper 16-bit ALU 81f and the lower 16-bit ALU 81c are linked, an information transmission path such as a carry is provided between both ALUs.

  That is, the register file 71 and the calculation unit 81 described above constitute a processor element (hereinafter referred to as PE). That is, the SIMD type microprocessor 2 of this embodiment includes 352 sets of register files 71 and arithmetic units 81.

  Next, a method for selectively using (switching) the GP instruction and the PE instruction when performing distortion correction processing as image processing using the SIMD type microprocessor 2 having the above-described configuration will be described.

  FIG. 5 is an enlarged view of a part of the image shown in FIG. Here, five points of S00 (x, y), S10 (x + 1, y), S20 (x + 2, y), S30 (x + 3, y), and S40 (x + 4, y) are the target data (pixels) to be corrected. Will be described as an example. When correcting these five points, the offset values as conversion position information of each data are respectively (0, 0) for S00, (0, 1) for S10, (0, 3) for S20, and (0) for S30. , 4) and S40 are (0, 5). This offset value is obtained in advance from the aberration value of the lens for each pixel.

  In the case of FIG. 5, this offset value is the distance of the pixel to be referred to for correction processing, S00 is 0 pixel in the sub-scanning direction, S10 is 1 pixel in the sub-scanning direction, and S20 is sub-scanning. The distance is 2 pixels in the direction, S30 is the distance of 3 pixels in the sub-scanning direction, and S40 is the distance of 5 pixels in the sub-scanning direction.

  In the case of the SIMD type microprocessor 2 shown in FIG. 4, since the 7to1 multiplexer 81a can access only the data for 3PE before and after itself as described above, the PE can be accessed up to S30, which is the distance that can be accessed at one time. The distortion correction processing can be performed efficiently by executing the command and executing S40 with the GP command. In other words, by executing up to S30 with the PE instruction and S40 with the GP instruction, it is possible to reduce the number of steps and the number of operations required for the distortion correction processing, thereby shortening the processing time.

In this embodiment, the conversion position information is calculated for each SIMD unit, and the PE command and the GP command are switched . Immediately Chi, for each of a plurality of pixels capable of performing processing at a time in PE instruction, and switches between GP instructions and PE instruction.

FIG. 7 is a flowchart showing a selection operation between the GP instruction and the PE instruction when the conversion position information is calculated in advance in SIMD units and stored in a memory or the like. 7 is executed by the global processor 3 of the SIMD type microprocessor 2.

  First, in step S41, conversion position information (a switching condition between the PE instruction and the GP instruction) is calculated for each SIMD unit and stored in the memory before processing, and in step S42, a pixel in one SIMD unit is input. In step S44, it is determined whether to switch between the PE instruction and the GP instruction by referring to the conversion position information for each SIMD unit stored in the memory. If it is switched to the GP command in S43, the process is performed using the GP command in Step S45. If all the pixels have been processed in Step S46, the process ends. If not, the process returns to Step S43.

FIG. 8 is a flowchart showing a selection operation between the GP instruction and the PE instruction when the conversion position information is calculated for each SIMD unit when determining the switching between the PE instruction and the GP instruction. Note that the flowchart of FIG. 8 is executed by the global processor 3 of the SIMD type microprocessor 2.

  First, in step S51, a pixel in one SIMD unit is input. In step S52, conversion position information (a switching condition between a PE command and a GP command) is calculated in the input SIMD unit. In step S53, the calculated conversion position information is calculated. To determine whether to switch between the PE instruction and the GP instruction. When the instruction is switched to the PE instruction, the processing is performed with the PE instruction in step S54. When the switching to the GP instruction is performed in step S53, the GP instruction is performed in step S55. In step S56, the process ends if all pixels have been processed. Otherwise, the process returns to step S51.

  According to the present embodiment, the conversion position information is calculated for each SIMD unit and the PE instruction and the GP instruction are switched, so that the frequency of the PE instruction and GP instruction determination processing is reduced and the processing speed can be improved. . Further, it is possible to select an appropriate instruction suitable for the processing for each of a plurality of pixels that can be processed at once by the PE instruction.

  In addition, the offset value is calculated in advance for each SIMD unit, stored in the memory, and the offset value is referred to when switching between the PE instruction and the GP instruction. Therefore, the instruction that is more efficient for each SIMD unit is selected and executed. By doing so, the processing speed can be improved.

  In addition, since the offset value is calculated each time when switching between the PE instruction and the GP instruction is determined, the processing speed is improved by selectively executing the more efficient instruction for each pixel while suppressing the memory usage. be able to.

[ Second Embodiment]
Next, a second embodiment of the present invention with reference to FIGS. The same parts as those in the first and second embodiments described above are denoted by the same reference numerals and description thereof is omitted. FIG. 9 is an explanatory diagram showing an example when distortion correction processing is performed using a GP instruction. FIG. 10 is an explanatory diagram showing an example when distortion correction processing is performed using a PE instruction. FIG. 11 is a flowchart showing a combination pattern of GP instructions and PE instructions.

When the GP command of FIG. 9 is used, in the target pixel (x + M, y + N), if the offset value calculated in advance for each pixel is (3, 3), the pixel position after processing (x + M + 3, y + N + 3) The pixel value of the target pixel (x + M, y + N) after processing is determined with reference to the pixel value.

Next, when the PE instruction in FIG. 10 is used, the pixel of interest after processing is referred to by referring to the pixel position after processing by searching in an m × n matrix centered on the pixel of interest (x + M, y + N). The pixel value of x + M, y + N) is determined. That is, when using the SIMD type microprocessor shown in FIG. 4, it is possible to search for a range of a maximum of 7 × 7.

In the case of the algorithm shown in FIG. 10 , for example, 10 instructions are required to move one pixel. In the PE instruction, if the SIMD type microprocessor 2 shown in FIG. The processing for 1 SIMD unit (352 pixels) can be completed by 490 times of the command = 490 times, whereas in order to process 352 pixels for 1 SIMD unit in the GP command, 490 × 352 = 172480 times are required.

Further, the algorithm using the GP instruction of FIG. 9 and the algorithm using the PE instruction of FIG. 10 are different, but the same result can be obtained if they are the same input image. Therefore, as shown in FIG. 11, when a plurality of processes are continuously performed, the PE instruction and the GP instruction can be combined. That is, in the processing pattern 1 of FIG. 11 , the interpolation processing is performed with the GP command and then the correction processing is performed with the GP command. In the processing pattern 2, the interpolation processing is performed using the GP command, and then the correction processing is performed using the PE command. In processing pattern 3, after performing interpolation processing with a PE command, correction processing is performed with a PE command. In the processing pattern 4, the interpolation processing is performed using the PE command, and then the correction processing is performed using the GP command. Calculation time can be reduced by combining such combinations for each pixel or SIMD unit of the input image.

  According to the present embodiment, the same result is obtained with the GP instruction and the PE instruction, but processing is performed using different algorithms, so that an appropriate process is selected according to the characteristics of the input image, that is, the converted position information. be able to.

  The embodiment described above is a case where the SIMD type microprocessor 2 shown in FIG. 4 is used, and any microprocessor such as another SIMD type microprocessor capable of executing PE instructions and GP instructions may be used. it can. Of course, in that case, the switching condition (distance of 4 pixels or more, range of 7 × 7, etc.) between the PE instruction and the GP instruction may be changed according to the microprocessor.

  The image processing to which the present invention is applied is not limited to distortion correction processing, and may be used for processing such as image rotation processing.

  The present invention is not limited to the above embodiment. That is, various modifications can be made without departing from the scope of the present invention.

It is explanatory drawing which shows the structure of the image in which distortion correction processing is made as image processing in this embodiment. It is explanatory drawing which shows a SIMD unit. It is explanatory drawing which shows the relationship between the image shown by FIG. 1, and a SIMD unit. 1 is a block diagram of a SIMD type microprocessor as an image processing apparatus according to a first embodiment of the present invention. FIG. 2 is an enlarged view of a part of the image shown in FIG. 1. It is explanatory drawing in the case of performing a process per SIMD unit. It is a flowchart which shows selection operation | movement of GP command and PE command at the time of calculating beforehand conversion position information per SIMD unit, and preserve | saving in memory etc. FIG. It is a flowchart which shows the selection operation | movement of GP command and PE command at the time of calculating conversion position information for every SIMD unit at the time of switching judgment of PE command and GP command. It is explanatory drawing which shows the example at the time of performing a distortion correction process using GP command. It is explanatory drawing which shows the example at the time of performing a distortion correction process using PE command. It is the flowchart which showed the combination pattern of GP command and PE command. It is explanatory drawing which shows the image with lens distortion, and its correction example.

2 SIMD type microprocessor 3 Global processor S 43 Judgment of switching between GP instruction and PE instruction S 44 Processing by PE instruction S 45 Processing by GP instruction

Claims (5)

In an image processing method for selecting and executing a GP instruction for processing each input image data for each single pixel of the image data and a PE instruction for processing a plurality of pixels at a time,
When performing predetermined processing on the image data, based on the conversion position information that is a parameter that moves from the origin to the moving point, the command that has a shorter processing time of either the GP command or the PE command An image processing method characterized by switching and executing.   2. The image processing method according to claim 1, wherein the GP instruction and the PE instruction process the image data according to different algorithms each having the same processing result.   3. The image processing method according to claim 1, wherein the GP command and the PE command are switched for each of a plurality of pixels that can be processed at once by the PE command of the image data. The conversion position information, the image processing method according to any one of claims 1 to 3, characterized in that calculating for each of the plurality of pixels capable of performing processing at a time in the PE instruction. The image processing method according to claim 4 , wherein the conversion position information is calculated and held in advance.

Images