JP4929474B2 - Image processing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To construct an apparatus for quickly processing image correlation operation and space filters and quickly executing a plurality of small image operations simultaneously or combinedly, regarding an image processing apparatus for performing image correlation operation. <P>SOLUTION: The image processing apparatus is provided with: a shared image input bus for transferring data composed of a plurality of pixels from a memory in one cycle; a means for selecting data of specified vertical and horizontal sizes from specified horizontal and vertical start points out of the data of the plurality of pixels transferred to the shared image input bus; an input buffer for storing the selected data; a systolic array for capturing data from the input buffer and performing operation thereof; a synchronizing memory for synchronizing results operated by the systolic array; and a means for pipeline-processing the operation or peak extraction of a plurality of synchronized operation results. <P>COPYRIGHT: (C)2008,JPO&amp;INPIT

Description

  The present invention relates to an image processing apparatus that performs correlation calculation of images.

  In order to cope with the constantly changing surrounding environment, the mobile robot needs to process the image input from the camera in real time to grasp the surrounding environment. The amount of image calculation is enormous, and high-speed processing of frequently used correlation calculation, spatial filter, and feature extraction is required.

  Conventionally, there is a technique for obtaining a correlation value of a photographed image based on fixed reference image data relating to an object to be tracked, detecting the peak position, and tracking the object. In the technique, the correlation calculation between the search image data and the reference image data of m pixels × n pixels is performed a plurality of times if the position of the reference image with respect to the search image is moved, and a correlation value is output to obtain the accumulation, Equivalently, a correlation value between reference image data and search image data for a reference image of am pixels × bn pixels is output at high speed (Patent Document 1).

  In addition, with regard to the technique for performing high-speed processing by LSI that performs parallel operation in a pipelined manner with a systolic array in which PE (processor elements) are arranged in a two-dimensional array in order to process correlation operations at high speed in the former, This will be briefly described with reference to FIGS. 14 and 15.

  FIG. 14A shows the overall configuration of a conventional image processing apparatus. The video input / output circuit handles video input and video output control, and the memory control circuit handles control with an external memory, and captures a video input image into the external memory. When a command is given from the outside, image data is transferred from the external memory to the parallel operation circuit in response to a request from the operation control circuit. The parallel arithmetic circuit has a configuration in which PEs are arranged in a two-dimensional array like the parallel arithmetic circuit of FIG. 14B, and image data is input to the lower left PEn1, and PEn1 → PEn2 → PEn3. , A circuit that performs an operation by flowing data in a pipeline, and outputs a calculation result of an SAD correlation operation of an image expressed by the following equation.

In this configuration, parallel computation is performed by a plurality of PEs, and correlation computation processing can be performed at high speed.
JP-A-10-040392

  When grasping the surrounding environment in real time such as autonomous movement of robots, the former is insufficient in performance, and the latter conventional technology is not sufficient in performance, and further performance improvement is required. The

  With the recent miniaturization of semiconductor processes, more circuits can be mounted on LSIs, and it is conceivable that more PEs are mounted and processed in order to improve performance.

  However, even if the number of elements is simply increased from, for example, 8 × 8 to 16 × 16 in the above-described conventional array configuration, when the correlation calculation is performed on a large reference area of 16 × 16, the improvement in the calculation performance is simply four times. Must not.

  For example, FIG. 14E shows a timing chart when correlation calculation is performed on the reference image 8x8 in FIG. 14C and the search range 16 × 16 in FIG. 14D with an 8 × 8 array configuration. In this calculation, as shown in FIG. 14 (e), it takes 64 cycles to load the reference image, 529 cycles to load the search image, and 10 cycles to output the calculation result, and the total processing time is 603 cycles. It becomes. When the calculation is performed with the reference image size set to 16 × 16, this calculation is performed four times, so that the processing time is 2412 cycles.

  On the other hand, FIG. 15C shows a timing chart when the correlation calculation of the reference image 16x16 and the search range 16x16 in FIG. 15A is executed in the 16 × 16 array configuration. In this calculation, as shown in FIG. 15 (c), it takes 256 cycles to load the reference image, 961 cycles to load the search image, and 10 cycles to output the calculation result, and a total processing time of 1227 cycles. It becomes. Even if 4 times the number of PEs are used, the performance improvement does not increase so much, about twice.

  This is because the systolic array loads and processes data pixel by pixel from the input port, so the image data load time determines the processing time. For this reason, there is a problem that the performance is not improved by increasing the number of PEs, and the operation rate of each PE needs to be improved.

  In image recognition, a plurality of operations with a small image size are often used. For example, in feature extraction, a luminance gradient image is generated using two 7 × 7 spatial filters, and a process for obtaining a covariance matrix is often performed. . There is a need for a method capable of simultaneously executing a plurality of such small-size image operations and cascading the operation results.

  In order to solve these problems, the present invention can process image correlation operations and spatial filters at a high speed, and further constructs hardware for executing a plurality of small image operations simultaneously or in combination. Like to do.

  Claim 1 is a shared image input bus capable of transferring data of a plurality of pixels from an image memory in one cycle, and horizontal / vertical specified from a specified horizontal / vertical start point among input data to each systolic array. A means for selecting size data, a means for synchronizing the operation results of each systolic array, and a means for performing pipeline processing for calculating or peak extraction of the operation results of synchronized systolic arrays. is there. By transmitting data for multiple pixels on the shared image input bus and fetching the data required by each systolic array into the input buffer, the data waiting time is reduced, and each systolic array is stored in the input buffer. By providing means for synchronizing the results of reading and processing the data at each timing, it becomes possible to perform the calculation in synchronism with the timings of the calculation results appearing at different timings.

  According to a second aspect of the present invention, as image input data to each systolic array, means for selecting input data to each systolic array from data from the shared image input bus and data of a calculation result of the systolic array, and each systolic array A divider is provided after the calculation result of the array. The calculation result of the spatial filter using the systolic array and the divider can be used as the input of the systolic array in the subsequent stage.

  Claim 3 performs correlation calculation with a large reference region using a plurality of systolic arrays, Claim 4 performs calculation by cascading spatial filters, and Claim 5 performs normalized correlation calculation Therefore, claim 6 is for performing the feature extraction processing.

  When a conventional systolic array takes data individually, it is necessary to prepare a memory for each data transfer bus and write the same data to each memory, which increases the required memory capacity and power consumption. End up.

  In the present invention, considering that the image areas required by the individual systolic arrays overlap, data transmission for a plurality of pixels is performed on the shared image input bus, and the individual systolic arrays are required. By taking the data into the input buffer, it is possible to reduce the processing time by reducing the data waiting time while reducing the necessary memory capacity using a single memory.

  Furthermore, by providing means (synchronization memory) for synchronizing the results obtained by reading the data from the input buffer at each timing by each systolic array and processing them, the timings of the computation results appearing at different timings It is possible to calculate the evaluation value and extract the peak value by pipeline processing.

  For example, compared to a case where a 16 × 16 size correlation calculation is performed with one systolic array composed of 64 PEs, a 16 × 16 size composed of four systolic arrays composed of 64 PEs in the configuration of the present invention. When performing the correlation calculation, the performance is improved by 3.4 times.

  According to the invention of claim 2, there is provided means for processing the results of the systolic array by cascading them by inputting to another systolic array, and processing that requires a plurality of operations is performed once in the data. It becomes possible to process by input.

  In consideration of the fact that the image areas required by the individual systolic arrays overlap, the present invention requires individual systolic arrays by transmitting data for a plurality of pixels on the shared image input bus. By capturing data into the input buffer, a single memory can be used to reduce the processing time by reducing the data waiting time while reducing the required memory capacity. A means (synchronization memory) that synchronizes the results of reading and processing the data at the timing is provided, and it is possible to perform the calculation by matching the timings of the calculation results that are output at different timings. The calculation of the evaluation value and the peak extraction were realized.

FIG. 1 shows a system configuration diagram of the present invention.
In FIG. 1, an image memory control circuit 1 stores an image (for example, an image taken by a camera attached to a robot) in an image memory 2, or outputs a plurality of pixels to the shared image input bus 3 in one cycle. (See FIG. 2).

The image memory 2 temporarily holds an image taken by the camera.
The shared image input bus 3 is a bus for sending out a plurality of pixels from the image memory 2 in one cycle, and causing the input buffer 41 to take in each predetermined image (see FIG. 2). In the embodiment described with reference to FIG. 2, the shared image input bus 3 is configured to transfer, for example, 4-pixel data in one cycle (see FIG. 2).

  The parallel arithmetic circuit 4 is provided with a control circuit (not shown) for selecting horizontal / vertical size data designated from the horizontal / vertical start point designated from the shared image input bus 3 or via the shared image input bus 3. The acquired image is subjected to parallel calculation (correlation calculation or the like). Here, the image is constituted of an input buffer 41, a systolic array 42, a divider 43, and the like.

  The input buffer 41 selects and fetches a plurality of pixels sent from the image memory 2 in one cycle via the shared image input bus 3 and temporarily stores an image addressed to itself (see FIG. 2).

  The systolic array 42 is a parallel arithmetic circuit (for example, see FIG. 15B described above) in which a large number of PEs (processor elements) are arranged, and is a circuit that performs high-speed arithmetic.

The divider 43 divides the result calculated by the systolic array 42.
The selector 5 selects specified data from the image (data) on the shared image input bus 3 or the image (data) obtained by dividing the result calculated by the systolic array 42 by the divider 43 to select the input buffer 41. To be stored.

  The synchronization memory 6 synchronizes the result calculated by the systolic array 42 (or the result of further division by the divider 43) with the result calculated by the other systolic array 42. It is memory.

  The pipeline operation circuit 7 performs high-speed operation on the data after synchronizing the operation result (or the result of further division) of the systolic array 42 in the synchronization memory 6, in this case, the synchronized system The addition of the calculation results of the trick array 42 and the peak extraction are performed by pipeline processing.

  The calculation result memory 8 is a memory for storing the result calculated by the pipeline calculation circuit 7.

  FIG. 2 is an explanatory diagram (FIG. 1) of the present invention. This shows the operation of each systolic array 42. Four illustrated systolic arrays A, B, C, and D (corresponding to systolic arrays A, B, C, and D in FIG. 1) 42 allocate 8 × 8 partial reference image areas obtained by dividing a 16 × 16 reference image into four. (See (a) in FIG. 2). First, an 8 × 8 partial reference image area in charge of each of the 16 × 16 image areas flowing in the shared image input bus 3 is captured (corresponding to any of 8 × 8 of A, B, C, and D in FIG. 2A). Each one is taken into the input buffer 41). When the search image is loaded, the search image regions (A of (b-1) in FIG. 2B, B of (b-2), (b-3) required for the calculation of each partial region are loaded. C and D of (b-4) are respectively fetched into the input buffer 41, and correlation operations are respectively executed in each systolic array 42. Output data from each systolic array 42 Are synchronized by the synchronization memory 6, and all data are added by the pipeline operation circuit 7 and the operation result is output to the operation result memory 8.

  In this embodiment, as shown in the load / computation flow of FIG. 2C, it takes 64 cycles for loading the reference image and 248 cycles for loading the search image. The output of the calculation result of the systolic array D having the latest timing is 595 cycles after the reference image is loaded, and the output of all the calculation results is 14 cycles after that. From calculation start to calculation output of all results is 709 cycles, and the performance improvement is about 3.4 times. This is because, in the case of the conventional configuration of FIGS. 15 and 16, even if the number of pixels is increased by 4 times from 8 × 8 to 16 × 16, the performance can be improved only by about 2 times. According to the configuration of FIG. 1 and FIG. 2 of the example, when the number of pixels is increased 4 times from 8 × 8 to 16 × 16, a significant performance improvement of about 3.4 times close to 4 times can be achieved.

FIG. 2A shows a state in which the reference image 16 × 16 is divided into four 8 × 8 pixels A, B, C, and D. The load time is
34 (array D reference image loading waiting time) +64 (array D reference image loading waiting time) +66 (array D searching image loading waiting time) +529 (array D searching image loading waiting time) +14 (calculation delay of systolic array 42) = 709 cycles (see the corresponding items in the load / computation flow in FIG. 2C and their numbers for each number).

  Here, 529 is a multiplication result of size 23 × 23 in the areas A, B, C, and D in FIG.

  FIG. 2B shows an example of the systolic array 42. 2B-1 is an example of a search image of the systolic array A, FIG. 2B-2 is an example of a search image of the systolic array B, and FIG. 2B-3 is a systolic array. An example of a search image of C, (b-4) of FIG. 2 shows an example of a search image of the systolic array D, respectively. Here, each search area of A, B, C, and D is 23 × 23 pixels.

  FIG. 2C shows the reference images A, B, C, and D in FIG. 2A and the search images A, B, C, and D in FIG. A flow (waveform) when loading each input buffer 41 via the input bus 3 is shown. Here, the waveform diagram (flow) of the systolic array C is omitted in (c) of FIG. 2 (the same as other B and D).

  FIG. 3 shows a system configuration diagram (part 2) of the present invention. FIG. 3 shows a system configuration example in the case where multiple spatial filters are processed or a difference image between images is generated. More specifically, as image input data to each systolic array 42, input data to each systolic array 42 is selected from data from the shared image input bus 3 and data of calculation results of the systolic array 42. A divider 43 is attached after the operation result of the selector 5 and each systolic array 42. The calculation result output from the systolic array 42 is divided by the divider 43 to calculate the output of the spatial filter, and the output is transmitted to the synchronization memory 6 and the next-stage systolic array 42. The next-stage systolic array 42 similarly performs a spatial filter operation, and further transmits the operation result to the next-stage systolic array 42. In this way, the spatial filter can be processed in a multiplex manner by cascade connection.

  In addition, by synchronizing the calculation results output from each systolic array 42 with the synchronization memory 6 and taking the difference, it is possible to generate a difference image between the images resulting from the spatial filter.

  3A shows the system configuration, and FIG. 3B shows the load / calculation flow corresponding to FIG. 2C described above. When the systolic array 42 is cascade-connected as shown in FIG. 3A, the reference image loading, search image loading, systolic arrays A and B (C, (D is omitted) is loaded as shown in the cycle.

  FIG. 4 shows a system configuration diagram (part 3) of the present invention. This is when the reference image is loaded

Is calculated first. When the reference image is loaded, the three systolic arrays 42

Is calculated by the pipeline arithmetic circuit 7.

The normalized correlation operation of the image is performed by calculating.
These states are described in the parallel arithmetic circuit 4 in the system configuration of FIG. Further, FIG. 4B is the same as (Formula 9) in which the calculation result is described.

  FIG. 5 shows a reference image setting flowchart of the present invention. This is a detailed flowchart when setting (loading) a reference image from the image memory 2 to each input buffer 41 in FIG. 1, for example.

  In FIG. 5, S <b> 1 designates the transfer start coordinates (horizontal / vertical) and size (horizontal / vertical) to the image memory control circuit 1. For example, the image memory control circuit 1 shown in FIG. 1 (or the same applies to FIG. 2 to FIG. 4) is used as a transfer start coordinate of a reference image (see FIG. 2A) from a host not shown. Specify vertical and horizontal coordinates, and horizontal and vertical coordinates as the size. At this time, the shared image input bus 3 in FIG. 1 is assumed to be capable of transferring data of 4 pixels in one cycle.

  S2 designates the capture start coordinates (horizontal / vertical) and size (horizontal / vertical) for each of the parallel arithmetic circuits AD. This is because the parallel arithmetic circuit A.1 in FIG. In the B, C, and D input buffers 41, horizontal and vertical are designated as the start coordinates of the reference image fetched from the shared image input bus 3, and horizontal and vertical are designated as the sizes thereof.

  In S3, the reference image data is sent from the image memory 2 to the shared image input bus 3 here, four pixels per cycle.

  In S4, only the data within the size range from the capture start coordinates is captured in the input buffer. This is because each of the parallel arithmetic circuits A, B, C and D takes in only the reference image data designated in S2 from the reference image data sent to the shared image input bus 3 in S3 and stores it in the input buffer 41. . Thereby, for example, only the reference images of the portions A, B, C, and D in FIG. 1A are captured and stored in the input buffers 41 of A, B, C, and D, for example.

  As described above, the reference image data from the image memory 2 is sequentially transmitted to the shared image input bus 3 in one cycle, and each parallel arithmetic circuit A, B, C, D takes in only the reference image designated by itself. The A, B, C, and D reference images (8 × 8) in FIG. 1A are stored in the input buffer 41 and set in the input buffer 41, respectively.

  FIG. 6 shows a search image setting flowchart of the present invention. This is a detailed flowchart when setting (loading) a search image from the image memory 2 to each input buffer 41 in FIG. 1, for example.

  In FIG. 6, S11 designates the transfer start coordinate (horizontal / vertical) and size (horizontal / vertical) to the image memory control circuit 1. For example, the image memory control circuit 1 shown in FIG. 1 (or the same applies to FIGS. 2 to 4) applies to the horizontal start coordinates of the search image (see FIG. 2B) from a host not shown. Specify vertical and horizontal coordinates, and horizontal and vertical coordinates as the size. At this time, the shared image input bus 3 in FIG. 1 is assumed to be capable of transferring data of 4 pixels in one cycle.

  S12 designates the capture start coordinates (horizontal / vertical) and size (horizontal / vertical) for each of the parallel arithmetic circuits AD. This is because the parallel arithmetic circuit A.1 in FIG. In the B, C, and D input buffers 41, horizontal and vertical are specified as the start coordinates of the search image taken in from the shared image input bus 3, and horizontal and vertical are specified as the sizes thereof.

  In step S13, search image data is sent from the image memory 2 to the shared image input bus 3 in this case, four pixels per cycle.

  In S <b> 14, only the data within the size range from the capture start coordinates is captured in the input buffer 42. This is because each of the parallel arithmetic circuits A, B, C and D takes in only the search image data designated in S12 from the search image data sent to the shared image input bus 3 in S13 and stores it in the input buffer 41. . As a result, only the search images of the portions A, B, C, and D in FIG. 1B are captured and stored in the input buffers 41 of A, B, C, and D, for example.

  As described above, the search image data from the image memory 2 is sequentially transmitted to the shared image input bus 3 in one cycle, and each parallel arithmetic circuit A, B, C, D takes in only the search image designated by itself. The search images (23 × 23) of A, B, C, and D in FIG. 1B are stored in the input buffer 41 and set in the input buffer 41, respectively.

  FIG. 7 is an explanatory diagram of search image transfer (in the case of 31 × 31 pixels) according to the present invention. This shows a state where the search image is transferred to the shared image input bus 3 in the flowchart of FIG.

  In FIG. 7, vertical coordinates = V, V + 1,... Are vertical coordinates for transferring a search image from the image memory, and are 31 pixels here. The horizontal direction is the number of pixels in the horizontal direction of the search image, and here it is 31 pixels.

  Search images expressed as described above (search images A, B, C, and D in (b-1), (b-2), (b-3), and (b-4) in FIG. 2 (23 .Times.23 pixels)) in cycles 1, 2, 3, 4, 5, 6, 7, 8 in the horizontal direction with vertical coordinates = v in FIG. 7, and cycles 9, 10,... With vertical coordinates = v + 1. In addition, it is possible to sequentially send out from the image memory 2 to the shared image input bus 3 every four pixels in one cycle. Then, the search image shown in FIG. 7 sent to the shared image input bus 3 is selectively fetched only by the parallel arithmetic circuits A, B, C and D of FIG. As a result, the search image load of the systolic array A, the B search image load, the C search image load, and the D search image load shown in FIG. It will be. Here, a 31 × 31 area is transmitted from the image memory, and the parallel arithmetic circuit is configured to capture a 23 × 23 area.

FIG. 8 is an explanatory diagram of the present invention. This schematically represents a state in which a correlation function (correlation function in the case of implementing the image size 16 × 16 and the search range 16 × 16) is calculated using the system configuration of FIG. In the figure,
S is a search image R is a reference image Parallel processing circuits A, B, C, and D are parallel processing circuits A, B, C, and D in FIG.
Respectively.

  Here, after the search image and the reference image described above are stored in the input buffer 41 of each of the parallel arithmetic circuits A, B, C, and D from the image memory 2, respectively (see FIGS. 1, 2, and 5 to 7). ), And the systolic arrays A, B, C, and D respectively execute the correlation calculation represented by the equation (a) in FIG. 8 to calculate the result (b) in FIG. 8.

FIG. 9 is an explanatory diagram of the present invention. This schematically represents a state in which the system configuration of FIG. 1 is used to perform a filter operation (filter operation when the filter size 16 × 16 and the processing range 16 × 16 are performed). In the figure,
-K is a filter-I is an image to be processed-Parallel arithmetic circuits A, B, C, D are parallel arithmetic circuits A, B, C, D in FIG.
Respectively.

  Here, after storing the filter (16 × 16 pixels) and the processing range (16 × 16 pixels) described above from the image memory 2 in the input buffers 41 of the parallel arithmetic circuits A, B, C, and D, respectively (see FIG. 1, FIG. 2, and FIGS. 5 to 7), the systolic arrays A, B, C, and D respectively execute the filter operation represented by the expression of FIG. 9A to obtain the result of FIG. 9B. Is calculated.

  FIG. 10 shows a flowchart for setting the filter coefficient / division of the present invention. This shows a flowchart for setting the filter coefficient in the input buffer 21 and setting the divisors C1, C2, C3, and C4 in the divider 43 when performing the filter operation of FIG.

  In FIG. 10, in S21, transfer start coordinates (horizontal / vertical) and size (horizontal / vertical) are designated to the image memory control circuit 1. For example, the horizontal and vertical coordinates and the horizontal and vertical coordinates are designated as the filter coefficient transfer start coordinates from a host (not shown) to the image memory control circuit 1 in FIG. At this time, it is assumed that the shared image input bus 3 in FIG. 1 can transfer data of four pixels in one cycle.

  In S22, the acquisition start coordinates (horizontal / vertical) and size (horizontal / vertical) are designated for each of the parallel arithmetic circuits AD. This is because the parallel arithmetic circuit A.1 in FIG. In the B, C, and D input buffers 41, horizontal and vertical are specified as the start coordinates of the filter coefficients to be fetched from the shared image input bus 3, and horizontal and vertical are specified as the sizes thereof.

  In step S23, the filter coefficients K1, K2, K3, and K4 are sent from the image memory 2 to the shared image input bus 3 here, four pixels per cycle.

  In S24, only the data in the size range from the capture start coordinates is captured in the input buffer. This is because each of the parallel arithmetic circuits A, B, C, and D takes in only the filter coefficient designated in S22 from the filter coefficient K sent to the shared image input bus 3 in S23 and stores it in the input buffer 41. As a result, only the respective filter coefficients are captured and stored in the input buffers 41 of A, B, C, and D.

  In S25, the divisor C is set in the parallel arithmetic circuit 4. This is because a host (not shown) sets the corresponding divisors C1, C2, C3, and C4 in the division circuit 43 of each parallel arithmetic circuit 4, respectively.

  As described above, the filter coefficients are sequentially transmitted from the image memory 2 to the shared image input bus 3 in one cycle, and each parallel arithmetic circuit A, B, C, D takes in and inputs only the filter coefficients designated by itself. The data is stored in the buffer 41, and the divisors C1, C2, C3, and C4 are stored in the divider 43, and the preparation for the filter calculation process is completed.

  FIG. 11 is an explanatory diagram of the present invention. This schematically shows the state of the filter operation cascade under the system configuration of FIG.

  (1) For the input image IO (eye zero), the filter arithmetic processing of the filter A is performed by the parallel arithmetic circuit A in FIG. 1, and the result (output of the divider 43) is generated as an image I1 (the arithmetic is performed in the lower stage). Performs the operation represented by the expression).

  (2) The image I1 generated in (1) is further subjected to the filter operation processing of the filter B by the parallel operation circuit B in FIG. 1, and the image 12 that is the result (output of the divider 43) is generated (the operation is , The operation represented by the lower equation is performed).

(3) The image I1 and the image I2 are subtracted to generate the image I3.
As described above, it is possible to cascade filter operations (filter operations are cascaded such as filter A and further filter B) by the system configuration in FIG. 3 (four filter operations are possible in FIG. 3).

FIG. 12 shows a flowchart for setting the normalized correlation reference image of the present invention.
In FIG. 12, the transfer start coordinates (horizontal / vertical) and size (horizontal / vertical) are designated to the image memory control circuit 1 in S31. For example, the image memory control circuit 1 shown in FIG. 1 sends the horizontal and vertical coordinates as the transfer start coordinates of a reference image (see FIG. 2B) from a host (not shown) and the horizontal and vertical sizes as the transfer coordinates. Specify coordinates. At this time, it is assumed that the shared image input bus 3 in FIG. 1 can transfer data of four pixels in one cycle.

  In S32, the acquisition start coordinates (horizontal / vertical) and size (horizontal / vertical) are designated in the parallel arithmetic circuit C. This designates horizontal and vertical as the start coordinates of the reference image fetched from the shared image input bus 3 and horizontal and vertical as the sizes thereof in the input buffer 41 of the parallel arithmetic circuit C of FIG.

  In S33, the reference image data is sent from the image memory 2 to the shared image input bus 3 here, four pixels per cycle.

  In S <b> 34, only data within the size range from the capture start coordinates is captured in the input buffer 42. This is because the parallel arithmetic circuit C takes in only the reference image data designated in S32 from the reference image data sent to the shared image input bus 3 in S33 and stores it in the input buffer 41.

In S35, the calculation shown in the figure is performed to generate a normalized correlation reference image.
As described above, the reference image data is sequentially transmitted from the image memory 2 to the shared image input bus 3 in one cycle, and the parallel arithmetic circuit C fetches only the reference image designated by itself and stores it in the input buffer 41. It is possible to execute the operation of S35.

FIG. 13 shows a flowchart for setting a search image for normalized correlation calculation according to the present invention.
In FIG. 13, S41 designates the transfer start coordinates (horizontal / vertical) and size (horizontal / vertical) to the image memory control circuit 1. For example, the image memory control circuit 1 in FIG. 4 sends horizontal and vertical coordinates as transfer start coordinates of a search image (see FIG. 2B) from a host (not shown) and horizontal and vertical sizes as the transfer coordinates. Specify coordinates. At this time, it is assumed that the shared image input bus 3 in FIG. 4 can transfer data of four pixels in one cycle.

  In S42, the start coordinates (horizontal / vertical) and the size (horizontal / vertical) are designated for each of the parallel arithmetic circuits A-C. This is because the parallel arithmetic circuit A.1 in FIG. In the B and C input buffers 41, horizontal and vertical are specified as the start coordinates of the search image to be fetched from the shared image input bus 3, and horizontal and vertical are specified as sizes thereof.

  In the normalized correlation calculation, the same value is specified as the start coordinate and size specified in the input buffers of the parallel arithmetic circuits A, B, and C.

  In S43, the search image data is sent from the image memory 2 to the shared image input bus 3 in this case, 4 pixels per cycle.

  In S44, only the data within the size range from the capture start coordinates is captured in the input buffer. This is because each of the parallel arithmetic circuits A, B, and C takes in only the search image data designated in S42 from the search image data sent to the shared image input bus 3 in S43 and stores it in the input buffer 41.

In S45, the parallel calculation circuit 4 performs the calculation shown in the figure.
In S46, calculation of the equation shown in the figure is performed to obtain coordinates that take the maximum absolute value.

  As described above, the search image data is sequentially transmitted from the image memory 2 to the shared image input bus 3 in one cycle, and each parallel arithmetic circuit A, B, C takes in only the search image designated by itself and inputs the input buffer. 41, the calculation of S45 is executed, and then the calculation of S46 is performed to obtain the coordinates that take the maximum absolute value of the result.

  In consideration of the fact that the image areas required by the individual systolic arrays overlap, the present invention requires individual systolic arrays by transmitting data for a plurality of pixels on the shared image input bus. By capturing data into the input buffer, the processing time can be shortened by reducing the data waiting time while using a single memory to reduce the required memory capacity, and each systolic array can receive each timing from the input buffer. A means (synchronization memory) is provided to synchronize the result of reading and processing the data at the same time, so that the calculation results that come out at different timings can be operated at the same time. The present invention relates to an image processing apparatus that performs calculation and peak extraction.

It is a system configuration diagram of the present invention. It is explanatory drawing (FIG. 1) of this invention. It is a system configuration figure (the 2) of the present invention. It is a system configuration figure (the 3) of the present invention. It is a reference image setting flowchart of the present invention. It is a setting flowchart of a search image of the present invention. It is transfer explanatory drawing (in the case of 31x31) of the search image of this invention. It is explanatory drawing of this invention. It is explanatory drawing of this invention. It is a setting flowchart of the filter coefficient / divisor of the present invention. It is explanatory drawing of this invention. It is a setting flowchart of the normalized correlation search image of this invention. It is a setting flowchart of the search image of the normalization correlation calculation of this invention. It is explanatory drawing (the 1) of a prior art. It is explanatory drawing (the 2) of a prior art.

Explanation of symbols

1: Image memory control circuit 2: Image memory 3: Shared image input bus 4: Parallel operation circuit 41: Input buffer 42: Systolic array 43: Divider 5: Selector 6: Synchronization memory 7: Pipeline operation circuit 8: Calculation result memory

Claims (4)

In an image processing apparatus that performs correlation calculation of images,
A shared image input bus for transferring data of a plurality of pixels from a memory in one cycle;
Means for selecting data for a specified vertical and horizontal size from a specified horizontal and vertical start point among a plurality of pixel data transferred on the shared image input bus;
An input buffer for storing the selected data;
A systolic array that performs operations by taking data from the input buffer;
A divider for dividing the operation result of the systolic array;
Selection means for fetching data designated in advance from the shared image input bus or data divided by the divider in the previous stage into the input buffer and inputting it to the systolic array;
The result calculated by the sheet Strick array, a synchronization memory for synchronization,
An image processing apparatus comprising: means for performing pipeline processing on computation or peak extraction between the plurality of synchronized computation results.   2. The partial area obtained by dividing the reference image area into each systolic array according to claim 1, the correlation calculation with the search area is calculated for each partial area, and the individual calculation results output at different timings are synchronized. An image processing apparatus characterized in that, after synchronization in a memory, calculation results are added together and peak detection is pipeline processed.   2. The system according to claim 1, wherein the systolic arrays are connected in cascade to perform spatial filter operations in each systolic array, and the operation output results of the individual spatial filters are synchronized in the synchronization memory, and then the operation results are subtracted. An image processing apparatus. In claim 1, when the reference image is loaded,

When loading the search image,

3 is calculated by each of the three systolic arrays, and a normalized correlation operation is processed by performing a pipeline operation on the following equation.

Images