JP5146270B2 - Normalized correlation processor - Google Patents

Description

  The present invention relates to a normalized correlation processing apparatus in image matching.

  In the field of image processing applications, there is a method of calculating a normalized correlation value and a sum of absolute differences SAD as one method of image pattern matching. Since this processing requires a large amount of calculation by the CPU, a dedicated circuit is often used to increase the speed, and various circuit configurations have been proposed.

  In normalized correlation, a normalized correlation value is calculated between an image to be searched (called a texture) and a partial rectangle (called a window) of the search target image. Then, this is generally performed by moving the window position of the search target image in small increments, for example, in units of pixels for a certain texture.

The following formula is a formula for calculating the normalized correlation value r.
Here, f is the pixel value (luminance value) of the source image (window), g is the pixel value (luminance value) of the template image (texture), and n is the number of effective pixels in the template area. When matching is performed, the texture g is shifted little by little to calculate r. Hereinafter, the position where the texture is applied is called a “window”. In the following, g is x and f is y. Since Σx, (Σx) ² and Σg ² do not change in each window, they are calculated in advance. Since Σxy, Σy ² and (Σy) ² change in each window, they are calculated every time. These are called “correlation element values”.

FIG. 13 is a diagram illustrating a method for calculating a normalized correlation value.
Only the correlation element value is calculated by the accelerator, and finally the correlation value is calculated by software. The coordinates at the upper left of the window are defined as the window position, and the correlation value when the window position is (0, 0) is calculated using the texture image x and the original image y in the window. Next, the window position is moved horizontally by one pixel, and the correlation value is obtained when the window position is (1, 0). As described above, correlation values are obtained for all original images y, such as when the window position is (3, 3).

FIG. 14 is a diagram for explaining how to set the window position.
At this time, there are two methods for determining the position of the window: a method of setting a regular position and a method of setting a random position.

  When the positions of the windows are regular, the windows for which correlation values are to be obtained are regularly shifted by one or several pixels. When the window positions are random, the window positions for which correlation values are desired are scattered without regularity.

FIG. 15 is a diagram for explaining the outline of processing by an accelerator when the window positions are regular.
Only the correlation element values (Σxy, Σy, Σy ² ) are calculated by the accelerator, and finally the correlation value r is calculated by software. When calculating Σxy, the multiplier 10 multiplies the texture and window pixels at each of the window positions (0, 0), (1, 0)... (3, 3),. xy is calculated, and the adder 11 adds the calculation results to calculate Σxy. The calculation result is stored in the register 12. The machine cycle required for the calculation of Σxy is n × m cycles corresponding to the window size of n × m pixels. In FIG. 15, since the window size is 6 × 4, it takes 24 cycles to calculate one Σxy.

  In this case, it is conceivable to perform parallel processing in order to speed up this processing. The parallel processing includes a method of performing parallel processing in units of windows and a method of performing parallel processing in units of pixels.

  In the parallel processing method in units of windows, each computing unit is in charge of a different window. The latency per window is proportional to n × m which is the size of the window. Throughput is multiplied by the number of calculators, but the window position is given individually. If the window position is random, if the window is far away, the same pixel cannot be accessed and calculated. Unlike the case where the window position is given to, it is not suitable for parallel processing.

  There is also a method of processing a plurality of pixels in parallel in one window. In this case, a single arithmetic unit set capable of processing a plurality of pixels in one cycle is all inserted into one window process, and can be processed in parallel in one window, which affects the window position. This is suitable when the window position is random.

FIG. 16 is a diagram illustrating a hardware configuration example for image template matching.
The case where the window positions to be processed are densely fixed is shown, and 16 4 × 4 window positions from (0, 0) to (3, 3) are processed in parallel. The data of Area 1-16 from one input image is input to GSEU (Gray-scale Search Engine Unit). One GSEU is provided for each Area 1-16 to be processed in parallel, and each Area 1-16 is processed in parallel. Each GSEU is provided with an arithmetic unit for calculating each normalized correlation element value, and performs an operation using the input data of Area 1-16 and the reference data. The processing performed by each arithmetic unit is one operation, two operations, or three operations depending on the processing to be performed. The GSEU operates with, for example, a 300 MHz clock, and the calculation result is output via the bus.

  When the windows are arranged in an overlapping manner, it is an efficient method only when the window positions are regular by setting only one pixel of the reference image to be referred to in a certain cycle. If it is random, only one window can be processed at a time.

FIG. 17 is a diagram for explaining a method of performing parallel processing in units of windows.
Each computing unit is in charge of a separate window. In FIG. 17, 16 arithmetic units each including a multiplier 10, an adder 11, and a register 12 are provided, and each of 16 windows from windows (0, 0) to (3, 3) is provided. Handle. The coordinates representing the position of the window are the coordinates of the upper left pixel of the window. The latency per window is proportional to the number of pixels n × m which is the size of one window. The throughput is improved by a factor of the number of computing units compared to the case of one computing unit.

  On the other hand, one arithmetic unit set that can process 4 or 8 or 9 or 16 pixels per cycle is put into one window processing, or 4 or 8 or 9 or 16 pixels are 2x2 or 4x2 or 3x3 or 4x4 For example, a method may be considered in which tiled areas are divided and processing is performed for each area.

FIG. 18 is a diagram for explaining a method of processing a plurality of pixels in parallel in one window.
FIG. 18 shows a case where pixels are read and processed by dividing a 6 × 8 texture into 4 × 2 tile areas. By using a 4 × 2 tile area as a unit, one window is read in 6 cycles. The input image is also read in units of 4 × 2 tile areas. The arithmetic unit is an example in the case of calculating Σxy, and includes eight multipliers 10-1 to 10-8, and further includes an adder 11 and a register 12.

FIG. 19 is a diagram for explaining the configuration of a conventional normalized correlation processing apparatus.
The window position control parameters are specified on the assumption of rule access, and the window position control parameters include the position of the top window, the thinning width, the number of vertical and horizontal windows, and the like. As shown in FIG. 19A, the control register 25 stores parameters to be given to the control circuit 26 that performs window reading control. For example, the buffer filling sizes bh, bw, texture sizes th, tw, The first window position x, y, the vertical and horizontal thinning widths cx, cy, and the vertical and horizontal window numbers nx, ny. According to the configuration diagram of FIG. 19B, the memory 20 is connected to the texture buffer 22 and the window buffer 23 via the bus 21. The texture buffer 22 stores a template to be matched. The window buffer 23 reads image data of an area to be matched from the original image stored in the memory 20. The normalized correlation calculation circuit 24 reads pixel data from the texture buffer 22 and the window buffer 23 and calculates a normalized correlation value.

FIG. 20 is a diagram for explaining the configuration of a conventional double buffer type normalized correlation processing apparatus.
The window position control parameter is specified by the position of the top window and the thinning width based on rule access. As shown in FIG. 20A, parameters are input from the control register 25 to the control circuit 26 that controls reading of the window. Parameters stored in the control register 25 are, for example, buffer filling sizes bh and bw, texture sizes th and tw, leading window positions x and y, vertical and horizontal thinning widths cx and cy, and vertical and horizontal window numbers nx and ny. As shown in FIG. 20B, the memory 20 is connected to the bus 21 and transmits / receives data to / from the texture buffer 22, window buffer # 0 23-1, and window buffer # 1 23-2. Different windows are stored in the window buffer # 0 23-1 and the window buffer # 1 23-2, and are sequentially switched by the selector 27 to supply window data to the normalized correlation calculation circuit 24.

In this way, by providing a double buffer, it is possible to parallelize data supply processing and arithmetic processing.
In the conventional configuration, the speeding-up method has been studied on the assumption that the window position is regular such that the calculation is performed at every position by shifting one pixel at a time within a certain rectangle. However, in recent image processing applications, not only in such a regular case, but also in a way that the window position is randomly present at an arbitrary position of the search target image, in such a case, the conventional method is used. There was a problem that the performance could not be fully exhibited if applied as it is.

Parallel processing is required for speeding up, and as the method, parallel processing in window units and parallel processing in pixel units can be considered. In addition, for speeding up, it is necessary to push out multiple windows at once. When there is a premise that the window position is regular, this method has been conventionally used because parallel processing in units of windows is more efficient in order to reduce the memory access amount. However, if the window unit parallel processing is used, if the position is random, it becomes difficult to realize the memory access from a random address for each window to be processed in parallel. Further, since the collective push-out method is based on a regular window position, the push-out in the case of random positions has not been considered.

  For example, in the case of the conventional method (Patent Documents 1 to 3) that performs window unit parallel processing, since the window position of the parallel processing is regular, if it is random, batch processing cannot be pushed out, one window at a time It can only be processed.

Further, in the case of the conventional method for performing parallel processing by image division (Patent Document 4), the load time of the memory is reduced by using data locality and parallel processing by image division is performed. In the case of random access, The locality of data cannot be used, and even if many arithmetic units are installed, the memory access speed becomes a bottleneck and the arithmetic units are idle.
JP 09-009269 A Japanese Patent Laid-Open No. 10-134183 JP 2006-013873 A JP 2008-84034 A

  As described above, a method based on the assumption that the window position is regular has been considered so far, but there is a device configuration capable of high-speed processing flexibly corresponding to both a regular case and a random case. is necessary.

  The conventional normalized correlation circuit method is configured to increase the search speed on the assumption that search window positions are regularly concentrated. However, in recent image processing applications, not only such a regular case but also the window position is often random, and in such a case, if the conventional method is applied as it is, the performance cannot be fully exhibited. There was a problem.

  An object of the present invention is to provide a normalized correlation processing apparatus that can cope with a case where a window position is random.

  A normalized correlation processing apparatus according to the present invention is a normalized correlation processing apparatus for performing pattern matching processing of an image, and includes template buffer means for holding a template for pattern matching, and arbitrary vertical and horizontal sizes of a search rectangle. A plurality of search rectangle buffer means for specifying and holding the search rectangle of the input image, position information of the search rectangle, and a control register means for holding data for controlling writing to the search rectangle buffer means Computing means for computing a normalized correlation value between the template stored in the template buffer means and data in the search rectangle stored in one of the plurality of search rectangle buffer means.

  ADVANTAGE OF THE INVENTION According to this invention, the normalization correlation processing apparatus which can respond also when a window position is random can be provided.

In this embodiment, it is possible to individually specify the window positions to be processed in a lump in the window position array for each window, and when filling the window into the window buffer, the vertical and horizontal sizes bh and bw are arbitrarily set regardless of the texture size th and tw. Can be specified. The window buffer has a double buffer configuration and performs a window filling operation in parallel with the calculation. Switching of the double buffer is automatically performed by a circuit that performs control so that switching is not performed when a window to be processed next is included in the sequence number in the currently used buffer. On the other hand, there is a control circuit in which the hardware automatically calculates the area to be filled in the buffer, facilitating use by the user. These allow processing with the same procedure regardless of whether the window position is regular or not.

1 and 2 are diagrams for explaining the outline of the embodiment of the present invention.
As shown in FIG. 1A, the buffer filling sizes bh and bw, the texture sizes th and tw, and the window position array are set in the control register 30. The buffer filling size can arbitrarily specify the size of the rectangular area stored in the window buffer regardless of the texture size. The texture size specifies the size of the texture. The window position array is an array for individually specifying a random position of the window when the position of the window is random. When the position of the window is regular, the window position can be specified by the vertical and horizontal sizes and the thinning distance from the head window position. The control circuit 31 stores the buffer next filling start position search circuit for determining the start position of the next filling window in the window buffer having the double buffer structure and the buffer having the double buffer structure in which the window to be processed is stored. And a buffer switching determination circuit which is a circuit for switching a buffer to be read based on the determination result. This is a circuit that automatically performs switching determination and buffer filling rectangular area determination from parameters.

  FIG. 1B shows the configuration of the normalized correlation processing device. Texture data is sent from the memory 20 to the texture buffer 22 via the bus 21, and window data is sent to the window buffer # 0 23-1 and window buffer # 1 23-2. In this embodiment, the size of the storage area of the window buffer can be arbitrarily specified. Window data is read from the window buffer selected by the selector 27 and input to the normalized correlation calculation circuit 24. The normalized correlation calculation circuit 24 receives the texture data from the texture buffer 22 and is used for the normalized correlation calculation together with the window data. The normalized correlation calculation circuit 24 is a circuit that performs parallel processing in units of pixels within one window.

  In FIG. 2, rectangular area vertical and horizontal size designation parameters bh and bw to be filled in the buffer are newly provided so that they can be freely designated (th ≠ bh, tw ≠ bw). By making it possible to freely set the rectangular size to be filled in the window buffer, when windows overlap at a randomly given window position, all the overlapping windows are filled in the window buffer. As a result, windows that overlap each other can be read without performing buffer switching when reading.

The parameters representing the performance of the normalized correlation processing apparatus include the following.
-Basic data supply performance-Number of pixels that can be brought from external memory per cycle-Real data supply performance-Basic data supply performance Obtained per cycle, taking into account the cache effect of pixel reuse in the buffer Number of pixels / calculation performance-The number of pixels that the computing unit can process per cycle

FIG. 3 is a diagram for explaining the operation when the window position is regular.
In FIG.
-Basic data supply performance <calculation performance.
・ Position sequence is A, B, C, D, ..., L.
・ Assume that bh and bw are set so that AD, EH, and IL are filled together.
・ Assuming that the actual data supply performance> calculation performance due to the cache effect.
The actual data supply performance is given by the following equation.
(Actual data supply performance = basic data supply performance + cache effect)

The operation is as follows.
1. The window buffer 0 is filled with rectangles bh and bw from the upper left coordinate of A. The area of AD is filled.
2. Calculate the correlation value of A. At the same time, the next filling start window position E to be filled in the window buffer 1 is searched by the next filling start window position search circuit, and the buffer 1 is filled. Further, the next filling start window position I is searched by the next filling start window position search circuit. Since buffer 0 is in use, filling is suspended until buffer 0 is released.
3. When the calculation of A is completed, the buffer switching determination circuit determines whether B is included in the buffer 0 or not.
4). Since B is included, B is calculated without switching the buffer.
5. Similarly, calculate up to D.
6). When the calculation of D is completed, the buffer switching determination circuit determines whether E is included in the window buffer 0 in the same manner.
7). Since E is not included, the buffer is switched to window buffer 1, E is read, and E is calculated. In parallel, since the window buffer 0 is vacant, the rectangle from I that has been previously reserved is filled. Subsequently, the next filling start window position is determined. However, since all the windows up to the end L are included in the window buffer 0, the filling ends.
8). As before, each time F, G, H and one window ends, calculation is performed while performing buffer switching determination. Until H, the buffer is not switched.
9. When the buffer switching determination is performed at the end of H, since I is not included in the window buffer 1, switching to the window buffer 0 is performed and the processing is similarly continued until the end.

  The example of FIG. 3 shows a case where the texture is relatively small and the window positions are regular. The filling rectangle size is set to bh> th, bw> tw, and a plurality of windows are filled into one buffer at a time.

FIG. 4 is a diagram for explaining the operation when the window position is random.
In FIG. 4, it is assumed that the following conditions are satisfied.
• The basic data supply performance is less than the calculation performance circuit configuration.
・ The position array is in the order of A, B, C.
Suppose bh and bw are set so that A, B, and C are filled one by one.
Suppose that there is no cache effect and the actual data supply performance is less than the computation performance.

The operation is as follows.
1. The window buffer 0 is filled with rectangles bh and bw from the upper left coordinate of A. Only area A is filled.
2. Calculate the correlation value of A. At the same time, the next filling start window position B for filling the window buffer 1 is determined by the next filling start window position search circuit, and the window buffer 1 is filled. Further, the next filling start window position C is calculated by the next filling start window position determination circuit. Since window buffer 0 is in use, filling is deferred until window buffer 0 is released.
3. When the calculation of A is completed, the buffer switching determination circuit determines whether B is included in the window buffer 0 or not.
4). Since B is not included, switch the buffer and wait for B to fill before calculating B. In parallel, since the window buffer 0 is vacant, the rectangle from C that was previously reserved is filled. Subsequently, the next filling start window position is determined. Since all of the windows up to the end C are included in the window buffer 0, the filling ends.
5. Similarly, when the calculation of B is completed, a buffer switching determination is performed. Since C is not included, the buffer is switched, and after waiting for C to be filled, C is calculated and the processing ends.

  FIG. 4 shows an example in which the texture is relatively large and the window position is random. The filling rectangle size is set to be the same as the texture size (bh = th, bw = tw), and is normally operated as a double buffer.

FIG. 5 is a diagram for explaining the operation when the window position where the cache effect occurs is random.
In FIG. 5, it is assumed that the following conditions are satisfied.
• The basic data supply performance is less than the calculation performance circuit configuration.
・ The position array is in the order of A, B, C.
Suppose bh and bw are set so that A and B are filled in window buffer 0 and C is filled in window buffer 1, respectively.
A case where there is a cache effect and the actual data supply performance is almost equal to the computation performance.

The operation is as follows.
1. The window buffer 0 is filled with rectangles bh and bw from the upper left coordinate of A. A and B areas are filled.
2. Calculate the correlation value of A. At the same time, the next filling start window position C to be filled in the window buffer 1 is searched by the next filling start window position search circuit, and the window buffer 1 is filled. Further, the next filling start window position is searched, but since the end has been reached, the filling ends.
3. When the calculation of A is completed, the buffer switching determination circuit determines whether B is included in the window buffer 0 or not.
4). Since B is included, B is calculated without switching the buffer.
5. Similarly, when the calculation of B is completed, a buffer switching determination is performed. Since C is not included, the buffer is switched, C is calculated, and the process ends.

  FIG. 5 shows a case where the texture is relatively large and the window position is random. If the overlapping area is large, setting the filling rectangle size to (bh> th, bw> tw) produces a cache effect and improves processing performance.

FIG. 6 is a diagram illustrating the operation timing when the window positions are regular.
FIG. 6 shows a case where ABCD is filled in the window buffer 0 in a batch and EFGH is filled in the window buffer 1 in a batch. Also, basic data supply performance <assuming computation performance, actual data supply performance> basic data supply performance, actual data supply performance = basic data supply performance + cache effect is assumed.

  The window positions up to windows A to L are stored in the control register. First, window buffer 0 is filled with window AD. At this time, since the time required to store the windows A to D collectively in one buffer is smaller than the total time required to individually store the four windows in the buffer, the efficiency increases. When the window buffer 0 is completely filled, calculation processing is performed in order from the window A. During this calculation process, window buffer 1 is filled with window E-H. Since the calculation process and the filling process are executed in parallel, the filling time for the window 1 is hidden. When the processing up to window D is completed for window buffer 0, window E is read from window buffer 1, and the window calculation processing is sequentially performed thereafter. While the window E-H of the window buffer 1 is being processed, the window buffer 1 is filled with the window I and subsequent windows, and the filling time is hidden.

FIG. 7 is a diagram showing the operation timing in the case of using as a normal double buffer when the window position is random and without the cache effect.
If the basic data supply performance is less than the calculation performance (for example, the data supply performance is 8 pixels / cycle and the calculation performance is 16 pixels / cycle), the filling time becomes a critical path. That is, in the case of FIG. 7, the window position of the window AC is stored in the control register, but each window is stored in the window buffer one by one. First, window A is filled in window buffer 0. When filling is completed, window A is calculated. When filling of window A in window buffer 0 is completed, filling of window buffer 1 is started in parallel with the calculation processing of window A. Window buffer 1 is filled with window B. When filling of window B is completed, filling of window C into window buffer 0 is started at the same time as calculation processing of window B is started. Here, since it takes more time to fill the window buffer than the window calculation processing, the processing time is determined by the window buffer filling time, that is, a critical path.

FIG. 8 is a diagram for explaining the operation timing when the cache effect is present even when the window position is random.
FIG. 8 shows a case where the basic data supply performance is less than the calculation performance, and the filling time can be concealed by the cache effect of batch filling because the overlap between A and B is large. The control register stores the window position of the window AC. When the window buffer 0 is filled, the windows A and B are filled. When the filling of the window buffer 0 is finished, the calculation processing of the window A and the filling of the window buffer 1 are started. Although the calculation processing time of one window is shorter than the filling time of the window buffer, since two windows are stored in the window buffer 0, the window buffer is not processed until the calculation processing of the window in the window buffer 0 is completed. 1 filling is completed and the filling time can be concealed. Further, when the windows A and B are filled in the window buffer 0 all at once, the filling time can be shorter than the total time for filling the windows A and B separately, so that the actual data supply performance is improved.

  FIG. 9 is a diagram for explaining a window buffer filling method and a cache effect. The window buffer has a double buffer configuration (buffers 0 and 1), and the texture size is, for example, a maximum of 16 KB (equivalent to 128 × 128). The buffer fill sizes bh and bw can be freely set, and are not fixed to the texture sizes th and tw.

The window buffer switching determination is as follows.
If the next window is in the current buffer, do not switch buffers. This is automatically determined within the hardware from the next window position. At the start, a) Fill the buffer 0 with the bh and bw rectangles of the first entry in the window position array. b) Look through the entries in order from the top entry, and when the entry x not included in the rectangle filled in the buffer 0 is reached, the bh and bw rectangles of the entry x are filled in the buffer 1. When the processing of the entry used for filling the window is reached, as in b), the rectangle of the entry that is not included in the current rectangle after that entry is filled in the other buffer.

FIG. 9A shows a case where the window position is regular and the texture is relatively small compared to the buffer capacity. The window AD is stored in the window buffer 0, and the window EH is stored in the window buffer 1. When all the windows in the window buffer 0 have been processed, the window buffer 0 is filled with the window IL. FIG. 9B shows a case where the window position is random and there is no cache effect. Window A is stored in window buffer 0 and window B is stored in window buffer 1. Since there is little overlap between windows, when a plurality of windows cannot be stored in one window buffer, one window is stored in one window buffer. On the other hand, the case of FIG. 9C is a case where the window position is random and there is a cache effect. This shows a case where windows A and B overlap each other, and windows A and B fit in one window buffer. When a plurality of windows can be stored in one window buffer, one window buffer is filled with a plurality of windows. Since multiple windows are stored in one window buffer at a time, data transfer efficiency is good, and even when switching windows for calculation processing, there is no need to switch buffers, so window data can be read quickly. I can do it.

FIG. 10 is a diagram showing a processing flow for filling the window buffer.
In FIG. 10, win [i] is the i-th window (i⊂ {0,1, ..., N-1}), buf [b] is b buffer, (b⊂ {0,1}, Double buffer).

  It is assumed that there are N windows from 0 to N-1. In step S10, i = 0 and b = 0 are initialized. In step S11, the window buffer buf [b] is filled with a rectangle starting from win [i]. In step S12, i is increased by 1. In step S13, it is determined whether i is N or more. If the determination in step S13 is Yes, the process ends. If the determination in step S13 is No, in step S14, whether or not the i-th window can be filled in the b window buffer buf [b]. Judging. If the determination in step S14 is yes, the buffer is filled, and the process returns to step S12. If the determination is no, the process proceeds to step S15. In step S15, b is replaced with a bit inverted version of b (0 is set to 1 and 1 is set to 0), and the process proceeds to step S16. In step S16, it is determined whether or not the buffer buf [b] is empty. If the determination in step S16 is No, step S16 is repeated until the buffer is empty. If the determination in step S16 is yes, the process returns to step S11.

FIG. 11 is a diagram showing a window buffer switching process flow.
In the flow for determining the buffer to be used for the next window processing in FIG. 11, switching is performed only when the buffer currently in use is not included.

  In FIG. 11, ic is the window number currently being processed, bc is the buffer number currently used, and bn is the buffer number to be used next. In step S20, i = ic + 1 and b = bc are initialized. In step S21, it is determined whether or not the window win [i] is filled in the window buffer buf [b]. If the determination in step S21 is Yes, the process proceeds to step S23, and if the determination is No, the process proceeds to step S22. In step S22, the bit of b is inverted. In step S23, b is set to bn, and the process ends.

FIG. 12 is a diagram for explaining an automatic determination method for filling and switching the window buffer.
The determination method of the condition win [i] ⊆buf [b]? (FIGS. 10 and 11) where win is in buf is as follows.

The upper left coordinates (x, y) of win [i] and the upper left coordinates (px, py) of buf [b] are used. The following judgment formula is determined which means that the upper left coordinate of win is lower right than the upper left coordinate of buf, and the lower right coordinate of win is upper left than the lower right coordinate of buf.
((px <= x) && (py <= y)) && (((px + bw)> = (x + tw)) && ((py + bh)> = (y + th)))
FIG. 12 shows an example in the case of (px, py) = (x0, y0), (x, y) = (x1, y1). Initial setting values include bh (buffer height), bw (buffer width), th (window height), and tw (window width). As win [i], window position arrays x and y are given. px <= x && py <= y is a judgment formula that means that the upper left coordinate of win is at the lower right of the upper left coordinate of buf. Also, ((px + bw)> = (x + tw)) && ((py + bh)> = (y + th)) means that the lower right coordinate of win is at the upper left of the lower right coordinate of buf This is a judgment formula.

The address generation method for the buffer filling operation fill win [i], buf [b] is as follows. The input image is assumed to be a continuous address on the main memory. iB (start address of input image), ih (change in address for one column of input image height), iw (change in address for one input image width), bh (buffer height) , bw (buffer width), and x [y] of win [i] coordinates, an address is generated according to the following formula.
iB + (y + H) * iw + x + W,
where 0 <= H <= bh, 0 <= W <= bw
H and W are the upper left positions of the window.
By combining the access patterns of a) access at a random window position and b) access at a regular window position, the following effects can be obtained.
a) In the case of large texture and random, if th, tw = bh, bw, it can be used as a normal double buffer.
b) For small textures and rules, by setting th, tw <bh, bw, the surrounding window is also brought into the buffer when the first window is filled. Latency can be concealed by not switching the buffer while the window is in the buffer.

  According to the above embodiment, both the case where the window position is regular and the case where it is random can be dealt with flexibly. Furthermore, regardless of whether the window position is regular or random, the user does not need to switch the buffer or specify the area to be filled. The window position order win [N] and the buffer are filled. By simply specifying the vertical and horizontal sizes bh and bw, they can be handled consistently in the same usage method.

In addition to the above embodiment, the following supplementary notes are disclosed.
(Appendix 1)
A normalized correlation processing device for pattern matching processing of an image,
A template buffer means for holding a template for pattern matching;
A plurality of search rectangle buffer means for arbitrarily specifying the vertical and horizontal sizes of the rectangle data including the search rectangle and holding the rectangle data including the search rectangle of the input image;
Control register means for holding position information of the search rectangle and data for controlling writing to the plurality of search rectangle buffer means;
Computing means for computing a normalized correlation value between the template stored in the template buffer means and data in the search rectangle stored in one of the plurality of search rectangle buffer means;
A normalized correlation processing device comprising:
(Appendix 2)
The control register means individually holds the position information of each search rectangle taking a normalized correlation one by one without depending on each other, and when the position of the search rectangle is regular, from the start search rectangle position The normalized correlation processing apparatus according to appendix 1, wherein the position of the search rectangle can be specified also by a vertical and horizontal size and a thinning distance.
(Appendix 3)
The correlation rectangle is calculated using the search rectangle filled in the first search rectangle buffer means included in the plurality of search rectangle buffer means, and the search rectangle used for the next correlation value calculation is calculated as the plurality of search rectangles. 2. The normalized correlation processing apparatus according to appendix 1, wherein the second search rectangular buffer means included in the rectangular buffer means is filled.
(Appendix 4)
When the calculation of the first search rectangle is finished and the calculation of the second search rectangle is started, if the search rectangle buffer currently used for the calculation includes the second search rectangle, Without switching the search rectangle buffer, it is used for the calculation of the second search rectangle as it is, and if the second search rectangle is not included, the search rectangle buffer is switched and used for the calculation of the second search rectangle. The processing apparatus according to appendix 1, wherein:
(Appendix 5)
When determining whether or not the first search rectangle is in the currently used search rectangle buffer means, the upper left coordinate of the search rectangle is lower right than the upper left coordinate of the rectangular area filled in the search rectangle buffer means. The normalized correlation processing device according to appendix 4, wherein the normalization correlation processing apparatus determines whether the lower right coordinate of the search rectangle is located on the upper left side of the lower right coordinate of the rectangular area filled in the search rectangle buffer means .
(Appendix 6)
To again filling the rectangle data of the current addition to the search rectangle buffer means is not used, the search rectangle buffer means currently in use, the search rectangle that is currently processed in order of the search rectangle position array It is judged in order whether the next search rectangle is included, and when a search rectangle not included is reached, the next rectangle data is filled in one of the search rectangle buffer means starting from that search rectangle. The normalized correlation processing device according to attachment 4.
(Appendix 7)
The normalized correlation processing apparatus according to appendix 1, wherein the computing means is capable of processing a plurality of pixels in parallel .
(Appendix 8)
The calculation means is the sum of the pixel values in the search rectangle, the sum of the squares of the pixel values in the search rectangle, the pixel values in the search rectangle, and the pixels in the template, which are element values of the normalized correlation value The normalized correlation processing device according to appendix 1, wherein a sum of products of pixels having the same position is calculated.
(Appendix 9)
The normalized correlation processing apparatus according to claim 1, wherein a size of rectangular data including the search rectangle is independent of a size of the template.

It is FIG. (1) explaining the outline | summary of embodiment of this invention. It is FIG. (2) explaining the outline | summary of embodiment of this invention. It is a figure explaining operation | movement in case a window position is position regular. It is a figure explaining operation | movement in case a window position is random. It is a figure explaining operation | movement in case the window position where a cache effect produces is random. It is a figure which shows the operation | movement timing in case a window position is regular. It is a case where it is a case where it uses as a normal double buffer at the time of window position random, Comprising: It is a figure which shows the operation timing in case there is no cache effect. It is a figure explaining the operation timing when there is a cache effect even when the window position is random. It is a figure explaining the filling method and cache effect of a window buffer. It is a figure which shows the processing flow for filling of a window buffer. It is a figure which shows the switching process flow of a window buffer. It is a figure explaining the automatic determination method of filling and switching of a window buffer. It is a figure explaining the calculation method of a normalized correlation value. It is a figure explaining how to take a window position. It is a figure explaining the outline | summary of a process by the accelerator when a window position is regular. It is a figure which shows the hardware structural example for the template matching of an image. It is a figure explaining the system which processes in parallel per window. It is a figure explaining the system which processes in parallel several pixels by 1 window. It is a figure explaining the system which processes in parallel several pixels by 1 window. It is a figure explaining the structure of the normalization correlation processing apparatus of the conventional double buffer system.

Explanation of symbols

10, 10-1 to 10-8 Multiplier 11 Adder 12 Register 20 Memory 21 Bus 22 Texture buffer 23, 23-1, 23-2 Window buffer 24 Normalized correlation operation circuit 25, 30 Control register 26, 31 Control circuit 27 Selector

Claims (5)

A normalized correlation processing device for pattern matching processing of an image,
A template buffer means for holding a template for pattern matching;
A plurality of search rectangle buffer means for arbitrarily specifying the vertical and horizontal sizes of rectangular data including a search rectangle and holding the rectangle data including one or more search rectangles of an input image;
Control register means for holding position information of the search rectangle and data for controlling writing to the plurality of search rectangle buffer means;
Computing means for computing a normalized correlation value between the template stored in the template buffer means and data in the search rectangle stored in one of the plurality of search rectangle buffer means;
Equipped with a,
When the calculation of the first search rectangle is completed and the calculation of the second search rectangle is started, if the search rectangle buffer currently used for the calculation does not include the second search rectangle, by switching the search rectangle buffer, normalized correlation processing apparatus characterized that you use in the calculation of the second search rectangle. When filling the search rectangle buffer means that is not currently used with the rectangular data, the search rectangle buffer means that is currently being used has the search rectangles currently processed in the order of the position arrangement of the search rectangles. It is judged in order whether the next search rectangle is included, and when a search rectangle not included is reached, the next rectangle data is filled in one of the search rectangle buffer means starting from that search rectangle. The normalized correlation processing apparatus according to claim 1. The control register means individually holds the position information of each search rectangle taking a normalized correlation one by one without depending on each other, and when the position of the search rectangle is regular, from the start search rectangle position The normalized correlation processing apparatus according to claim 1, wherein the position of the search rectangle can be specified also by a vertical and horizontal size and a thinning distance. The correlation rectangle is calculated using the search rectangle filled in the first search rectangle buffer means included in the plurality of search rectangle buffer means, and the search rectangle used for the next correlation value calculation is calculated as the plurality of search rectangles. 4. The normalized correlation processing apparatus according to claim 1, wherein the second search rectangular buffer means included in the rectangular buffer means is filled. When determining whether or not the first search rectangle is in the currently used search rectangle buffer means, the upper left coordinate of the search rectangle is lower right than the upper left coordinate of the rectangular area filled in the search rectangle buffer means. There, and any one of claims 1 to 4, wherein the determining by the lower right coordinates of the search rectangle in the upper left from the lower right coordinates of the filled rectangular area in the search rectangle buffer means Normalized correlation processing apparatus described in 1.