JP5499203B2 - Block matching circuit and data update method - Google Patents

Description

  The present invention relates to image processing, and more particularly to block matching for determining similarity between images.

  For example, by determining the similarity between a certain frame of the moving image and another frame, a motion vector indicating the motion of the object appearing in each frame is calculated. Specifically, an image region including the object is cut out from a certain frame, and it is determined which region of the other frame the image region is similar to. In order to determine the similarity between such images, block matching is employed (see, for example, the following patent document).

  In block matching, a target region cut out from a frame and a region obtained by scanning the region (a whole region may be referred to as a “scan region” hereinafter) as a specific region of another frame (hereinafter referred to as “comparison”). Similarity to the “region” is determined for the target region and the comparison region as a whole. Specifically, for example, one pixel in the target area is compared with one pixel or a plurality of pixels corresponding to the one pixel in the comparison area. Then, the comparison results between these pixels are accumulated over the entire comparison area, and the similarity between the target area and the comparison area is determined. As a comparison result between pixels, a value indicating a difference between the two pixels (hereinafter referred to as “pixel difference value”), for example, an absolute value of a difference (for example, luminance) indicated by each pixel (hereinafter referred to as “pixel difference absolute value”). ) Is adopted. The similarity is calculated as the sum of the pixel difference values.

  Since the similarity is obtained for each comparison area, a comparison area having the highest similarity with the target area in the scanning area is obtained. Therefore, a motion vector is calculated based on the position occupied by the target area in the frame and the position occupied by the comparison area in the frame.

JP 2006-33145 A

  In order to specify a comparison area similar to the target area in the scanning area, a process for obtaining a pixel difference value is obtained by multiplying the number of operations in the target area and the comparison area by the total number of comparison areas obtained by scanning. Must be executed in number of times.

  Here, for simplicity, it is assumed that the target area and the comparison area have the same size. It is assumed that a pixel difference value is obtained between one pixel occupying a certain relative position in the target region and one pixel occupying the relative position in the comparison region.

  When the target area is represented as a pixel group arranged in M rows and N columns and the scanning area is represented as a pixel group arranged in J rows and K columns (J, K, M, and N are J> M, K > A positive integer satisfying> N), the number of comparison areas obtained by scanning in one scanning area is (J−M + 1) · (K−N + 1).

  If the pixel data is accumulated in advance for the number of comparison regions, and the pixel difference value is calculated for the corresponding pixels simultaneously with the pixel data of the target region, the processing can be performed quickly. . However, for that purpose, M · N pixel data, which is the total number of pixels in the target area, must be stored, and (J−M + 1) · (K−N + 1) · M · N pixel data in the comparison area must be stored. . Such a technique has a drawback of requiring a large storage area.

  As a technique for solving such a drawback, a technique for obtaining a pixel difference value for a plurality of comparison areas at once using one pixel of a target area and a pixel corresponding to the one pixel in each of a plurality of comparison areas. Can be considered.

  In this case, since the number of pixel data to be stored for the target region is one, the number of pixel data to be stored can be reduced to 1 / (M · N) as compared with the technique described above. However, in block matching, as described above, image similarity is calculated as the sum of absolute values. Therefore, in order to obtain the similarity with a certain target image, naturally, the processing time of M · N times is required for the technique described above. In addition, pixel data must be captured M · N times for the target area. Furthermore, since overhead usually occurs in capturing pixel data, it can be seen that this technique increases processing time.

  An object of the present invention is to solve the above-described problems, and provides a technique for reducing the processing time by reducing the number of data fetches while reducing the storage area by reducing the number of data to be stored.

  According to a first aspect of the block matching circuit of the present invention, scan image data that is pixel data corresponding to each pixel of J rows and K columns included in a scan region that is an image region to be compared for block matching is stored. Pixels in M rows and N columns (J, K, M, N are positive integers satisfying J> M, N <K) included in the first storage area and the target area which is an image area serving as a comparison reference for the block matching Pixel data corresponding to pixels in M rows and N columns included in each of a second storage area that stores target image data that is pixel data corresponding to each of the plurality of comparison areas obtained by scanning the scan area Between the comparison image data and the target image data, M · N arithmetic circuits for obtaining a pixel difference value indicating a difference between mutually corresponding pixels, and the first storage area M / N selection circuits for selecting comparison image data for each comparison region and outputting to the arithmetic circuit; a summation circuit for obtaining a cumulative value of the pixel difference values for each comparison region; A matching determination unit that compares the accumulated values for each comparison area and identifies the comparison area that minimizes the accumulated value;

  The matching determination unit sequentially inputs the cumulative value, stores a stored value that is a value to be stored, and the cumulative value that is input when the input cumulative value is less than the stored value A minimum value storage circuit for updating the stored value using the data, a minimum value position storage circuit for storing first data taking binary values in each of (J−M + 1) · (K−N + 1) digits, and the first And an update circuit for updating one data.

  The update circuit takes binary values in each of (J−M + 1) and (K−N + 1) digits, and synchronizes the second data in which one digit takes the first value sequentially with the cumulative value. And the first data is updated using the second data when the input cumulative value is less than the stored value.

  A second aspect of the block matching circuit according to the present invention is the first aspect, wherein the update circuit has the second data when the input cumulative value is equal to the stored value; The first data is updated by using a logical OR for each digit with the first data stored immediately before.

  A third aspect of the block matching circuit according to the present invention is any one of the first and second aspects, wherein the pixel data in the target image data is a group of pixels belonging to the same row in the target area. It is handled as cluster data that is corresponding pixel data.

  A fourth aspect of the block matching circuit according to the present invention is the third aspect, and further includes a plurality of selectors for swapping the cluster data for each column in the target area.

  A fifth aspect of the block matching circuit according to the present invention is any one of the first to fourth aspects, wherein the pixel data in the scanned image data is a group of pixels belonging to the same row in the scanned region. It is handled as cluster data that is corresponding pixel data.

  A sixth aspect of the block matching circuit according to the present invention is the fifth aspect, and further includes a plurality of selectors for swapping the cluster data for each column in the scanning region.

  A seventh aspect of the block matching circuit according to the present invention is the first to sixth aspects, further comprising a register for storing a flag. Within the scanning region, scanning is performed in a row direction from the first row to the Jth row, and scanning in a column direction from the first column to the Kth column is performed with priority to the row direction. The comparison region is obtained. The flag is set when scanning in the column direction is performed, and the flag is reset when the scanning reaches the Kth column. When the flag is set, the update of the scanned image data is rejected.

  According to the data update method of the present invention, in the block matching circuit according to any one of the first to sixth aspects of the block matching circuit, scanning is performed in the row direction within the scanning region, and priority is given to the row direction. The plurality of comparison areas are obtained by scanning in the column direction, and the scanned image data (s is 0 to (J−M)) corresponding to the pixel located in the (s + 1) th row and the Kth column of the scanning area. Is obtained as the pixel data of the comparison image data, and the pixel difference value between the target image data corresponding to the pixels in the first row and the Nth column of the target region is calculated. The scanned image data corresponding to the pixels located in the (s + 1) th row and in the first to Kth columns is updated.

  According to the first and second aspects of the block matching circuit according to the present invention, it is sufficient to store J · K pixel data as the scanned image data and M · N pixel data as the target image data. Accordingly, (J−M + 1) · (K−N + 1) · M · N pieces of comparison image data for all of the plurality of comparison regions and M · N pieces of pixel data as target image data are stored. The number of pixel data to be stored is reduced. Compared with the case of storing one piece of target image data and (J−M + 1) · (K−N + 1) pieces of pixel data corresponding to the target image data for all of the plurality of comparison regions, the pixel data is captured. Is reduced to 1 / (M · N), and the processing speed is high. The position of the comparison area that gives the minimum cumulative value is indicated by the first data.

  According to the third aspect of the block matching circuit of the present invention, data update in the second storage area is accelerated.

  According to the fourth aspect of the block matching circuit of the present invention, it becomes easy to handle cluster data regardless of whether the target image data adopts little endian or big endian.

  According to the fifth aspect of the block matching circuit of the present invention, data update in the first storage area is accelerated.

  According to the sixth aspect of the block matching circuit of the present invention, it becomes easy to handle cluster data regardless of whether scanned image data employs little endian or big endian.

  According to the seventh aspect of the block matching circuit of the present invention, the update of the scanned image data does not hinder block matching.

  According to the data update method of the present invention, the scanned image data is updated without disturbing block matching.

It is a figure which shows typically the pixel arrange | positioned at J row K column which scanning area CA employ | adopted in embodiment concerning this invention has. It is a figure which shows typically the pixel arrange | positioned at M row N column which the target area | region TA employ | adopted in embodiment concerning this invention has. In an embodiment concerning this invention, it is a figure showing typically relation between scanning field CA and comparison field CM. It is a block diagram which shows the selection circuit Smn and the arithmetic circuit Qmn. FIG. 6 is a block diagram showing a configuration in which pixel difference values Δmnst are accumulated over the entire comparison area CM for each comparison area. It is a block diagram which shows the outline of the structure by which the block matching concerning this Embodiment is performed. It is a flowchart which performs matching determination. It is a block diagram which illustrates other composition of operation circuit Qmn. 3 is a graph showing a first characteristic of a coring circuit 19; 6 is a graph showing a second characteristic of the coring circuit 19; It is a block diagram which illustrates other composition of operation circuit Qmn. It is a block diagram which illustrates other composition of operation circuit Qmn. It is a block diagram which shows the structure which calculates | requires the average value of target image data Gmn. It is a block diagram which shows the structure which calculates | requires the average value of scanning image data C (m + s) (n + t). It is a circuit diagram which illustrates the composition of matching judgment circuit MC. 4 is a timing chart showing signals RST, CAL, data LOC, a cumulative value Dst, and an operation clock CLK. It is a figure which shows the example of position information PSF. It is a block diagram which illustrates the composition of the 2nd storage area RTA.

  FIG. 1 is a diagram schematically showing pixels arranged in J rows and K columns included in a scanning area CA employed in the embodiment according to the present invention. The scanning area CA is an image area to be compared for block matching.

  Each pixel in the scanning area CA is marked with a symbol of scanned image data, which is pixel data corresponding to the pixel. For example, the scanned image data Cjk corresponds to a pixel (j is an integer from 1 to J, k is an integer from 1 to K) located in the j-th row and the k-th column.

  Here, the value “8” is adopted as the maximum values J and K that can be taken by the indices j and k, respectively. Accordingly, the scanned image data C11 to C88 correspond to the scanning area CA.

  FIG. 2 is a diagram schematically showing pixels arranged in M rows and N columns included in the target area TA employed in the embodiment according to the present invention. The target area TA is an image area serving as a comparison reference for block matching.

  Each pixel in the target area TA is marked with a symbol of target image data which is pixel data corresponding to the pixel. For example, the target image data Gmn corresponds to a pixel (m is an integer from 1 to M and n is an integer from 1 to N) located in the m-th row and the n-th column.

  Here, as the maximum values M and N that can be taken by the indices m and n, the value “4” is adopted. Accordingly, target image data G11 to G44 correspond to the target area TA.

  FIG. 3 is a diagram schematically showing the relationship between the scanning area CA and the comparison area CM in the embodiment according to the present invention. Each of the comparison areas CM has pixels arranged in M rows and N columns, and pixel data corresponding to these pixels is referred to as comparison image data.

  In FIGS. 1 to 3, the scanned image data Cjk and the target image data Gmn appear to be arranged in a matrix. However, this is for the sake of convenience, and in the drawing showing the pixels arranged in a matrix, data corresponding to each pixel is added. That is, the scanned image data Cjk and the target image data Gmn are not actually arranged in a matrix.

  Normally, the comparison area CM scans the scanning area CA in the row direction from the first row to the Jth row, and scans in the column direction from the first column to the Kth column preferentially in the row direction. The scanning is performed for each pixel. Of course, it is possible to perform so-called “thinning” for each of a plurality of pixels, but the number of comparison areas CM per scanning area CA increases and the processing amount increases. This is the case of every minute. Therefore, the present embodiment assumes such a case, and explains that there is a merit of the present invention even in such a severe situation.

  The position of the comparison area CM is the pixel at the position where the row number is the smallest and the column number is the smallest in the comparison area CM (hereinafter also referred to as “first pixel in the comparison area”), the first row, the first in the scanning area CA It can be set according to the positional relationship with the pixel in the column (hereinafter also referred to as “first pixel in the scanning region”).

  Here, the position of the comparison area CM is the position (s, t) when the deviation of the first pixel in the comparison area from the first pixel in the scanning area is the number of rows s in the row direction and the number of columns t in the column direction. Can be expressed as However, s is an integer of 0 to (J−M), and t is an integer of 0 to (K−N). When s = t = 0, the first pixel in the comparison area is not shifted from the first pixel in the scanning area.

  In the comparison area CM at the position (s, t), the target image data Gmn and the comparison image data C (m + s) (n + t) correspond. Taking the case shown in FIG. 3 as an example, the comparison area CM is at position (2, 3). Therefore, the comparison image data C34 corresponds to the first pixel in the comparison area that corresponds to the target image data G11. Considering that M + s = 6 and N + t = 7, the comparison area CM in the position (2, 3) has a matrix-like comparison image data C34 to C37, in which the comparison image data C34 and C67 form a diagonal line. C44 to C47, C54 to C57, and C64 to C67 correspond.

  In the present embodiment, comparison image data is not accumulated for all comparison areas CM. Instead, J · K pieces of scanned image data Cjk corresponding to J rows and K columns of pixels in the scanning area CA are accumulated, and every time the position (s, t) and thus the comparison area CM are updated by scanning. Then, MN comparison image data C (m + s) (n + t) are selected from the scanned image data Cjk.

  FIG. 4 is a block diagram showing the selection circuit Smn and the arithmetic circuit Qmn, both of which correspond to the pixels arranged in the mth row and the nth column in the target area TA. Therefore, as an actual circuit, the selection circuit Smn and the arithmetic circuit Qmn are provided from 1 to M for the row number m and from 1 to N for the column number n, and M · N are provided respectively.

The j _m is an integer of m to (m + J-M), a k _n is an integer of n to (n + K-N), selected (J-M + 1) · (K-N + 1) pieces of scanned image data Cj _m k _n Input to the circuit Smn. The comparison of the scanned image data Cj _m k _n image data C (m + s) (n + t) is selected and output. The selection by the selection circuit Smn is controlled by scanning information PSV that designates the number of rows s and the number of columns t indicating the position of the comparison region CM. That is, the comparison image data C (m + s) (n + t) to be calculated together with the target image data Gmn is output according to the position of the comparison area CM with respect to the scanning area CA. For example, the scanning information PSV is parallel data in which two values are taken in each of (J−M + 1) · (K−N + 1) digits, and one digit is exclusively taken in the first value.

  The target image data Gmn is input to the arithmetic circuit Qmn. In the arithmetic circuit Qmn, the multiplier (or inverter) 11 multiplies the target image data Gmn by the value “−1” (or inverts the sign). The adder 12 adds the output of the multiplier 11 and the comparison image data C (m + s) (n + t). The absolute value circuit 13 obtains the absolute value of the addition result output from the adder 12. This absolute value can be adopted as a pixel difference value Δmnst that is a value indicating a difference between mutually corresponding pixels in the target area TA and the comparison area CA.

  Since the absolute value circuit 13 functions in this way, the multiplier 11 and the adder 12 are grasped as a subtraction circuit for obtaining a difference between the comparison image data C (m + s) (n + t) and the target image data Gmn. can do.

  As a specific example, consider a selection circuit S34 and an arithmetic circuit Q34 corresponding to pixels arranged in the third row and the fourth column in the target area TA (the target pixel data G34 corresponds to this).

  25 (= 5 × 5) pieces of scanned image data C34 to C38, C44 to C48, C54 to C58, C64 to C68, and C74 to C78 are input to the selection circuit S34. When the comparison area CM is at the position (2, 3) (corresponding to the comparison area CM shown in FIG. 3), m + s = 3 + 2 = 5 and n + t = 4 + 3 = 7, and the scanning image data C57 is output from the selection circuit S34. Are output as comparison image data. This coincides with the fact that the scanned image data C57 corresponds to the pixels arranged in the third row and the fourth column in the comparison area CM shown in FIG.

  The target pixel data G34 is input to the arithmetic circuit Q34. When the comparison area CM is at the position (2, 3), the comparison image data C57 selected from the scanned image data C34 to C38, C44 to C48, C54 to C58, C64 to C68, and C74 to C78 is input to the arithmetic circuit Q34. To do. The arithmetic circuit Q34 outputs a pixel difference value Δ3423.

  FIG. 5 is a block diagram showing a configuration in which the pixel difference value Δmnst is accumulated over the entire comparison area CM for each comparison area. The accumulation circuit SUM receives M · N pixel difference values Δmnst obtained over the entire comparison area CM at a certain position (s, t), and accumulates these to output a cumulative value Dst. The accumulation may be simple addition or may be added with weighting depending on the row number m and the column number n.

  The accumulated value Dst is stored in a different register Rst for each position (s, t) based on the scanning information PSV. For example, when the comparison area CM is at the position (2, 3), the cumulative value D23 is stored in the register R23.

  Since the comparison area CM is obtained by scanning the scanning area CA, the cumulative value Dst is also serially output for each position (s, t). Based on the scanning information PSV, only one register Rst out of (J−M + 1) · (K−N + 1) registers R00 to R (JM) (KN) is activated, and a cumulative value Dst corresponding to this is obtained. Store.

  FIG. 6 is a block diagram showing an outline of a configuration in which block matching according to the present embodiment is executed. Image data PIC is input to the frame buffer FB for each frame, and under the control of the central processing unit CPU, scanned image data Cjk is stored in the first storage area RCA, and target image data Gmn is stored in the second storage area RTA. Remembered.

  Compared to the case where (J−M + 1) · (K−N + 1) · M · N pieces of comparison image data for all of the plurality of comparison areas CM and M · N pieces of pixel data as target image data are stored. In the above-described technique, it is sufficient to store J · K pixel data as scanned image data and M · N pixel data as target image data. That is, the number of pixel data to be stored is reduced.

  When the number of pixel data to be stored is reduced in this way, data access to the first storage area RCA and the second storage area RTA does not employ DMA (direct memory access), and central processing It is desirable from the viewpoint of rapid processing speed that the unit CPUs directly perform data access.

  Compared with the case of storing one piece of target image data and (J−M + 1) · (K−N + 1) pieces of pixel data corresponding to the target image data for all of the plurality of comparison regions, the pixel data is captured. The number of times is reduced to 1 / (M · N), and the processing speed is shortened.

  The matching circuit HW configured by hardware is provided with the M · N selectors Smn and the arithmetic circuit Qmn described above, which are collectively illustrated as a selector group SY and an arithmetic circuit group QY, respectively. Scanned image data Cjmkn is input from the first storage area RCA, and target image data Gmn is input from the second storage area RTA for all m and n, respectively. Scanned image data Cjmkn and target image data Gmn are input to selector group SY and arithmetic circuit group QY, respectively. The central processing unit CPU generates scanning information PSV and inputs it to the selector group SY. The position (s, t) is designated by the scanning information PSV, and M · N comparison image data C (m + s) (n + t) are output from the selector group SY to the arithmetic circuit group QY.

  The above-described cumulative circuit SUM is also provided in the matching circuit HW, and the cumulative value Dst is distributed and stored in the register Rst for each position (s, t) designated by the scanning information PSV. The (J−M + 1) · (K−N + 1) registers Rst are collectively drawn as a register group RY in FIG. However, in order to simplify the illustration, H = J-M and V = K-N are introduced in the notation in FIG.

  The matching circuit HW is further provided with a matching determination unit MC. The matching judgment unit MC receives (J−M + 1) · (K−N + 1) accumulated values Dst from the register group RY, and performs central processing on position information PSF indicating the position (s, t) from which the minimum value is obtained. Output to the processing unit CPU. Then, it is determined that the comparison area CM at the position (s, t) indicated by the position information PSF is most similar to the target area TA.

  FIG. 7 is a flowchart for performing matching determination by updating the position (s, t). Steps Z1 to Z8 are executed by the central processing unit CPU, and steps Z9 and Z10 are executed by the matching circuit HW.

  After one scanning area CA is determined, the number of rows s and the number of columns t are initialized to a value “−1” in step Z1. Thereafter, the number of rows s is increased in step Z2, and after step Z2 is executed, the number of columns t is increased in step Z3. As described above, in this embodiment, since the scanning area is scanned without performing “thinning”, the increment value of the number of rows s and the number of columns t is set to “1”.

  When step Z2 is executed for the first time after execution of step Z1, the number of rows s takes the value “0” in step Z2. That is, the row number of the first pixel in the comparison area matches the row number of the first pixel in the scanning area. When step Z3 is executed for the first time after execution of step Z2, the number of columns t takes the value “0” in step Z3. That is, the column number of the first pixel in the comparison area matches the column number of the first pixel in the scanning area.

  After execution of step Z3, in step Z4, scan information PSV for specifying the number of rows s and the number of columns t obtained in steps Z2 and Z3 is set.

  After execution of step Z4, in step Z10, the cumulative value Dst is obtained at the position (s, t) specified by the scanning information PSV and stored in the register Rst. The details of the operation of step Z10 have been described as the operations of the selector Sm, the arithmetic circuit Qmn, and the cumulative circuit SUM.

  Thereafter, in step Z5, it is determined whether or not the number of columns t has reached the value (K−N). As described above, since the maximum value of the column number t is (K−N), if the determination in step Z5 is affirmative, the column number t is initialized to the value “−1” in step Z6. If the determination in step Z5 is negative, step Z3 is executed again to increase the number of columns t. Since the processes in steps Z5, Z6, and Z3 are repeated after step Z2 is executed once, the comparison area CM is obtained by scanning in the column direction with priority over the row direction in the scanning area CA. It is done.

  Thereafter, step Z7 described later is executed (step Z7 may be omitted), and it is determined in step Z8 whether or not the number of rows s has reached a value (JM). As described above, since the maximum value of the number of rows s is (J−M), if the determination in step Z8 is negative, step Z2 is executed again to increase the number of rows s. If affirmative, position information PSF is set in step Z9, and step Z1 is executed again. Since the processes in steps Z2 to Z8 are repeated every time the scanning area CA is set, the comparison area CM is obtained by scanning in the row direction subordinately in the column direction in the scanning area CA. is there.

  Step Z7 is a process for updating the scanned image data. When step Z6 is executed, K pieces of scanned image data C (s + 1) 1 in the (s + 1) th row corresponding to the pixels located in the (s + 1) th row in the first to Kth columns of the scanning area CA. C (S + 1) K is provided for the comparison region CM, and pixel difference values Δmn (s + 1) 1 to Δmn (s + 1) K for these are obtained. Therefore, even if the scanned image data C (s + 1) 1 to C (S + 1) K are updated in step Z7, block matching is not hindered.

  Conversely, updating the scanned image data Cjk between steps Z3 to Z5 prevents block matching. Therefore, as described above, step Z7 is desirably provided after step Z5.

  More specifically, the scan image data Cjk is updated using a flag FLG indicating that data update is rejected. Since the scanning area CA is scanned in the row direction in order to obtain the comparison area CM, the flag FLG is set each time the number of lines s indicating the shift in the row direction of the comparison area CM with respect to the scanning area CA is updated (data Update is rejected). Then, after it is determined in step Z5 that the number of columns t has reached K−N, the flag FLG is reset (data update rejection is cancelled).

  FIG. 6 shows a register RF in which a flag FLG is stored. The flag FLG is set / reset by the central processing unit CPU, and the value is referred to by the central processing unit CPU.

  FIG. 8 is a block diagram illustrating another configuration of the arithmetic circuit Qmn. In contrast to the configuration shown in FIG. 4, a coring circuit 19 is added after the absolute value circuit 13. The coring circuit 19 is given a positive threshold value CRTH and the output of the absolute value circuit 13. The threshold value CRTH may be stored in the coring circuit 19.

  The coring circuit 19 is zero when the output of the absolute value circuit 13 is less than or equal to the threshold value CRTH, and non-negative from the output of the absolute value circuit 13 when the output of the absolute value circuit 13 is greater than the threshold value CRTH. A value obtained by subtracting the value of is output as a pixel difference value Δmnst.

  In this way, adopting the value obtained by coring the output of the absolute value circuit 13 with the threshold value CRTH as the pixel difference value Δmnst brings about an advantage that similarity determination strong against noise can be made.

  FIGS. 9 and 10 are graphs showing the first and second characteristics of the coring circuit 19, respectively. In each case, the value of the output ABS of the absolute value circuit 13 is adopted on the horizontal axis, and the value of the pixel difference value Δmnst is calculated. The vertical axis is adopted.

  According to the characteristics shown in FIG. 9, when the output ABS is larger than the threshold value CRTH, a value obtained by subtracting zero from the output ABS, that is, the output ABS as it is is output as the pixel difference value Δmnst. According to the characteristics shown in FIG. 10, when the output ABS is larger than the threshold value CRTH, a value obtained by subtracting the threshold value CRTH from the output ABS is output as the pixel difference value Δmnst.

  How to select the characteristics of the coring circuit 19 is determined depending on how to perform higher-order processing that employs block matching, for example, motion vector detection.

  FIG. 11 is a block diagram illustrating another configuration of the arithmetic circuit Qmn. 4 has a configuration in which a subtracting circuit for subtracting the offset OFD0 from the value of the target image data Gmn is provided in the preceding stage of the multiplier 11. The subtracting circuit includes a multiplier (or an inverter) 14 that multiplies the offset OFD0 by a value “−1” (or inverts the sign), and an adder 10 that adds the output of the multiplier 14 to the target image data Gmn. Including.

  In the configuration illustrated in FIG. 11, the arithmetic circuit Qmn further includes a selector 15 that can select whether or not to employ the offset OFD0. The output of the multiplier 14 and the value “0” are input to the selector 15, and which is output to the adder 10 is controlled by a signal OFD that sets whether or not to adopt an offset.

  By providing such a subtraction circuit, it is possible to determine the similarity with the comparison area after offsetting the target area. Further, by providing the selector 15, it is possible to select whether or not the offset is adopted. If a constant value that does not depend on the value of the target image data Gmn is employed as the offset OFD0, a so-called DC offset can be employed for the target region.

  FIG. 12 is a block diagram illustrating another configuration of the arithmetic circuit Qmn. 11 has a configuration in which a subtracting circuit for subtracting the offset OFS2 from the value of the comparison image data C (m + s) (n + t) is added to the preceding stage of the adder 12. The subtracting circuit multiplies the offset OFS2 by the value “−1” (or inverts the sign), and outputs the comparison image data C (m + s) (n +) from the output of the multiplier 17. and adder 16 for addition to t).

  In the configuration illustrated in FIG. 12, the arithmetic circuit Qmn further includes a selector 18 that can select whether or not the offset OFS2 is adopted. The output of the multiplier 17 and the value “0” are input to the selector 18, and which is output to the adder 16 is controlled by a signal OFA that sets whether or not to adopt an offset.

  By providing such a subtraction circuit, it is possible to determine the similarity to the target area after offsetting the comparison area. Further, by providing the selector 18, it is possible to select whether or not the offset is adopted. If a constant value that does not depend on the value of the comparison image data C (m + s) (n + t) is adopted as the offset OFS2, a DC offset can be adopted for the comparison region.

  FIG. 12 illustrates an example in which the offset OFS1 is adopted for the target area, and the function of the selector 15 is controlled by the signal OFA together with the selector 18. However, the configuration shown in FIG. 11 may be adopted for the offset of the target region in the configuration of FIG. In this case, whether or not the offset of the target area is adopted and whether or not the offset of the comparison area is adopted can be independently selected by the signals OFD and OFA.

  In the configuration shown in FIG. 12, the average value of the target image data Gmn and the average value of the comparison image data C (m + s) (n + t) may be employed as the offsets OFS1 and OFS2, respectively. In this case, a so-called AC offset can be adopted for each of the target area and the comparison area.

  Further, instead of the value “0” input to the selectors 15 and 18, other fixed values may be employed. In this case, the DC offset and the AC offset are switched and adopted by the signal OFA.

  FIGS. 13 and 14 are block diagrams showing a configuration for obtaining the average value of the target image data Gmn and the average value of the comparison image data C (m + s) (n + t), respectively. The configurations illustrated in FIGS. 13 and 14 may be included in the arithmetic circuit Qmn or may be provided outside the arithmetic circuit Qmn.

  FIG. 13 illustrates a configuration in which the accumulation circuit 21 outputs the sum of M · N target image data Gmn, and the division circuit 22 divides the sum by M · N. As a result, an average value of the M · N target image data Gmn is obtained as the offset OFS1.

  In FIG. 14, the summation circuit 31 outputs the sum of M · N comparison image data C (m + s) (n + t) for a certain number of rows s and number of columns t, and the division circuit 32 outputs the sum. A configuration divided by M · N is illustrated. Accordingly, M × N comparison image data C (m +) corresponding to the pixels in the (J−s + 1) th to (J−s + M) th rows and the (K−t + 1) th to (K−t + N) th columns. s) The average value of (n + t) is obtained as the offset OFS2.

  FIG. 15 is a circuit diagram illustrating the configuration of the matching determination circuit MC (see FIG. 6). The matching determination circuit MC sequentially receives the accumulated value Dst in addition to the signals CAL and RST and the data LOC. Signal CAL permits matching determination. The signal RST sets an initial value for matching determination. Once the signal RST is activated, the activation of the signal CAL starts, and the signal CAL continues to be activated while all the accumulated values Dst for a certain target region are input.

  The data LOC is data in which each of (J−M + 1) · (K−N + 1) digits takes binary values, and one digit takes the first value sequentially. It is easy to obtain the data LOC by processing the scanning information PSV, for example. Hereinafter, for simplicity of illustration, (J−M + 1) · (K−N + 1) is represented by a value W.

  FIG. 16 is a timing chart showing signals RST, CAL, data LOC, cumulative value Dst, and operation clock CLK. In synchronization with the operation clock CLK, the signal RST is activated for one cycle of the operation clock CLK. After the activation of the signal RST is completed, the signal CAL is activated for the W period of the operation clock CLK in synchronization with the operation clock CLK. The data LOC [W: 1] is exclusively activated from the first digit to the Wth digit sequentially in a period in which the signal CAL is activated in synchronization with the operation clock CLK. The accumulated value Dst is represented as D1, D2, D3,..., DW, and these change sequentially in synchronization with the operation clock CLK. For example, the cumulative value Dst is sequentially read from the register group RY, and the cumulative value Dst is input to the matching determination unit MC.

  The matching determination circuit MC has a minimum value storage circuit that stores the value MINVAL and updates the stored value MINVAL using the input cumulative value Dst when the input cumulative value Dst is less than the stored value. Yes. The following configuration is exemplified as a specific minimum value storage circuit.

  The value MINVAL is stored in the D flip-flop D1. The D flip-flop D1 outputs the value MINVAL as its Q output in synchronization with the operation clock CLK during the period in which the signal CAL is activated.

  The value MINVAL and the cumulative value Dst input sequentially are input to the comparator CMP and the selector L1.

  In the comparator CMP, the value MINVAL and the accumulated value Dst are compared. The comparator CMP is activated when the accumulated value Dst is smaller than the value MINVAL (in FIG. 15, “A> B” is added to the output terminal for outputting the first output) and the accumulated value. A second output that is activated when Dst is equal to the value MINVAL (in FIG. 15, “A = B” is added to the output terminal that outputs the second output) is output.

  The selector L1 gives the cumulative value Dst to the input terminal of the D flip-flop D1 if the first output is activated, and otherwise gives the value MINVAL to the input terminal of the D flip-flop D1.

  Through the above processing, the value MINVAL is updated with a smaller cumulative value Dst. Therefore, if all the cumulative values Dst for a certain target area (that is, W cumulative values Dst) are input, the minimum value among them is obtained as the value MINVAL from the Q output of the D flip-flop D1. . In order to obtain such an operation, it is desirable to store in the D flip-flop D1 an initial value of the value MINVAL that is greater than that assumed as the cumulative value Dst by activating the signal RST.

  The D flip-flop D2 stores binary data in each of the W digits, and outputs data stored by itself as its Q output in synchronization with the operation clock CLK during the period in which the signal CAL is activated. To do.

  Through the processing described later, the data obtained as the Q output of the D flip-flop D2 is obtained as the position information PSF. That is, the D flip-flop D2 functions as a minimum value position storage circuit that stores a position that gives the minimum value when the value MINVAL takes the minimum value of the cumulative value Dst.

  The gate G1 outputs a logical sum regarding activation of the first output and the second output of the comparator CMP. That is, the output of the gate G1 is activated when the cumulative value Dst is equal to or less than the value MINVAL.

  The gate G2 outputs a logical OR of activation for each digit of the data LOC, which is W digit data, and the Q output of the D flip-flop D2. Therefore, the output of the gate G2 is also W-digit data.

  The selector L2 selectively outputs the output of the gate G2 and the data LOC based on the second output of the comparator CMP. Specifically, the output of the gate G2 is output when the second output is activated (when the cumulative value Dst is equal to the value MINVAL), and the data LOC is output otherwise.

  The selector L3 selectively outputs the output of the selector L2 and the Q output of the D flip-flop D2 based on the output of the gate G1. Specifically, when one of the first output and the second output of the comparator CMP is activated (when the cumulative value Dst is less than or equal to the value MINVAL), the output of the selector L2 is used. Otherwise, the D flip-flop D2 is used. Q outputs are respectively output.

  As a result of the activation of the signal RST, the D flip-flop D2 employs data in which all W digits are deactivated as the initial value of the stored value.

  When the value MINVAL is updated by the accumulated value Dst, the output of the gate G1 is activated, but the second output of the comparator CMP is not activated. Therefore, the selectors L2 and L3 give the data LOC as the D input of the D flip-flop D2. As a result, the data stored in the D flip-flop D2 is updated, and the digit activated in the data causes the value MINVAL to be updated in the data LOC (that is, adopted for updating the value MINVAL). The digit is activated in synchronization with the input of the cumulative value Dst.

  Two cases are assumed when the value MINVAL is not updated by the cumulative value Dst. In the first case, the cumulative value Dst is equal to the stored value MINVAL. The accumulated value Dst at this time can also be a minimum value of the accumulated value Dst input after the signal RST is activated. Therefore, it is desirable to activate the digit activated in the data LOC synchronized with the input of the cumulative value Dst in the data stored in the D flip-flop D2.

  At this time, since both the output of the gate G1 and the second output of the comparator CMP are activated, the selectors L2 and L3 give the output of the gate G2 to the D flip-flop D2. As described above, since the gate G2 takes the logical OR of the data LOC and the Q output of the D flip-flop D2 for each digit, the digit corresponding to the cumulative value Dst that has given the minimum value so far, Both of the digits corresponding to the accumulated value Dst that invited the first case are stored in the D flip-flop D2.

  On the other hand, among the cases where the value MINVAL is not updated by the accumulated value Dst, the second case is when the accumulated value Dst is larger than the stored value MINVAL. In this case, the W-digit data stored in the D flip-flop D2 should not be updated. In this case, since the output of the gate G1 is not activated, the selector L3 gives the Q output of the D flip-flop D2 as its D input regardless of the output of the selector L2, and the data stored in the D flip-flop D2 is maintained. .

  The order in which the cumulative value Dst is input to the matching determination circuit MC depends on the reading order of the cumulative value Dst from the register group RY. Since the order of reading the cumulative value Dst from the register group RY is normally known, the digit activated in the data stored in the D flip-flop D2 is a position (s that gives the minimum value of the current cumulative value Dst (s , T). Accordingly, when the value MINVAL takes the minimum value for the cumulative value Dst, the D flip-flop D2 performs a function of storing a position that gives the minimum value.

  FIG. 17 shows an example of the Q output of the D flip-flop D2 after all the accumulated values Dst for a certain target area (that is, W accumulated values Dst) are inputted, and are grasped as position information PSF. . Here, since the third digit is activated, the comparison area CA at the position (s, t) that gives the third input cumulative value Dst after activation of the signal RST is most similar to the target area TA. It will be shown that.

  The target image data Gmn may be handled as data divided for each corresponding pixel, or may be handled as cluster data that is pixel data corresponding to a group of images belonging to the same row in the target region. Such data handling is desirable from the viewpoint of speeding up the data update of the second storage area RTA.

  FIG. 18 is a block diagram illustrating the configuration of the second storage area RTA. The central processing unit CPU outputs an address signal ADR, a write permission signal WE, and read data RD. The decoder DEC receives these signals and outputs a 4-bit enable signal that is alternatively activated. The central processing unit CPU outputs a 32-bit signal G [31: 0] as cluster data. Here, it is assumed that four columns of pixels are arranged in one row as the target area TA, and pixel data corresponding to one pixel is assumed to be 8 bits. Therefore, the number of bits of the cluster data is 32 bits.

  The second storage area RTA has selectors L11, L12, L13, L14 corresponding to the number of columns of the target area TA, and a D flip-flop having the same number of pixels as the target area TA.

  Cluster data G [31: 0] is branched into 8 bits and input to selectors L11, L12, L13, and L14. Specifically, the cluster data G [31:24] is the first input terminal of the selector L11 (the input terminal on the side labeled “1” in the figure: the same applies to the other selectors L12, L13, L14) and It is given to the second input terminal of the selector L14 (the input terminal on the side labeled “0” in the figure: the same applies to the other selectors L12, L13, and L11). Cluster data G [23:16] is applied to the first input terminal of selector L12 and the second input terminal of selector L13, and cluster data G [15: 8] is supplied to the first input terminal of selector L13 and the second input terminal of selector L12. The cluster data G [7: 0] is supplied to the input terminal, and is supplied to the first input terminal of the selector L14 and the second input terminal of the selector L11. The selectors L11, L12, L13, and L14 output any one of the cluster data input to the first input terminals and the cluster data input to the second input terminals as target image data Gm1, Gm2, Gm3, and Gm4. .

  The output of the selector L11 is given to all D input terminals of the D flip-flops arranged on the uppermost side in the horizontal direction in the drawing. The output of the selector L12 is given to all D input terminals of the D flip-flops arranged second from the top in the horizontal direction in the figure. The output of the selector L13 is given to all D input terminals of the D flip-flops arranged in the third horizontal direction in the figure. The output of the selector L14 is given to all the D input terminals of the D flip-flops arranged on the lowermost side in the horizontal direction in the drawing.

  Each 4-bit enable signal corresponds to D flip-flops arranged in four columns in the vertical direction in the figure. As described above, since the 4-bit enable signal is activated alternatively, the D flip-flops arranged in the vertical direction are activated / deactivated in synchronization with each other.

  Therefore, the D flip-flops arranged in the vertical direction in FIG. 18 store pixel data for pixels arranged in the same row of the target area TA. For example, the D flip-flops arranged in the vertical direction on the leftmost side in the drawing store target image data G11, G12, G13, and G14 as pixel data for the pixels arranged in the first row of the target area TA. The D flip-flops arranged in the vertical direction on the right side thereof store target image data G21, G22, G23, and G24 as pixel data for the pixels arranged in the second row of the target area TA. Further, the D flip-flops arranged in the vertical direction on the right side thereof store target image data G31, G32, G33, and G34 as pixel data for the pixels arranged in the third row of the target area TA. The D flip-flops arranged in the vertical direction on the rightmost side in the drawing store target image data G41, G42, G43, and G44 as pixel data for the pixels arranged in the fourth row of the target area TA.

  Now, as the bit arrangement of the cluster data G [31: 0], big endian may be adopted, or little endian may be adopted. Therefore, by controlling the selective outputs of the selectors L11, L12, L13, and L14, the cluster data G [31:24], G [23:16], and G [15 are obtained as the target image data Gm1, Gm2, Gm3, and Gm4. : 8], G [7: 0], or cluster data G [7: 0], G [15: 8], G [23:16], G [31:24] can be used. . That is, it becomes easy to handle cluster data regardless of whether little endian or big endian is adopted. FIG. 18 illustrates a case where such a function selection of the selector is controlled by the signal BYTESWAP stored in the register RB.

  The data swap as described above may be performed not only on the target image data but also on the scanned image data. Specifically, the pixel data in the scanned image data may be handled as cluster data that is pixel data corresponding to a group of images belonging to the same row in the scanned region. Such data handling is desirable from the viewpoint of speeding up the data update of the first storage area RCA.

CA scanning area Cjk, C11 to C88 scanned image data RCA first storage area TA target area Gmn, G11 to G44 target image data C (m + s) (n + t), C34 to C67 Comparative image data Δmnst Pixel difference value Qmn , Q11 to QMN arithmetic circuits L1, L2, L3, L11 to L14, Smn, S11 to SMN selector Dst cumulative value SUM cumulative circuit MC matching determination unit 11, 14 multiplier (or inverter)
10,12 Adder 13 Absolute value circuit OFS0, OFS1, OFS2 Offset CRTH threshold 19 Coring circuit MINVAL Value stored in D flip-flop D1 D1, D2 Flip-flop G1, G2 Gate LOC [0] to LOC [W] Data G [31: 0] Cluster data

Claims (8)

A first storage area that stores scan image data that is pixel data corresponding to each pixel in the J rows and K columns included in the scan area that is an image area to be compared with the block matching;
Pixels corresponding to pixels of M rows and N columns (J, K, M, N are positive integers satisfying J> M, N <K) included in the target region, which is an image region serving as a reference for block matching. A second storage area for storing target image data as data;
Pixels corresponding to each other between the comparison image data, which is pixel data corresponding to pixels in M rows and N columns included in each of the plurality of comparison regions obtained by scanning the scanning region, and the target image data M · N arithmetic circuits for obtaining a pixel difference value as a value indicating a difference between
MN selection circuits for selecting the comparison image data for each comparison area from the first storage area and outputting the selected comparison data to the arithmetic circuit;
A cumulative circuit for obtaining a cumulative value of the pixel difference values for each comparison region;
A matching determination unit that compares the cumulative value for each comparison region and identifies the comparison region that minimizes the cumulative value;
The matching determination unit
Enter the cumulative value sequentially,
A minimum value storage circuit that stores a stored value that is a value to be stored, and updates the stored value using the cumulative value that is input when the input cumulative value is less than the stored value;
A minimum value position storage circuit for storing first data taking binary values in each of (J−M + 1) · (K−N + 1) digits;
An update circuit for updating the first data,
The update circuit includes:
(J−M + 1) · (K−N + 1) takes binary values in each digit, and inputs second data in which one digit takes the first value sequentially in synchronization with the cumulative value,
A block matching circuit that updates the first data using the second data when the inputted cumulative value is less than the stored value. The update circuit includes:
When the input cumulative value is equal to the stored value, the first data is calculated using a logical sum for each digit between the second data and the first data stored immediately before. The block matching circuit according to claim 1, wherein data is updated.   3. The block matching according to claim 1, wherein pixel data in the target image data is handled as cluster data that is pixel data corresponding to a group of pixels belonging to the same row in the target area. 4. circuit. The block matching circuit according to claim 3, further comprising a plurality of selectors for swapping the cluster data for each column in the target area.   5. The block matching according to claim 1, wherein pixel data in the scanned image data is handled as cluster data that is pixel data corresponding to a group of pixels belonging to the same row in the scanning region. 6. circuit. The block matching circuit according to claim 5, further comprising a plurality of selectors for swapping the cluster data for each column in the scanning region. A register for storing a flag;
Within the scanning region, scanning is performed in a row direction from the first row to the Jth row, and scanning in a column direction from the first column to the Kth column is performed with priority to the row direction. The comparison region is obtained,
The flag is set when scanning in the column direction is performed, and the flag is reset when the scanning reaches the Kth column,
The block matching circuit according to claim 1, wherein updating of the scanned image data is rejected when the flag is set. In the block matching circuit according to any one of claims 1 to 6,
Within the scanning region, scanning is performed in a row direction from the first row to the Jth row, and scanning in a column direction from the first column to the Kth column is performed with priority to the row direction. The comparison region is obtained,
The scanned image data (s is an integer from 0 to (J−M)) corresponding to the pixel located in the (s + 1) th row and the Kth column of the scanned region is used as pixel data of the comparative image data. Pixels located in the (s + 1) -th row and in the first to K-th columns of the scanning region after the pixel difference value is obtained from the target image data corresponding to the pixels in the first row and the N-th column A data update method for updating the scanned image data corresponding to the above.

Images