JP2011210178A - System and method for processing image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system which can shorten the maximum processing time required for image processing such as template matching.SOLUTION: The image processing system searches for a matching area which matches a template image from among images for search. The image processing system includes: an operation processing part 501; an extraction processing part 502; a calculation processing part 503; and a specification processing part 504. The operation processing part 501 sequentially operates each of a plurality of continuous first bits to sequentially generate each of a plurality of discontinuous second bits. The extraction processing part 502 extracts partial areas at search positions corresponding to the second bits among positions in a continuous search order among the images for search for each second bits to be sequentially generated. The calculation processing part 503 calculates an index regarding similarity between the extracted partial area and the template image. The specification processing part 504 specifies the matching area from a plurality of partial areas on the basis of a plurality of indexes calculated for the plurality of partial areas corresponding to the plurality of second bits to specify a matching position as a position of the matching area.

Description

  The present invention relates to an image processing system and an image processing method, and more particularly to an image processing system and an image processing method for performing template matching.

  In recent years, a system equipped with a camera or installed in a vehicle safety field or a traffic safety system infrastructure field has been rapidly spreading. Many of the objectives of the system are to prevent accidents by finding people and objects from the images shown in camera images and issuing warnings and warnings. In many cases, a process of finding a person or an object from a video displayed in a camera image is realized by using a method called template matching.

  Template matching is a type of image processing for finding a partial image similar to a template image prepared in advance from the search target image. Template matching is a relatively simple method that does not require manual image feature extraction or parameter setting for searching. Therefore, template matching is used in various situations and is one of important techniques in image processing.

  In general, real-time processing is required for image processing used in the field of driving safety of automobiles. In an image processing system that requires real-time processing, it is necessary to ensure that any input image is processed within a predetermined time and the result is output. Usually, at the time of designing an image processing system, the allowable time until the processing for an input image is completed is determined in consideration of the maximum processing time. That is, the responsiveness of the image processing system is improved by improving the maximum processing time required for image processing. As a result, the image processing system can be applied to a system with more severe time constraints, and the adaptive range of the image processing system can be expanded. For this reason, there is always a demand (necessity) for shortening the maximum processing time.

  As a technique for template matching, Barnea D. I. , Silverman H .; F. "A Class of Algorithms for Fast Digital Image Registration", IEEE Trans. Comput. , Vol. c-21, no. 2, p. 179-186 (Feb. 1972) (hereinafter also referred to as non-patent document 1) describes the technology of the SSDA method in template matching. 1A and 1B are schematic diagrams illustrating a template matching state by the SSDA method described in Non-Patent Document 1. FIG. However, in FIGS. 1A and 1B, the template image is T201 (FIG. 1B), the search target image is U204 (FIG. 1A), the partial image having the same size as the template image T201 is Q203 (FIG. 1A), and the partial image Q203 is A movable area is defined as a search area P202 (FIG. 1A). At this time, the template matching is a process of searching for a partial image Q203 having the highest similarity with the template image T201. Assume that template image T201 and search region P202 have an origin (0, 0) at the upper left of the image, template image T201 has a size of M × N, and search region P202 has a size of W × H.

In the search, it is necessary to define an index indicating how similar the template image T201 and the partial image Q203 are. In the SSDA method described above, the similarity is the total sum (L1 norm) of the differences between the corresponding pixels, and the smaller the value of L1 _{(x, y)} defined by the following equation (1), Defined as having high similarity between images. Hereinafter, the process for obtaining the similarity by matching the template image of the following formula (1) and the image of the rectangular area is expressed as a collation process. Note that T _{(i, j)} and P _{(i, j} ) in Equation (1) indicate the luminance values of the pixels at the coordinates (i, j) in the template image and the search area image, respectively.

（但し ０≦ｘ＜Ｗ、かつ、０≦ｙ＜Ｈ）

(However, 0 ≦ x <W and 0 ≦ y <H)

  Here, when the matching process according to the equation (1) is performed on all the pixels in the search region P, M × N × W × H matching occurs, which is a process with a very high processing load. In the SSDA method, as in the example shown in FIGS. 1A and 1B, when only one best partial image Q203 is searched in the search region P202, the partial image Q203 that is the optimal solution is represented by coordinates (x ′, y ′). )). In that case, in the subsequent search, when it is found that the similarity (L1 norm) at other coordinates exceeds the similarity (L1 norm) at coordinates (x ′, y ′), the expression (1) It is possible to abort the calculation. That is, by omitting a part of the calculation of Expression (1), the processing load can be reduced, and the template matching process can be speeded up.

  As related techniques, there are Japanese Patent Laid-Open No. 2000-322571 (Patent Document 1) and Japanese Patent Laid-Open No. 11-195122 (Patent Document 2) which take up SSDA method and normalized correlation technology. Japanese Patent No. 4208011 (Patent Document 3) describes a technique for changing the amount of movement according to the similarity of template images by raster scanning. In addition, as a technique for improving the problems of each of the above-described techniques, Japanese Patent Application Laid-Open No. 2007-128480 (Patent Document) performs extraction by using a table in order to extract a rectangular area to be subjected to the next collation process. 4). Japanese Patent Laid-Open No. 01-181178 uses a random number generator to extract a rectangular area to be subjected to the next collation process (Patent Document 5).

JP 2000-322571 A JP 11-195122 A Japanese Patent No. 4208011 JP 2007-128480 A (US2007070951 (A1)) Japanese Patent Laid-Open No. 01-181178

Barnea D. I. , Silverman H .; F. "A Class of Algorithms for Fast Digital Image Registration", IEEE Trans. Comput. , Vol. c-21, no. 2, p. 179-186 (Feb. 1972).

  FIG. 2 is a graph showing an example of the distribution of similarity (L1 norm). In a general image (natural image) taken from a camera, adjacent rectangular areas Q203 and Q'207 have an image correlation. Therefore, it is considered that the rectangular regions Q203 and Q′207 can obtain a good similarity (L1 norm) in the vicinity of the matching position 208 (x ′, y ′). However, conversely, as shown in FIG. 2, the degree of similarity (L1 norm) obtained from the extracted rectangular area is considered to rapidly deteriorate as the distance from the matching position 208 (x ′, y ′) increases.

  For this reason, in the SSDA method, when a matching position such as the matching position 208 (x ′, y ′) in FIG. 2 can be found at an early stage of search (upper left of the image), the calculation is quickly performed by truncating the calculation of Expression (1). Template matching processing is possible. However, when finding a matching position at the end of the search (lower right of the image), it is not possible to increase the speed by truncating the calculation of Equation (1). Therefore, in the SSDA method, the processing time until the search is completed is greatly influenced by the location of the matching position between the template image in the search area and the image in the rectangular area.

  Hereinafter, means for solving the problem will be described using the numbers and symbols used in the embodiments for carrying out the invention. These numbers and symbols are added with parentheses in order to clarify the correspondence between the description of the claims and the mode for carrying out the invention. However, these numbers and symbols should not be used for interpreting the technical scope of the invention described in the claims.

  The image processing system of the present invention searches for a matching area (Q ′) that matches the template image (T) from the search target image (U). This image processing system includes an operation processing unit (501), an extraction processing unit (502), a calculation processing unit (503), and a specific processing unit (504). The operation processing unit (501) sequentially operates each of a plurality of continuous first bits (r) to sequentially generate each of a plurality of discontinuous second bits (s). For each second bit (s) that is sequentially generated, the extraction processing unit (502) searches for the second bit (s) in the continuous search order position in the search target image (U). The partial region (Q) is extracted at the position (x, y). The calculation processing unit (503) calculates an index (L1) related to the similarity between the extracted partial region (Q) and the template image (T). The specific processing unit (504), based on the plurality of indices (L1) calculated for the plurality of partial regions (Q) corresponding to the plurality of second bits (s), from the plurality of partial regions (Q) to the matching region (Q) is specified, and the matching position (x ′, y ′) as the position of the matching region (Q) is specified.

  The present invention performs a bit operation process of successive first bits (r) like the value of the number of searches up to the present, and next time a search position (x, y) (partial region (Q)) for evaluating similarity Generation of a discontinuous second bit (s) for. Then, a partial region (Q) at a position (x, y) corresponding to the discontinuous second bit (s) in a continuous search order (for example, a space filling curve represented by a Hilbert curve and a Z order) is extracted. . Thereby, the partial area (Q) for which the similarity is evaluated next can be extracted so that the correlation of the images between the partial areas (Q) becomes lower. As a result, it is possible to extract the partial region (Q) without deviation from the entire search target image (U). As a result, a search independent of the appearance position of the match position (x ′, y ′) (match area (Q)) can be performed. As a result, the processing time required for template matching can be leveled. Further, it is not necessary to prepare a special storage device for extracting the partial area (Q) whose similarity is evaluated next time, and the extraction position of the next partial area (Q) can be uniquely determined by a calculation operation or the like. It can be applied in an environment where resource-saving operation is required. In addition, by using a continuous search order (for example, a space filling curve typified by a Hilbert curve and a Z order), the entire search region (P) is searched without extracting the partial region (Q). It can be performed.

The image processing method of the present invention searches for a matching area (Q ′) that matches the template image (T) from the search target image (U). In this image processing method, each of a plurality of continuous first bits (r) is sequentially operated to sequentially generate each of a plurality of discontinuous second bits (s); For each bit (s), a partial region (Q) is extracted at a search position (x, y) corresponding to the second bit (s) out of consecutive search order positions in the search target image (U). A step of calculating an index (L1) related to the similarity between the extracted partial area (Q) and the template image (T), and a plurality of partial areas (Q) corresponding to the plurality of second bits (s) ) Based on the plurality of indices (L1) calculated for the matching region (Q) from the plurality of partial regions (Q), and the matching position (x ′, y ′) as the position of the matching region (Q) And a step of specifying.
A program according to the present invention causes a computer as an image processing system to execute the above-described image processing method.
In these image processing methods and programs of the present invention, the same operations and effects as those of the image processing system can be obtained.

  According to the present invention, it is possible to provide an image processing system and an image processing method capable of reducing the maximum processing time required for image processing such as template matching.

FIG. 1A is a schematic diagram showing a state of template matching by the SSDA method described in Non-Patent Document 1. FIG. 1B is a schematic diagram showing a state of template matching by the SSDA method described in Non-Patent Document 1. FIG. 2 is a graph showing an example of the distribution of similarity (L1 norm). FIG. 3 is a block diagram showing an example of the configuration of the image processing system according to the first embodiment of the present invention. FIG. 4 is a block diagram showing a configuration of the image processor of FIG. FIG. 5 is a block diagram showing another example of the configuration of the image processing system according to the first embodiment of the present invention. FIG. 6 is a block diagram illustrating the configuration of the image processing processor and the external arithmetic device in FIG. FIG. 7 is a flowchart showing an example of the operation (image processing method) of the image processing system according to the first and second embodiments of the present invention. FIG. 8 is a schematic diagram showing a state in which bits are converted in reverse order with respect to 8-bit data r in step S103. FIG. 9 is a schematic diagram showing a state in which a rectangular area to be collated next time in step S104 is extracted. FIG. 10 is a schematic diagram showing a state where the data r of 8 bits in step S103 is rotated left by 2 bits. FIG. 11 is a schematic diagram showing a state in which a rectangular region to be subjected to the next collation process in step S104 is extracted. FIG. 12 is a schematic diagram illustrating a state in which a rectangular area to be collated next time in step S104 is extracted. FIG. 13 is a schematic diagram illustrating a state in which a rectangular region to be collated next time in step S104 is extracted. FIG. 14A is a schematic diagram showing the concept of the present invention. FIG. 14B is a schematic diagram showing the concept of the present invention.

  Embodiments of an image processing system and an image processing method according to the present invention will be described below with reference to the accompanying drawings.

(First embodiment)
The configuration of the image processing system according to the first embodiment of the present invention will be described. FIG. 3 is a block diagram showing an example of the configuration of the image processing system according to the first embodiment of the present invention. The image processing system 1 includes an image processing processor 302, a host microcomputer (host microcomputer) 301, a camera 303, and a monitor 304. The image processor 302 is connected to the host microcomputer 301, the camera 303, and the monitor 304. The camera 303 captures an image and outputs it to the image processor 302. The monitor 304 displays the output of the image processor 302.

  FIG. 4 is a block diagram showing a configuration of the image processor of FIG. The image processor 302 includes a CPU 402, a (template image storage) memory 403, a (search target image storage) memory 404, and a template matching processing unit 405. The memory 403 stores a template image. The memory 404 stores a search target image as an image to be obtained from the camera 303 or the like and subjected to template matching processing. The template matching processing unit 405 holds software that performs template matching processing. When the CPU 402 executes the software, the template matching process (image processing method) is realized. The software performs the template matching process on the data in the memories 403 and 404.

  Here, the template matching process specifies and extracts the memory address of a rectangular area (partial area that is an area of a partial image) for performing an operation process for manipulating the bit order exemplified by bit reverse ordering and the next matching process. This is an image processing method that includes an extraction process to be performed, a calculation process for the similarity L1 norm, and a specifying process for specifying a matching position of a rectangular area (partial area) that matches the template image. The template matching processing unit 405 includes, as software, an operation processing unit 501 that performs the operation processing, an extraction processing unit 502 that performs the extraction processing, a calculation processing unit 503 that performs the calculation processing, and the specific processing. And a specific processing unit 504 that implements the above.

  FIG. 5 is a block diagram showing another example of the configuration of the image processing system according to the first embodiment of the present invention. The image processing system 1a further includes an external arithmetic device 305 (hardware) connected to the image processing processor 302, as compared with the case of FIG. FIG. 6 is a block diagram illustrating the configuration of the image processing processor and the external arithmetic device in FIG. The external arithmetic device 305 includes a bit manipulation measure 407 and an address determination device 408 connected to the image processor 302. In this image processing system 1a, the bit operation device (operation processing unit) 407 executes an operation process for operating a bit order exemplified by bit reverse order. An address determination device (extraction processing unit) 408 executes an extraction process for specifying and extracting a memory address of a rectangular area (partial area) to be collated next time. When the CPU 402 executes the software of the template matching processing unit 405, template matching processing (similarity L1 norm calculation processing, specific processing for specifying a matching position of a rectangular region (partial region) that matches the template image) is performed. Realize. That is, the template matching process (image processing method) is realized by the execution of the external arithmetic unit 305 (hardware) and the software of the CPU 402 (calculation processing unit 503 and specific processing unit 504).

  In the present embodiment, the image processing system 1 shown in FIGS. 3 and 4 or the image processing system 1a shown in FIGS. 5 and 6 can be used. However, these configurations are examples, and other configurations may be used as long as the following functions and operations can be realized.

  Next, the operation (image processing method) of the image processing system according to the first embodiment of the present invention will be described. In the present embodiment, the number of searches up to the present time is converted in bit reverse order by the software of the image processor 302 (image processing system 1) or the hardware of the external arithmetic unit 305 (image processing system 1a), and the Hilbert curve Based on the above, the memory address of the rectangular area to be checked next time is specified.

  FIG. 7 is a flowchart showing an example of the operation (image processing method) of the image processing system according to the first embodiment of the present invention.

(1) Step S101
The operation processing unit sets the maximum integer value that can be set by the image processor 302 as the threshold value Th as an initial value for the threshold value Th for abortion.

(2) Step S102
The operation processing unit starts a search loop for a search area using the variable r. That is, the initial value is r = 0, and the search loop is executed in the range of r <2 ^2D (D is a natural number).

(3) Step S103
The operation processing unit maps the variable r to the variable s using the function f defined in Expression (2). If the size of the search area is 2 ^D x2 ^D = 2 ^2D , the function f is defined below.
s = f (r) (where 0 ≦ r <2 ^2D and D is a natural number) (2)
Here, the function f is a process that reverses the order of the MSB (Most Significant Bit) and LSB (Least Significant Bit) with respect to the binary value r having a 2D bit length. The function f can be defined by the following equations (3) and (4). Note that b _k is the value of bit position k in r and is 0 or 1.
r = b _2D-1 b _2D-2 b _2D-3 ... b ₂ b ₁ b ₀ (3)
f (r) = b ₀ b ₁ b ₂ ... b _2D-3 b _2D-2 b _2D-1 (4)
(However, D is a natural number and b _k is 0 or 1)
As an example, FIG. 8 shows a state in which bits are converted in reverse order to 8-bit data r. From FIG. 8, it can be seen that the most significant bit at r is converted to the least significant bit of s.

(4) Step S104
The extraction processing unit determines the position of the search coordinates (x, y). In the following equation (5), Hx (s) is the x coordinate of the sth coordinate selected in the search order based on the space filling curve, and Hy (s) is the sth in the search order based on the space filling curve. This is the y coordinate of the coordinates selected for.
(X, y) = (Hx (s), Hy (s)) (5)
(However, 0 ≦ s <2 ^2D and D is a natural number)
As an example, FIG. 9 shows a state in which a rectangular region to be collated next time is extracted. In this figure, r (belonging to the R space 205) is converted to the search target image 204 having the search region P202 of 16 × 16 (= 2 ⁴ × 2 ⁴ ; D = 4) by the function f to reverse the bit order. After obtaining s = f (r) (belonging to S space 206), mapping to the position in the search order (indicated by an arrow in search area P202) based on the Hilbert curve, which is one of the space filling curves, A rectangular area 203 for performing the matching process is extracted.
By incrementing the variable r on the R space 205, a discontinuous point on the search area P202 (example: 0th selected coordinate (0,0), 32nd selected coordinate (8,8) ), The 16th selected coordinate (8, 0),...) Can be selected as the corresponding point. That is, it is possible to increase the possibility of extracting a rectangular area 203 farther away from the rectangular area 203 currently being processed as the next rectangular area 203. As a result, the extracted rectangular area 203 is not concentrated locally in the search target area P202, and the entire search area can be searched widely.

(5) Step S105
The calculation processing unit initializes the initial value of the accumulated value of the similarity (L1 norm) at the coordinates (x, y) defined by the above formula (1) with 0. That is, L1 (x, y) = 0.

(6) Step S106
The calculation processing unit starts a loop of the template image height by the variable n. That is, the initial value is n = 0, and the loop is executed in the range of n <N (N is the height of the template image T201 (FIG. 1B)).

(7) Step S107
The calculation processing unit starts a loop of the template image width by the variable m. That is, the initial value is m = 0, and the loop is executed in the range of m <M (M is the width of the template image T201 (FIG. 1B)).

(8) Step S108
The calculation processing unit performs accumulation processing of similarity (L1 norm). The double loop process of steps S106 to S111 is a process (collation process) for calculating the above equation (1).

(9) Step S109
The calculation processing unit compares the similarity (L1 norm) up to the present and the threshold Th, and if the accumulated value of the similarity (L1 norm) exceeds the threshold Th (set in step S101), the process branches to step S114. To do.

(10) Step S110
When the variable m <M, the calculation processing unit increments m by 1 (m = m + 1) and branches to step S107.
Otherwise, the template image width loop with variable m is terminated.

(11) Step S111
When the variable n <N, the calculation processing unit increments n by 1 (n = n + 1) and branches to step S106.
Otherwise, the loop of the template image height by the variable n is terminated.

(12) Step S112
When the specific processing unit finds the best similarity (minimum L1 norm), it updates the threshold Th to the value of the found best similarity (L1 norm). This step S112 is performed only when the best similarity (minimum L1 norm) is found.

(13) Step S113
When the specific processing unit finds the best similarity (minimum L1 norm), it registers the candidate coordinates (x, y) of the matching position in (x ′, y ′). This step S113 is performed only when the best similarity (minimum L1 norm) is found.

(14) Step S114
When the variable r < ^22D , the operation processing unit increments r by 1 (r = r + 1), and branches to step S102.
Otherwise, the search loop of the search area P202 with the variable r is terminated.

(15) Step S115
The specific processing unit outputs the matching position (x ′, y ′) to the monitor 304.

  As described above, the operation (image processing method) of the image processing system according to the first embodiment of the present invention is performed.

  In the present embodiment, an image processor or an external arithmetic device performs bit operation processing (for example, processing for converting MSB and LSB in reverse order) for the number of searches (variable r) up to the present time, A rectangular area for performing a matching process is extracted. At this time, the mapping target is a function in which the search order is continuous (that is, a function that can be differentiated in many sections, such as a space-filling curve represented by a Hilbert curve, and is searched in the order of high image correlation. ), By performing the bit manipulation process, coordinates that are more discontinuous can be selected as the extraction position of the next rectangular area. This increases the possibility of extracting a rectangular area with less image correlation, that is, a rectangular area farther away from the currently processed rectangular area as the next rectangular area, compared to the location searched to date. be able to.

  In the SSDA method of Non-Patent Document 1, there is a problem that the processing time until search completion greatly affects the matching position between the template image and the partial image. However, in the present embodiment, for the reasons described above, the extracted rectangular area is not concentrated locally in the search target area, and the search is performed widely over the entire search area, so that the processing time required for the search is leveled. (Reduction of maximum processing time) is possible.

  For natural images, a rectangular area with little image correlation is extracted as the next rectangular area, so that the threshold for search termination is rapidly converged to the similarity value obtained at the collation position. Can do. For the reasons described above, when compared with the SSDA method of Non-Patent Document 1, the similarity close to the similarity obtained at the collation position is set as the threshold at an early stage of the search. Therefore, the censoring process works effectively even at an early stage of search, and when compared with the SSDA method of Non-Patent Document 1, it is expected that the maximum processing time until the search is completed and the average processing time are shortened. .

  In Japanese Patent Application Laid-Open No. 2007-128480, it is necessary to prepare a table in the storage device in order to extract a rectangular area to be collated next time. However, in the present embodiment, since the next rectangular area is extracted by a mathematical expression, it is not necessary to prepare a table in advance.

  In addition, Japanese Patent Laid-Open No. 01-181178 has a problem of overlapping search for the previously described rectangular area, and a problem that makes it difficult to search the entire search target image because of processing based on random numbers. However, in the present embodiment, since the number of searches up to the present time corresponds to the function (space filling curve, Hilbert curve) in which the search order is continuous, there is no overlap search for the previously described rectangular area, In addition, the entire search area can be easily searched.

(Second Embodiment)
In the second embodiment of the present invention, the configuration of the image processing system is the same as that of the first embodiment, but the operation (image processing method) of the image processing system is different. Hereinafter, the operation (image processing method) of the image processing system will be described.

  FIG. 7 is a flowchart showing an example of the operation (image processing method) of the image processing system according to the second embodiment of the present invention. The operation (image processing method) of the image processing system of the present embodiment is the same as that of the first embodiment except that step S103 is different. Accordingly, step S103 will be described below.

  In step S103 of the present embodiment, the coordinates (variable s) on the space filling curve are acquired based on the result of the bit rotation process. Specifically, the operation is as follows.

(3) Step S103:
The operation processing unit maps the variable r to the variable s using the function f defined in Expression (2). If the size of the search area is 2 ^D x2 ^D = 2 ^2D , the function f is defined below.
s = f (r) (where 0 ≦ r <2 ^2D and D is a natural number) (2)
Here, the function f is a process of rotating a bit with respect to a binary value r having a 2D bit length. The function f can be defined by the following expressions (6), (7), and (8). Note that b _k is the value of bit position k in r and is 0 or 1. In Expression (7), c is the number of bits rotated in the left direction. Further, mod means remainder calculation, and the remainder obtained by dividing (n + c) by 2D is set to g (n).
r = b _2D-1 b _2D-2 b _2D-3 ... b ₂ b ₁ b ₀ (6)
g (n) = (n + c) mod (2D) (7)
f (r) = b _{g (2D-1)} b _{g (2D-2)} b _{g (2D-3)} ... b _{g (2)} b _{g (1)} b _{g (0)} ... (8)
(However, D, n, and c are natural numbers, and b _k is 0 or 1.)
As an example, FIG. 10 shows a state where data is rotated left by 2 bits with respect to 8-bit data r. As can be seen from FIG. 10, the lower 2 bits are moved to the upper 2 bits and the upper 6 bits are moved to the right as a result of the rotation.

FIG. 11 shows the result of the extraction processing unit executing step S104 based on s = f (r) in step S103. That is, FIG. 11 shows a state in which a rectangular area to be collated next time is extracted. In this figure, s = f (r) (S space 206) is obtained by rotating r (belonging to the R space 205) to the left by 2 bits with the function f with respect to the search target image 204 having the 16 × 16 search region P202. To the search order (indicated by an arrow in the search area P202) based on the Hilbert curve, which is one of the space-filling curves, thereby extracting the rectangular area 203 to be subjected to the next matching process. ing.
By incrementing the variable r on the R space 205, a discontinuous point on the search area P202 (example: 0th selected coordinate (0,0), 16th selected coordinate (8,0)) ), The 32nd coordinate (8, 8),...) Can be selected as the corresponding point. That is, it is possible to increase the possibility of extracting a rectangular area 203 farther away from the rectangular area 203 currently being processed as the next rectangular area 203. As a result, the extracted rectangular area 203 is not concentrated locally in the search target area P202, and the entire search area can be searched widely.

  Also in the present embodiment, the problems of non-patent documents and patent documents can be solved as in the first embodiment. Also in this embodiment, the same effect as that of the first embodiment can be obtained.

  In addition, the rectangular area to be subjected to the collation process next time is uniquely extracted by using a mathematical formula, so that it is not necessary to prepare a table in advance and it is possible to ensure that an overlapping search with respect to the existing rectangular area does not occur.

  Further, as the function f of the expression (2), the bit reverse normalization process is used in the first embodiment, and the bit rotation process is used in the second embodiment. However, the present invention is not limited to this example. Basically, as long as the function f is a process in which the pre-conversion value r and the post-conversion value s are mapped on a one-to-one basis, an overlapping search for the previously described rectangular area does not occur. For this reason, it is possible to improve the processing performance by redefining the function f according to the input search target image.

(Third embodiment)
In the third embodiment of the present invention, the configuration of the image processing system is the same as that of the first embodiment, but the operation (image processing method) of the image processing system is different. Hereinafter, the operation (image processing method) of the image processing system will be described.

  FIG. 7 is a flowchart showing an example of the operation (image processing method) of the image processing system according to the third embodiment of the present invention. The operation (image processing method) of the image processing system of the present embodiment is the same as that of the first embodiment, except that step S104 is different. Accordingly, step S104 will be described below.

  In step S104 of the present embodiment, the coordinates (variable s) on the space filling curve are acquired by the address determination device based on the Z order. Specifically, the operation is as follows.

(4) Step S104
The extraction processing unit determines the position of the search coordinates (x, y). In the following equation (5), Hx (s) is the x coordinate of the sth coordinate selected in the search order based on the space filling curve, and Hy (s) is the sth in the search order based on the space filling curve. This is the y coordinate of the coordinates selected for.
(X, y) = (Hx (s), Hy (s)) (5)
(However, 0 ≦ s <2 ^2D and D is a natural number)
However, in this step S104, the Z order is used as shown in FIG. That is, FIG. 12 shows a state in which a rectangular region to be subjected to the next collation process is extracted using Z-ordering (Z-Ordering). In this figure, r (belonging to the R space 205) is converted to the search target image 204 having the 16 × 16 search region P202 by the function f in the bit reverse order, and s = f (r) (belonging to the S space 206). ), And then mapping to the position in the search order (indicated by an arrow in the search area P202) based on the Z-ordering curve which is one of the space-filling curves, the rectangular area where the next matching process is performed 203 is extracted.
By incrementing the variable r on the R space 205, a discontinuous point on the search area P202 (example: 0th selected coordinate (0, 0), 32nd selected coordinate (0, 8) ), The 16th selected coordinate (8, 0),...) Can be selected as the corresponding point. That is, it is possible to increase the possibility of extracting a rectangular area 203 farther away from the rectangular area 203 currently being processed as the next rectangular area 203. As a result, the extracted rectangular area 203 is not concentrated locally in the search target area P202, and the entire search area can be searched widely.

  Also in the present embodiment, the problems of non-patent documents and patent documents can be solved as in the first embodiment. Also in this embodiment, the same effect as that of the first embodiment can be obtained.

  In addition, the rectangular area to be subjected to the collation process next time is uniquely extracted by using a mathematical formula, so that it is not necessary to prepare a table in advance, and it can be ensured that there is no duplication search for the already-existing rectangular area.

  In addition, as Hx (s) and Hy (s) in Expression (5), the Hilbert curve is used in the first embodiment, and the Z order is used in the third embodiment. However, the present invention is not limited to this example. Basically, any search order may be used as long as the search order used is a search order that covers all pixels in the search region. For this reason, it is possible to improve the processing performance by redefining the search order according to the input search target image.

(Fourth embodiment)
In the fourth embodiment of the present invention, the configuration of the image processing system is the same as that of the second embodiment, but the operation of the image processing system (image processing method) is different. Hereinafter, the operation (image processing method) of the image processing system will be described.

  FIG. 7 is a flowchart showing an example of the operation (image processing method) of the image processing system according to the fourth embodiment of the present invention. The operation (image processing method) of the image processing system of the present embodiment is the same as that of the second embodiment except that step S104 is different. Accordingly, step S104 will be described below.

  In step S104 of the present embodiment, the coordinates (variable s) on the space filling curve are acquired by the address determination device based on the Z order. Specifically, the operation is as follows.

(4) Step S104
The extraction processing unit determines the position of the search coordinates (x, y). In the following equation (5), Hx (s) is the x coordinate of the sth coordinate selected in the search order based on the space filling curve, and Hy (s) is the sth in the search order based on the space filling curve. This is the y coordinate of the coordinates selected for.
(X, y) = (Hx (s), Hy (s)) (5)
(However, 0 ≦ s <2 ^2D and D is a natural number)
However, in this step S104, the Z order is used as shown in FIG. That is, FIG. 13 shows a state in which a rectangular region to be subjected to collation processing next time is extracted using the Z order. In this figure, r (belonging to the R space 205) is converted to the search target image 204 having the 16 × 16 search region P202 by the function f in the bit reverse order, and s = f (r) (belonging to the S space 206). ), The rectangular area 203 to be subjected to the next matching process is extracted by mapping to the position in the search order (indicated by an arrow in the search area P202) based on the Z-order curve that is one of the space filling curves. Yes.
By incrementing the variable r on the R space 205, a discontinuous point on the search area P202 (example: 0th selected coordinate (0,0), 16th selected coordinate (8,0)) ), The 32nd coordinate (0, 8),...) Can be selected as a corresponding point. That is, it is possible to increase the possibility of extracting a rectangular area 203 farther away from the rectangular area 203 currently being processed as the next rectangular area 203. As a result, the extracted rectangular area 203 is not concentrated locally in the search target area P202, and the entire search area can be searched widely.

  Also in this embodiment, the problems of non-patent documents and patent documents can be solved as in the second and third embodiments. Also in this embodiment, the same effect as in the second and third embodiments can be obtained.

According to the above embodiments of the present invention, the following effects can be obtained.
As a first effect, there is leveling of processing time required for searching. The reason is that the search positions are not concentrated in the local area but are distributed and searched. As a second effect, there is an increase in search processing speed. The reason is that, since the search is performed with priority given to the region having a lower image similarity from the previous search region, the threshold for search termination can be lowered more rapidly. As a third effect, a device for storing the next search position becomes unnecessary. The reason is that the rectangular area to be subjected to the next collation processing is uniquely extracted by a mathematical formula, and it is not necessary to prepare a table in advance. As a fourth effect, an overlapping search for the previously described rectangular area does not occur. The reason for this is that a duplicate search does not occur by mapping with a mathematical expression in the search order (space filling curve) that always passes through the search region only once instead of a search based on random numbers.

14A and 14B are schematic views showing the concept of the present invention. 14A and 14B, the template image is T201 (FIG. 14B), the search target image is U204 (FIG. 14A), the partial image having the same size as the template image T201 is Q203 (FIG. 14A), and the partial image Q203 is A movable area is defined as a search area P202 (FIG. 14A).
In the present invention, a bit operation process (for example, a process of switching the order of MSB and LSB in reverse order) is performed by an image processor or an external arithmetic unit, and a collation process is performed next time. The memory address of the area 203 is specified. Then, by mapping to a continuous search order (for example, a space-filling curve typified by a Hilbert curve), a region with a lower correlation of the image (a rectangle that is not adjacent to the current rectangular region 203 “0” and is not adjacent) The areas 203 “1”, “2”, “3”, “4”, “5”, “6”,...) Can be extracted as the rectangular area 203 to be subjected to collation processing next time. As a result, the rectangular area 203 can be extracted from the entire search target image U204 without deviation, and a search that does not depend on the appearance position of the matching position is possible. As a result, an effect of leveling the processing time required for template matching can be obtained.
In addition, the present invention does not need to prepare a special storage device for extracting a rectangular area to be template-checked next time, and can uniquely determine the extraction position of the next rectangular area by a mathematical expression. It can be applied in an environment where operation is required.
In addition, in the present invention, by using the search order represented by the space filling curve (the space filling curve is not limited to the Hilbert curve and the Z order), the entire search area can be extracted without extracting the previously described rectangular area. Search.

  The present invention is not limited to the embodiments described above, and it is obvious that the embodiments can be appropriately modified or changed within the scope of the technical idea of the present invention.

1, 1a Image processing system 201 Template image (T)
202 Search target area (P)
203 Rectangular area (Q)
204 Search target image (U)
205 R space 206 S space 207 Rectangular area Q ′
208 Coordinate coordinates (x ', y')
301 Host microcomputer (host microcomputer)
302 Image processing processor 303 Camera 304 Monitor 305 External arithmetic unit 402 CPU
403 Template image storage memory (frame buffer)
404 Search target image storage memory (frame buffer)
405 Template matching processing section (software section)
406 External arithmetic unit 407 Bit operation device (hardware part)
408 Address determination device (hardware part)
501 Operation processing unit 502 Extraction processing unit 503 Calculation processing unit 504 Specific processing unit

Claims (10)

An image processing system for searching a matching area that matches a template image from search target images,
An operation processing unit that sequentially operates each of a plurality of consecutive first bits and sequentially generates each of a plurality of discontinuous second bits;
An extraction processing unit that extracts a partial region at a search position corresponding to the second bit out of positions in a continuous search order in the search target image for each second bit generated sequentially;
A calculation processing unit that calculates an index related to the similarity between the extracted partial region and the template image;
Based on the plurality of indices calculated for the plurality of partial areas corresponding to the plurality of second bits, the matching area is identified from the plurality of partial areas, and the matching position is identified as the position of the matching area An image processing system comprising: a specific processing unit. The image processing system according to claim 1,
The image processing system, wherein the operation processing unit sequentially generates each of the plurality of second bits by processing for reversing the bits of each of the plurality of first bits or processing for rotating the bits. The image processing system according to claim 2,
The image processing system, wherein the extraction processing unit extracts the partial region from the search target image at the search position in which the second bit is mapped to a search order position based on a space filling curve. The image processing system according to claim 3.
The image processing system, wherein the space filling curve is a Hilbert curve or a Z-order curve. The image processing system according to claim 3.
The image processing system, wherein the calculation processing unit calculates an L1 norm as an index related to the similarity. An image processing method for searching a matching region that matches a template image from search target images,
Sequentially operating each of a plurality of consecutive first bits to sequentially generate each of a plurality of discontinuous second bits;
Extracting a partial region at a search position corresponding to the second bit out of consecutive search order positions in the search target image for each second bit generated sequentially;
Calculating an index related to the similarity between the extracted partial region and the template image;
Based on the plurality of indices calculated for the plurality of partial areas corresponding to the plurality of second bits, the matching area is identified from the plurality of partial areas, and the matching position is identified as the position of the matching area An image processing method comprising the steps of: The image processing method according to claim 6,
In the step of sequentially generating each of the plurality of second bits, each of the plurality of second bits is sequentially generated by a process of reverse-ordering the bits or a process of rotating the bits. An image processing method comprising the step of: The image processing method according to claim 7.
The step of extracting the partial region includes the step of extracting the partial region from the search target image at the search position where the second bit is mapped to a position in the search order based on a space filling curve. . The image processing method according to claim 8.
The step of specifying the match position includes a step of calculating an L1 norm as an index related to the similarity.   A program causing a computer as an image processing system to execute the image processing method according to any one of claims 6 to 9.

Images