JP2009199250A - Object detection system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object detection system for properly setting a decision reference value for deciding whether a template image is similar to an object in the case of extracting an object from an image as the object of retrieval such as a satellite image by using a template image including an object. <P>SOLUTION: When an object to be detected as an object is designated on a sample image, a pixel block whose shape is the same as the shape of the a template image is extracted from the sample image corresponding to the designated position, and as for both the pixel block and the template image, the collation of luminance histograms (S201), the collation of gradation fluctuation histograms (S202), the calculation of inter-centroid distance (S203), the collation of inertial moments (S204), the calculation of a gray gradation pixel difference (S205), and the calculation of normalization correlation coefficient (S206) are performed, and the values of 6 similarity evaluation values are updated. As for a prescribed position in the neighborhood of the designated position, the above processing is performed, and a decision reference value is set based on the minimum or maximum value of the six similarity evaluation values. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technique for detecting a target pattern from an image, and more particularly to a pattern detection technique suitable for a panchromatic satellite image.

  Land observation and earth observation satellites such as QuickBird (DigitalGlobe, USA), ALOS “Daichi” (Japan AXA), and TERRA / ASTER (Japan ERSDAC) orbit around the Earth in a polar orbit at an altitude of about 450 km. It is equipped with sensors and synthetic aperture radar (SAR), continuously scanning the ground, and transmitting image data to the earth station on a regular basis. Three types of images are mainly used, and the optical sensor's multi-spectral mode (color photo) can be photographed at night in a 4-band configuration that adds the near-infrared to the RGB band of visible light. It is effective for collecting disasters and other situations. The SAR image is a radar image generated by radio waves and transmits even when it is cloudy. Therefore, the image can be collected even when the weather is bad, and a ship in operation can be measured using the Doppler effect (see Patent Document 1). . Among them, the panchromatic mode (monochrome photo) of the optical sensor has a resolution of less than 1 m (the world's highest resolution QuickBird 0.6 m) that can identify passenger cars on the ground, and is useful for monitoring traffic conditions. With regard to traffic monitoring, aerial photography with a surveillance camera, helicopter, or aircraft installed on the ground provides clearer images, but it is difficult to observe a wide area at the same time, and it is hidden behind buildings. In recent years, satellite images have attracted attention due to the problem that a blind spot always occurs.

In addition, a target detection method using template matching that specializes in aircraft and ships (see Patent Document 2), a template is prepared in advance, and pixel blocks of the same size as the template are extracted while sequentially scanning from the first pixel of the image data A method of matching with a template (see Patent Document 3) has also been proposed.
Japanese Patent No. 3863014 Japanese Patent No. 3978979 JP 2001-67470 A

  However, the technique disclosed in Patent Document 2 has a problem in that a lot of search noise is included in the search result because the matching criterion is set in advance so that there is no search omission. In addition, the method of Patent Document 3 also has a problem that it is difficult to set an optimum determination criterion.

  Accordingly, the present invention uses a template image including a target to determine whether the template image and the target are similar when extracting the target from an image to be searched such as a satellite image. It is an object to provide a target detection system capable of appropriately setting a reference value.

  In order to solve the above-described problem, in the first aspect of the present invention, a template image including a target is used to detect a target from a search target image that is a search target, and the template image is stored. A template storage unit, and a pixel block having the same size as each template image is extracted from the sample image at each pixel position within a predetermined range from a pixel position designated on a sample image including a target sample; A criterion value setting means for calculating a similarity evaluation value between the image and the pixel block, and setting a value obtained by calculating a predetermined value for the similarity evaluation value as a criterion value; While sequentially extracting the extraction position from the pixel position to the final pixel position, the pixel block having the same size as each template image is searched for. Similarity determination means for extracting similarity from the target image, calculating a similarity evaluation value between the template image and the pixel block, and determining the similarity by comparing the similarity evaluation value with the set criterion value; When the similarity determination unit determines that there is similarity, the template image ID at that time, the pixel position for specifying the position of the pixel block on the search target image, and the rotation angle of the template image are output. A target detection system having search result output means is provided.

  According to the first aspect of the present invention, when detecting a target object from a search target image by comparing a template image including the target object and a pixel block in the search target image, determination of a similarity evaluation value used in the comparison is performed. Since the reference value is set based on the similarity evaluation value obtained by comparison with the sample image in advance, the setting of the determination reference value for determining whether the template image and the target object are similar is set appropriately. Can be done.

  Further, in the second aspect of the present invention, a polygonal effective image area is set in the template image in the first aspect of the present invention, and the determination reference value setting means and the similarity determination means include the template The similarity evaluation value is calculated based on a pixel group included in an effective image area in the image and a pixel group corresponding to the inside of the effective image area in the pixel block. . According to the second aspect of the present invention, the peripheral pixels of the effective image area included in the template image are excluded from the calculation of the similarity evaluation value in the first aspect of the present invention, so that high-precision calculation can be performed. .

  In the third aspect of the present invention, a plurality of similarity evaluation values are calculated in the determination reference value setting means and the similarity determination means in the first and second aspects of the present invention, and the determination corresponding to each similarity evaluation value is performed. When a reference value is set and the similarity determination means determines that there is no similarity with one similarity evaluation value, it is determined that there is no similarity without making a determination regarding another similarity evaluation value. Features. According to the third aspect of the present invention, when a plurality of similarity evaluation values are calculated and it is determined that there is no similarity based on one similarity evaluation value, it is determined that there is no similarity as a whole. Since the determination regarding the similarity evaluation value is not performed, the overall processing speed is increased.

  According to the present invention, in detecting a target object from a search target image by comparing a template image including the target object and a pixel block in the search target image, determination for determining whether the template image and the target object are similar to each other. It is possible to appropriately set the reference value.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(1. System configuration)
First, the configuration of the target object detection system according to the present invention will be described. FIG. 1 is a configuration diagram of a target object detection system according to the present invention. In FIG. 1, 10 is a search target image storage unit, 20 is a template preparation unit, 30 is a template storage unit, 40 is a target search unit, and 50 is a search result storage unit.

  The search target image storage unit 10 stores a search target image that is an image to be searched and a sample image that stores a sample of the target. If there are many targets to be searched on the search target image, one of them can be used as a sample. In this case, the search target image also serves as a sample image. In the present embodiment, a satellite image that is an image taken by an artificial satellite is stored as a search target image. The template preparation unit 20 interactively sets a sample image to prepare a template image, and includes a template creation unit 21, a determination reference value setting unit 22, and a template update unit 23. The template storage unit 30 stores the template image prepared by the template preparation unit 20. The target searching unit 40 searches for a target by comparing a predetermined region in the search target image with the template image, and includes an angle-independent similarity determining unit 41, an angle-dependent similarity determining unit 42, Search result output means 43 is provided. The search result storage unit 50 stores the results searched by the target object search unit 40. The target object detection system shown in FIG. 1 is actually realized by incorporating a dedicated program into a general-purpose computer. Each storage unit is realized by a storage device such as a hard disk built in or connected to the computer.

(2. Preparation of template image)
Next, the processing operation of the target object detection system shown in FIG. 1 will be described. First, the search target image storage unit 10 stores a satellite image captured by an artificial satellite as a search target image. In the present embodiment, the satellite image is in the Geo-TIFF format compliant with the Adobe TIFF standard, and the image data with geographic information (such as latitude / longitude information) added thereto is adopted.

  When the target detection system is activated, first, the template preparation unit 20 converts the search target image stored in the search target image storage unit 10 from the source image format to the work image format. This is done to increase the search efficiency. Specifically, it is converted into a work image format using a known histogram flattening technique. In the present embodiment, the work image is created in the RXY format. The template preparation unit 20 uses this search target image as a sample image.

  Subsequently, the template preparation unit 20 displays the search target image as the sample image converted into the work image format on the screen, and prompts the user to specify an image area for creating the template image. On the other hand, the user designates the contour of an image area having a polygonal shape (in this embodiment, a free rectangular shape including four vertices) in the search target image displayed on the screen. This is performed by designating four vertices of a square shape by operating the mouse. For example, when a ship is an object, the user finds a ship shown in a satellite image and designates an effective image area including the ship. Specifically, a square of a square shape is displayed as an initial state on the screen, and the user selects one of the vertices on the screen, designates the vertex by dragging the vertex to a desired position, and designates four vertices. The same operation is sequentially performed on the vertices. At this time, as shown in FIG. 2, even if the square is rotated 360 degrees outside the designated square, a circumscribed square shape (outer square shape in FIG. 2) is automatically generated at the same time so as not to protrude. Accordingly, a double square is displayed on the screen in the initial state, and when the user transforms the inner square shape into a desired quadrangle, the outer square shape changes while maintaining the square.

  When the user completes the specification of a predetermined quadrangular effective image area in the displayed search target image, the template creation means 21 defines a square defined outside the effective image area specified from within the search target image. Trimming based on shape. As a result, a square-sized template image is obtained, and at the same time, information for masking the outer portion of the rectangular area defined by the user in the template image is added. Specifically, as shown in FIG. 2, each pixel value of the template image has an 8-bit value from 0 to 255, but the inside of the square area defined by the user has a pixel value from 1 to 255. The outer side has a pixel value of 0.

  Furthermore, a polygon shape and a pattern in which only the inside thereof is rotated at a predetermined angle are also registered. In the present embodiment, the angle is rotated in units of 45 degrees, and template images according to eight different inclination angles are created. FIG. 3 shows eight template images having different angles. At this time, since the outer square shape is defined so that it does not protrude even if the polygon specified by the user is rotated 360 degrees, the polygons of the eight types of angles are represented by template images of the same size. can do. In this embodiment, patterns with different angles are registered with the same template ID as the same template image. Therefore, the template images shown in FIG. 3 all have the same template ID, and each pattern is specified by its angle. In this embodiment, eight template IDs can be registered. In this case, since eight template images are registered at eight angles, 64 patterns are registered in total.

  When the template image is registered, the template preparation unit 20 prompts the user to specify an object to be matched on the search target image. When the user designates an object to be matched, the determination reference value is calculated and the template image is updated using the pixel values near the designated position. The calculation of the determination reference value and the update of the template image will be described with reference to the flowchart of FIG.

  First, the user looks at the contents of the search target image displayed on the screen, finds a target object that he wants to include in the search target, and designates a point on the target object (S101). FIG. 5A is a diagram illustrating a state of a search target image when an object is specified. When the user designates a point on the target object to be included in the search target image in the search target image, as shown in FIG. 5A, a polygonal shape indicating a square outer frame of the template image and an effective image area is obtained. Is displayed. For example, when a ship is to be detected from a satellite image, a point on one ship in the satellite image is designated.

  Subsequently, matching processing with the template image is performed using the designated position as a base point (S102). The reason for this processing is that the point specified by the user on the screen is based on visual judgment and is ambiguous as shown in FIG. 5A. Therefore, the judgment reference value is calculated as it is based on this point. This is because there is a high possibility that an inappropriate search is performed. Therefore, in this process, the point designated by the user is corrected to an optimum position. Specifically, as shown in FIG. 5B, the base point (Xc, Yc) on the designated search target image and the center position of the template image are aligned, and from there, the template image is set to 1 in the x and y directions. Matching processing is performed while moving within a set range in pixel units. Details of this matching process will be described later. At the same position, matching processing is performed on eight types of template images with different angles, a template image having the highest similarity is selected, and the pixel position of the search target image is specified as the optimum position.

  Next, a determination reference value is set so that the object on the search target image is determined to be compatible (S103). When the setting of the determination reference value is completed, the pixel block of the search target image at the optimal position and the template image are combined, and the template image is updated with the combined image (S104).

  Next, the details of the matching process in S102 will be described with reference to the flowchart of FIG. When performing the matching process, first, each variable that becomes the similarity evaluation value is initialized. Specifically, the minimum value of the luminance histogram difference, the minimum value of the gradation fluctuation histogram, the minimum value of the distance between the centroids, the minimum value of the moment of inertia difference, the minimum value of the gray gradation pixel difference, and the normalized phase relationship Initialize the six variables with the maximum number. In the present embodiment, all of these six variables take values from 0 to 1000, with respect to the normalized correlation coefficient, the initial value is the minimum value 0, and for the other five variables, the initial value is the maximum value 1000. To do.

  When the initialization of each variable is completed, the luminance histogram is collated (S201). Specifically, first, the size of the template image is set to S × S, and the gradation value at the pixel (x, y) (0 ≦ x ≦ S−1, 0 ≦ y ≦ S−1) of the template image is set to Vt ( x, y) (0 to 255), and the image gradation value corresponding to the template position (x, y) in the region starting from the position of the search target image (Xc + dx, Yc + dy) is V (x + Xc + dx, y + Yc + dy) (1 to 255). Within the range of S × S pixels, V (x, y) / 16 of the pixel (x, y) where Vt (x, y)> 0 corresponding to the pixel in the effective image area shown in FIG. 16-gradation luminance histogram H (v) (0 ≦ v ≦ 15) of the search target image based on the value and 16-gradation luminance histogram Ht (v of the template image based on the value of Vt (x, y) / 16 ) (0 ≦ v ≦ 15) is calculated. Further, a luminance histogram difference is calculated by executing processing according to the following [Equation 1].

[Formula 1]
(Luminance histogram difference) = 1000 · _{Σv = 0,15} | H (v) −Ht (v) | / _{Σv = 0,15} (H (v) + Ht (v))

  Subsequently, the gradation fluctuation histogram is collated (S202). Specifically, as in S201, the size of the template image is S × S, and Vt (x, y)> 0 corresponding to the pixels in the effective image area shown in FIG. 2 within the range of S × S pixels. 16 pixel gradation fluctuation histogram HD (v) (0 ≦ v ≦) of the search target image based on the value of V (x, y)% 16 (the remainder divided by 16) in the pixel (x, y) 15) and a gradation fluctuation histogram HDt (v) (0 ≦ v ≦ 15) of 16 gradations of the template image based on the value of Vt (x, y)% 16. HD (v) is a gradation fluctuation histogram based on the lower 4 bits of each pixel, and HDt (v) is an upper 4 bits of each pixel. Furthermore, the gradation fluctuation histogram difference, that is, the Euclidean distance between the gradation fluctuation histograms is calculated by executing processing according to the following [Equation 2].

[Formula 2]
(Gradation fluctuation histogram difference) = 1000 · _{Σv = 0,15} | HD (v) −HDt (v) | / _{Σv = 0,15} (HD (v) + HDt (v))

  Next, the distance between the centers of gravity is calculated (S203). Specifically, first, the size of the template image is set to S × S, and the gradation value at the pixel (x, y) (0 ≦ x ≦ S−1, 0 ≦ y ≦ S−1) of the template image is set to Vt ( x, y) (0 to 255), the image gradation value corresponding to the position (x, y) of the template image in the region starting from the position (Xc + dx, Yc + dy) of the search target image is represented by V ( x + Xc + dx, y + Yc + dy) (1 to 255). Within the range of S × S pixels, using the pixel (x, y) for which Vt (x, y)> 0 corresponding to the pixel in the effective image area shown in FIG. , Yg) and the center of gravity (Xgt, Ygt) of the template image are calculated by executing processing according to the following [Equation 3].

[Formula 3]
Xg = Σy = _{0, S-1} Σx _{= 0, S-1 | Vt (x, y) ≠ 0} V (x + Xc + dx, y + Yc + dy) ・ x
/ Σ _{y = 0, S-1} Σ _{x = 0, S-1 | Vt (x, y) ≠ 0} V (x + Xc + dx, y + Yc + dy)
Yg = Σy = _{0, S-1} Σx _{= 0, S-1 | Vt (x, y) ≠ 0} V (x + Xc + dx, y + Yc + dy) ・ y
/ Σ _{y = 0, S-1} Σ _{x = 0, S-1 | Vt (x, y) ≠ 0} V (x + Xc + dx, y + Yc + dy)
_{Xgt = Σ y = 0, S} -1 Σ x = 0, S-1 | Vt (x, y) ≠ 0 Vt (x, y) · x / Σ y = 0, S-1 Σ x = 0, S _{-1 | Vt (x, y) ≠ 0} Vt (x, y)
_{Ygt = Σ y = 0, S} -1 Σ x = 0, S-1 | Vt (x, y) ≠ 0 Vt (x, y) · y / Σ y = 0, S-1 Σ x = 0, S _{-1 | Vt (x, y) ≠ 0} Vt (x, y)

  Furthermore, the process according to the following [Equation 4] is executed to calculate the distance between the centers of gravity.

[Formula 4]
(Distance between centroids) = 1000 · {(Xg−Xgt) (Xg−Xgt) + (Yg−Ygt) (Yg−ygt)} ^1/2 / S

  Next, the inertia moment is collated (S204). Specifically, first, using the centroid (Xg, Yg) of the search target image obtained in S203 and the centroid (Xgt, Ygt) of the template image, within the range of S × S pixels, the validity shown in FIG. Using the pixel (x, y) with Vt (x, y)> 0 corresponding to the pixel in the image area, the inertia moment M of the search target image and the inertia moment Mt of the template image are expressed by the following [Equation 5]. Execute the process according to

[Formula 5]
M = Σy _{= 0, S-1} Σx _{= 0, S-1 | Vt (x, y) ≠ 0} V (x + Xc + dx, y + Yc + dy) ・ [(x-Xg) (x -Xg) + (y-Yg) (y-Yg)]
/ Σ _{y = 0, S-1} Σ _{x = 0, S-1 | Vt (x, y) ≠ 0} V (x + Xc + dx, y + Yc + dy)
Mt = Σy _{= 0, S-1} Σx _{= 0, S-1 | Vt (x, y) ≠ 0} Vt (x, y) ・ [(x-Xgt) (x-Xgt) + (y-Ygt ) (y-Ygt)]
/ Σ _{y = 0, S-1} Σ _{x = 0, S-1 | Vt (x, y) ≠ 0} Vt (x, y)

  Further, a process according to the following [Equation 6] is executed to calculate the moment of inertia difference.

[Formula 6]
(Inertia moment difference) = 1000 · | M−Mt | / (M + Mt)

  Steps S201 to S204 are processes that do not depend on the angle. Therefore, it is only necessary to perform once for each template image. Since the template image at any angle may be used in the processes of S201 to S204, a template image of 0 degree is typically used in the present embodiment. Since S205 and S206 from here are processing depending on the angle, the template image having the same ID is also processed for each angle. First, a gray gradation pixel difference is calculated (S205). Specifically, first, the size of the template image is set to S × S, and the gradation value at the pixel (x, y) (0 ≦ x ≦ S−1, 0 ≦ y ≦ S−1) of the template image is set to Vt ( x, y) (0 to 255), and the image gradation value corresponding to the template position (x, y) in the region starting from the position (Xc + dx, Yc + dy) of the search target image is represented by V (x + Xc + dx, y + Yc + dy) (1 to 255). Here, the following tone conversion is performed to raise the shadow side to the highlight side and drop the halftone portion to the shadow side. When Vt (x, y) ≥128, Vt '(x, y) = Vt (x, y) -128, when Vt (x, y) <128, Vt' (x, y) = 127-Vt (x, y) When V (x, y) ≥128, V '(x, y) = V (x, y) -128, when V (x, y) <128, V' (x, y ) = 127−V (x, y) Within the range of S × S pixels, C pixels (x, x) corresponding to the pixels in the effective image area shown in FIG. Using y), processing according to the following [Equation 7] is executed to calculate a gray gradation pixel difference which is a pixel difference between the search target image and the template image.

[Formula 7]
(Gray gradation pixel difference) = 1000 · Σ _{y = 0, S-1} Σx _{= 0, S-1 | Vt (x, y) ≠ 0} | V '(x + Xc + dx, y + Yc + dy ) −Vt '(x, y) |
/ (V '(x + Xc + dx, y + Yc + dy) + Vt' (x, y))

  Next, a normalized correlation coefficient is calculated (S206). Specifically, first, the size of the template image is set to S × S, and the gradation value at the pixel (x, y) (0 ≦ x ≦ S−1, 0 ≦ y ≦ S−1) of the template image is set to Vt ( x, y) (0 to 255), and the image gradation value corresponding to the template position (x, y) in the region starting from the position (Xc + dx, Yc + dy) of the search target image is represented by V (x + Xc + dx, y + Yc + dy) (1 to 255). Within the range of S × S pixels, C pixels (x, y) satisfying Vt (x, y)> 0 corresponding to the pixels in the effective image area shown in FIG. The average gradation value Va and the average gradation value Vat of the template image are calculated by executing processing according to the following [Equation 8].

[Formula 8]
Va = Σy _{= 0, S-1} Σx _{= 0, S-1 | Vt (x, y) ≠ 0} V (x + Xc + dx, y + Yc + dy) / C
Vat = Σy _{= 0, S-1} Σx _{= 0, S-1 | Vt (x, y) ≠ 0} Vt (x, y) / C
C = Σy _{= 0, S-1} Σx _{= 0, S-1 | Vt (x, y) ≠ 0} 1

  Then, within a range of S × S pixels, a pixel (x, y) that satisfies Vt (x, y)> 0 corresponding to the pixel in the effective image area shown in FIG. Based on this, the normalized correlation coefficient is calculated by executing processing according to the following [Equation 9]. However, when the normalized correlation coefficient is a negative value (negative correlation), it is treated as 0 (no correlation).

[Formula 9]
(Normalized correlation coefficient) = 1000 · Σ _{y = 0, S-1} Σx _{= 0, S-1 | Vt (x, y) ≠ 0} (V (x + Xc + dx, y + Yc + dy) -Va) ・ (Vt (x, y) -Vat)
/ [Σ _{y = 0, S-1} Σ _{x = 0, S-1 | Vt (x, y) ≠ 0} (V (x + Xc + dx, y + Yc + dy) -Va) ²・ Σ _{y = 0, S-1} Σ _{x = 0, S-1 | Vt (x, y) ≠ 0} (Vt (x, y) -Vat) ² ] ^1/2

  Subsequently, each variable is updated (S207). Specifically, the luminance histogram difference calculated in S201 is smaller than the minimum value of the currently set luminance histogram difference, and the normalized correlation coefficient calculated in S206 is the normalized correlation coefficient currently set. If the value is larger than the maximum value, the minimum value of the brightness histogram difference, the minimum value of the gradation fluctuation histogram difference, the minimum value of the distance between the centroids, the minimum value of the moment of inertia difference, the minimum value of the gray gradation pixel difference, and the normalization phase Replace the six variables with the maximum number of relations with the values calculated this time. Then, the position, template ID, and angle values are updated with the position, template ID, and angle values at this time.

  When the update of each variable is completed, it is determined whether or not the processing of all angle patterns has been completed for one template image (S208). If the processing has not been completed for all the angle patterns, the process returns to S205 to calculate the gray gradation pixel difference depending on the angle and the normalized correlation coefficient. If the process is completed for all angle patterns, the process proceeds to S209. Therefore, in the present embodiment, the processing of S205 to S207 is repeated for eight angles for one template image.

  When processing is completed for all angles of one template image, it is determined whether or not processing has been completed for all template images (S209). If the processing has not been completed for all template images, the processing of S201 to S208 is repeated. If the processing is completed for all template images, the process proceeds to S210. Therefore, in the present embodiment, the processing of S201 to S208 is repeated for eight template images for one pixel position.

  When processing is completed for all template images, it is determined whether processing has been completed for all pixel positions (S210). All pixel positions mean the positions of pixels included in a predetermined pixel range in the x-axis and y-axis directions from the designated base point (Xc, Yc). As the predetermined pixel range, about 5% of each of the x-axis and y-axis directions of the template image is set. For example, when the template image is 100 × 100 pixels, the processing shown in FIG. 5 is repeated for a total of 25 × 5 × 5 pixel positions. If the processing has not been completed for all pixel positions, the processing of S201 to S209 is repeated. When the process is completed for all pixel positions, each variable is output and the matching process is terminated. Each variable is output for each template image.

  Next, details of setting the determination reference value in S103 will be described. The determination reference value is set by multiplying each variable output in the matching process by a predetermined value. In the present embodiment, for the normalized correlation coefficient, a value obtained by multiplying the maximum value of the output normalized correlation coefficient by 0.9 is used as a determination reference value, and for the other five variables, the output value A value obtained by multiplying 1.1 by 1.1 is used as a criterion value. Specifically, the calculation is performed by executing processing according to the following [Equation 10].

[Formula 10]
(Determination reference value of luminance histogram difference) = (Minimum value of luminance histogram difference) × 1.1
(Judgment reference value of gradation fluctuation histogram difference) = (minimum value of gradation fluctuation histogram difference) × 1.1
(Judgment reference value of distance between centroids) = (minimum value of distance between centroids) × 1.1
(Judgment reference value of moment of inertia difference) = (minimum value of moment of inertia difference) × 1.1
(Determination reference value of gray gradation pixel difference) = (Minimum value of gray gradation pixel difference) × 1.1
(Criteria value of normalized correlation coefficient) = (maximum value of normalized correlation coefficient) × 0.9

  Next, details of the template update in S104 will be described. In S104, each template image is updated. Specifically, the image of each angle of each template image is updated using the pixel block on the search target image at the optimum position of each template image. First, among the template images, the one with the optimum angle and the pixel block on the search target image are synthesized. Since the mask area is not synthesized in the template image, the composition is performed only within the polygon area in the template image.

  As a specific method of synthesis, an integer pseudo-random number is sequentially generated for each pixel in the polygonal area, and when a predetermined number is obtained, it is replaced with the value of the corresponding pixel in the pixel block on the search target image. For example, when an odd number appears, if it is set to be replaced with a corresponding pixel of a pixel block on the search target image, a pixel in the polygonal region is replaced with a pixel of the search target image with about half probability. FIG. 7 is a diagram illustrating an example of updating the template image by combining the search target image and the template image. In the example of FIG. 7, the image size is simplified to 5 × 5 pixels for convenience of explanation. FIG. 7B shows a template image, T indicates a pixel in the effective image area (polygonal area), and a blank indicates a pixel in the mask area. FIG. 7A shows a pixel block on the search target image determined to be the optimum position with respect to the template image, and W indicates a pixel on the search target image. As a result of performing the above-described combining process on these two images, as shown in FIG. 7C, about half of the pixels T in the effective image area of the template image are replaced with the pixels W of the search target image. A composite image is obtained. This composite image is registered as an updated template image. Note that the seeds are changed each time so that the random number generation method becomes a different series each time processing is performed.

  When the update process is completed for the optimum angle pattern, the other angle patterns are also updated. For other angle patterns, the difference between the angle and the optimum angle is obtained, the pixel block on the search target image is rotated by that angle, and the pixel block on the search target image is rotated for each pixel unit inside the polygonal area. Replace with the corresponding pixel. In this way, the template image is updated for all angles. All template images are updated by performing the same process on all template images prepared in advance. FIG. 8 is a diagram showing how the template image is updated. As shown in FIG. 8, images at all angles are updated using the pixel block at the optimum position on the search target image.

(3. Search process)
As described above, when the preparation of the template image is completed, the target object search unit 40 searches for the target object using the prepared template image. The target searching process will be described with reference to the flowchart of FIG. When the user instructs the target searching process, the angle-independent similarity determining unit 41 and the angle-dependent similarity determining unit 42 of the target searching unit 40 cooperate to sequentially perform the matching process (S301). Specifically, pattern matching processing is sequentially executed from the start pixel position in the search target image to find a position for determining similarity.

  FIG. 10 is a diagram showing an outline of the sequential matching process. In the sequential matching process, first, a square pixel block up to (S-1, S-1) from the start pixel position (0, 0) in the search target image is compared with the template image, A square area starting at (1, 0), a square area starting at (2, 0), and so on. Then, it moves to the Y direction scan range by one pixel, and the entire search target image is sequentially compared with the template image such as a square area starting from (0, 1), a square area starting from (1, 1), and so on. Make a comparison.

  In the matching process at each pixel position, first, angle-independent template matching for matching with one of the template images with the same ID and angle-dependent template for matching with all eight of the template images with the same ID. It is divided into matching. If it is determined that the angle-independent template matching is incompatible, the position shifts to the next pixel position without performing the angle-dependent template matching.

  Details of the sequential matching process when the pixel position is specified are shown in the flowchart of FIG. First, as in the matching process in S102, each variable that becomes a similarity evaluation value is initialized. Specifically, the minimum value of the luminance histogram difference, the minimum value of the gradation fluctuation histogram, the minimum value of the distance between the centroids, the minimum value of the moment of inertia difference, the minimum value of the gray gradation pixel difference, and the normalized phase relationship Initialize the six variables with the maximum number. All of the six similarity evaluation values used in the search process also take a value of 0 to 1000. For the normalized correlation coefficient, the initial value is set to the minimum value 0, and for the other five variables, the initial value is set to the maximum value 1000. And Subsequently, the brightness histogram is collated (S401). Specifically, as in S201 above, the process according to [Formula 1] is executed to calculate the luminance histogram difference. Then, the luminance histogram difference is compared with the determination reference value set in S103. Nonconforming means that there is no similarity. If it does not match, it is determined that the currently used template image does not match, and it is determined whether or not processing with all template images has been completed (S409). If processing with all template images has not been completed, the template image is changed to another template image (S410), and the process returns to S401.

  If the luminance histogram difference is smaller than the determination reference value in S401, the gradation fluctuation histogram is collated (S402). Specifically, similar to S202 above, the gradation fluctuation histogram difference is calculated by executing the processing according to [Formula 2]. Then, the gradation fluctuation histogram difference is compared with the determination reference value set in S103. Processing in the case of nonconformity is the same as that in the case of nonconformity in S401.

  If the gradation fluctuation histogram difference is smaller than the determination reference value in S402, the distance between the centers of gravity is calculated (S403). Specifically, similarly to S203, the processing according to [Formula 3] and [Formula 4] is executed to calculate the distance between the centers of gravity. Then, the distance between the centers of gravity and the determination reference value set in S103 are compared. Processing in the case of nonconformity is the same as that in the case of nonconformity in S401.

  If the distance between the centers of gravity is smaller than the determination reference value in S403, the inertia moment is collated (S404). Specifically, similar to S204, the processing according to [Formula 5] and [Formula 6] is executed to calculate the moment of inertia difference. Then, this moment of inertia difference is compared with the determination reference value set in S103. Processing in the case of nonconformity is the same as that in the case of nonconformity in S401. Since the processing of S401 to S404 is processing that does not depend on the angle, if the template image is not matched even once after matching with the template image, the process proceeds to processing with another template image.

  If the moment of inertia difference is smaller than the determination reference value in S404, the gray gradation pixel difference is calculated (S405). Specifically, similarly to S205 described above, the processing according to [Formula 7] is executed to calculate the gray gradation pixel difference. In the gray tone pixel difference calculation process, unlike the processes in S401 to S404, the same template image is processed for all angles. The gray gradation pixel difference is updated to the minimum, and the angle at that time is recorded. When the calculation of the gray gradation pixel difference is completed for all angles (S406), the minimum gray gradation pixel difference is compared with the determination reference value set in S103 to obtain the gray gradation pixel difference. If is more than the criterion value, it is not conforming. Processing in the case of nonconformity is the same as that in the case of nonconformity in S401.

  When the gray gradation pixel difference is smaller than the determination reference value, a normalized correlation coefficient is calculated (S407). Specifically, the normalized correlation coefficient is calculated by executing processing according to [Equation 8] and [Equation 9] as in S206. In the normalized correlation coefficient calculation process, similar to the gray gradation pixel difference calculation process in S405, the same template image is processed for all angles. The normalized correlation coefficient is updated to the maximum value, and the angle at that time is recorded. Then, the processing when the normalized correlation coefficient has been calculated for all the angles is the same as that in the case of nonconformity in S401.

  When the processing is completed for all template images, the normalized correlation coefficient is collated (S411). Specifically, the maximum normalized correlation coefficient is compared with the determination reference value set in S103, and if the normalized correlation coefficient is larger than the determination reference value, the result is relevant. If the normalized correlation coefficient is matched, it is determined whether there is a template that matches all items, that is, all six similarity evaluation values (S412). If there is a template in which all items are compatible, it is determined whether or not the angle of the template in which all items are compatible matches the angle (S413). That is, the template and the angle that give the minimum value in the calculation processing of the gray gradation pixel difference in S405 and S406 and the template and the angle that give the maximum value in the calculation processing of the normalized correlation coefficient in S407 and S408 respectively match. If it is determined in S401, S402, S403, S404, S406, and S411 that the matching templates are all matched, matching ends, and each variable, that is, template ID, pixel position (Xc, Yc), angle, and six similarities. The sex evaluation value is output. In any case, in the case of non-conformity, the non-conformance ends, and matching processing is sequentially performed on the next pixel position.

  In the sequential matching processing shown in FIG. 11, the luminance histogram collation in S401, the gradation variation histogram collation in S402, the distance between the centers of gravity in S403, the moment of inertia collation in S404, and the gray gradation pixels in S405 and S406. The six suitability determination processes of difference calculation and normalization correlation coefficient calculation of S407 and S408 are performed, and these suitability determination processes are set to be performed in order from the smallest processing load. . That is, in the present embodiment, the comparison of luminance histograms has the smallest processing load, and the calculation of the normalized correlation coefficient has the largest processing load. This is because, when it is determined that the non-conformity is determined in one process, it is more efficient to perform the processing first from the one with the smallest processing load if it is determined as non-conforming as a whole. In particular, since the calculation of the gray gradation pixel difference and the calculation of the normalized correlation coefficient are performed for each angle, the comparison of the brightness histogram, the comparison of the gradation fluctuation histogram, and the distance between the centroids performed only for one angle are performed. Compared to calculation and moment of inertia verification, the processing load is significantly increased.

  When the sequential matching process in S301 is completed, the target object search unit 40 next performs a local matching process (S302). The sequential matching process in S301 searches for a rough position of the target in the entire search target image, whereas the local matching process in S302 searches for a detailed position of the target. Specifically, template matching is performed by moving a predetermined minute width (ΔX, ΔY) in the pixel scan progression direction (rightward, downward) starting from the position (Xc, Yc) determined to be optimal by sequential matching processing. And the six similarity evaluation values are updated to the minimum value and the maximum value, and the template ID, pixel position (Xm, Ym), and angle Am at that time are output. The details of the matching process are the same as the matching process in S102 of FIG. 4, specifically the process shown in FIG. 6, unlike the sequential matching process shown in FIG. The sequential matching process only has to determine whether or not the matching is performed at a certain position (Xc, Yc), but the local matching process searches for a matching position under the best condition within a minute range in which the matching is determined. This is necessary.

  When the local matching process is completed, the search result output unit 43 of the target object search unit 40 next outputs the search result (S303). Specifically, the template ID, position (Xm, Ym), angle Am, and six similarity evaluation values are output as a list.

  Subsequently, the target searching unit 40 deletes the detection area (S304). Specifically, in the search target image, a portion corresponding to the inside of the effective image area shown in FIG. 2 of the template image at the optimum position is deleted. That is, all the pixel values on the search target image located inside the effective image area are set to zero. FIG. 12 is a diagram showing the optimum position of the search result on the search target image and the erased area. As a result of the search, when the position shown in FIG. 12A is the optimum position of the effective image area of the template image with respect to the target on the search target image, as shown in FIG. A portion corresponding to the effective image area on the search target image is deleted. However, the original search target image before erasure is backed up and saved in order to meet the request for graphically displaying the search position on the search target image after the end of the search process.

  The processing of S301 to S304 is performed for all pixels on the search target image. At that time, in S304, the inside of the conforming region is erased, so that matching is not performed even if the matching processing is sequentially performed on the neighboring pixels, and one target is found redundantly. Can be prevented.

  The result output in S303 is output as a search result. At this stage, all searched parts are erased. Therefore, when the search position is displayed graphically, it is necessary to reload the original search target image stored in the backup. Even if the process of S304 is performed, if there are overlapping results, an operation is performed in which one is manually deleted and the other is deleted.

It is a block diagram of the target object detection system which concerns on this invention. It is a figure which shows the relationship between a template image area | region and a polygonal shape. It is a figure which shows eight template images from which an angle differs. It is a flowchart which shows the process outline | summary of the calculation of a determination reference value, and the update of a template image. It is a figure which shows the mode of the process on a search object image. It is a flowchart which shows the detail of a matching process. It is a figure which shows the example which updates a template image by the synthesis | combination of a search object image and a template image. It is a figure which shows the mode of the update of a template image. It is a flowchart which shows the outline | summary of a search process. It is a figure which shows the outline of a sequential matching process. It is a flowchart which shows the detail of a sequential matching process. It is a figure which shows the optimal position of the result of the search on a search object image, and the erased area.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Search object image storage part 20 ... Template preparation part 21 ... Template preparation means 22 ... Determination reference value setting means 23 ... Template update means 30 ... Template storage part 40 ... Target searching unit 41... Angle-independent similarity determining means 42... Angle-dependent similarity determining means 43... Search result output means 50.

Claims (12)

A system for detecting a target from a search target image, which is a search target, using a template image including the target,
Template storage means for storing the template image;
A pixel block having the same size as each template image is extracted from the sample image at each pixel position within a predetermined range from a pixel position designated on the sample image including the target sample, and the template image and the pixel block are extracted. Determination reference value setting means for calculating a similarity evaluation value of the image and setting a value obtained by calculating a predetermined value to the similarity evaluation value as a determination reference value;
A pixel block having the same size as each template image is extracted from the search target image while sequentially moving the extraction position from the first pixel position to the last pixel position with respect to the search target image, and the template image and the pixel block are extracted. Similarity determination means for calculating the similarity evaluation value of the image and comparing the similarity evaluation value with the set criterion value;
A search for outputting the template image ID, the pixel position for specifying the position of the pixel block on the search target image, and the rotation angle of the template image when the similarity determination unit determines that there is similarity. A result output means;
A target detection system comprising: In claim 1,
A polygonal effective image area is set in the template image, and the determination reference value setting means and the similarity determination means include a pixel group included in the effective image area in the template image, and the pixel block. A similarity detection value is calculated based on a pixel group corresponding to the inside of the effective image area in the target object detection system. In claim 1 or claim 2,
In the determination reference value setting means and the similarity determination means, a plurality of similarity evaluation values are calculated, and a determination reference value corresponding to each similarity evaluation value is set. In the similarity determination means, one similarity A target object detection system characterized in that, when it is determined that there is no similarity based on an evaluation value, it is determined that there is no similarity without making a determination regarding another similarity evaluation value. In claim 1 or claim 2,
The similarity evaluation value in the determination reference value setting means and the similarity determination means is a predetermined collation between the pixels included in the template image and the corresponding pixels in the pixel block without rotating the angle of the template image. The similarity evaluation value calculated without depending on the angle by collating with the method, and the pixels included in the template image and the corresponding pixels in the pixel block while rotating the template image at a predetermined angle There are two types of similarity evaluation values that are calculated by depending on the angle by collating with a predetermined collation method. In claim 3 or claim 4,
The determination reference value setting means and the similarity determination means calculate a gradation fluctuation histogram for each of the pixels included in the template image and the corresponding pixels in the pixel block, and Euclidean between both gradation fluctuation histograms. A target detection system, wherein a distance is calculated as the similarity evaluation value. In claim 5,
The Euclidean distance between the gradation fluctuation histograms is a Euclidean distance between a gradation fluctuation histogram based on upper bits extracted from each pixel value and a gradation fluctuation histogram based on lower bits. In any one of Claims 3-6,
The determination reference value setting unit and the similarity determination unit calculate a centroid position using the gradation values of the pixels included in the template image and the corresponding pixels in the pixel block as weights, and between the centroids. A target object detection system using a difference value between a distance and a moment of inertia of both as a similarity evaluation value. In any one of Claims 3-7,
The determination reference value setting unit and the similarity determination unit use a difference value between gradation values of a pixel included in the template image and a corresponding pixel in the pixel block as the similarity evaluation value. A target detection system. In any one of Claims 3-8,
The determination reference value setting unit and the similarity determination unit use, as the similarity evaluation value, a normalized correlation coefficient based on a gradation value of a pixel included in the template image and a corresponding pixel in the pixel block. A target detection system characterized by being a thing. In any one of Claims 1-9,
A template image having a square shape defined so as to be included even if the effective image area of the template image is defined by cutting out a predetermined polygonal area specified from the sample image and the effective image area is rotated to an arbitrary angle. And a template creation means for storing the template image in the template storage means together with information on the defined effective image area. In claim 10,
When the similarity determination unit determines that there is similarity, the search result output unit includes an ID of a template image at that time, a pixel position that specifies the position of a pixel block on the search target image, and the template image A target object detection system, wherein a contour line of an effective image area of a template image corresponding to a rotation angle of the template image is superimposed on the search target image and output.   The program for functioning a computer as the target object detection system in any one of Claims 1-11.

Images