JP6582618B2 - Template matching program, template matching method, and template matching apparatus - Google Patents

Description

  The present invention relates to a template matching program and the like.

  Conventionally, a technique for photographing an object and detecting the object from the photographed image without human intervention has been widely used. For example, detection of a car number from an image obtained by photographing a car traveling on a road can be mentioned. By using the detection result, it is possible to automatically recognize the vehicle number in combination with the vehicle number display to the monitoring staff and the character recognition means. Moreover, the intruder detection to a building can also be performed by detecting a person. In these applications, since the detection process is performed in real time, the high speed of the process is important.

  There is a template matching as a basic method for detecting an object. In template matching, a template representing an object is prepared in advance, and the template is scanned on the photographed image to determine a portion having high similarity to the template as the object. In order to reduce the influence of external light or the like, matching is often performed by extracting features from the image, not the image itself. For feature extraction, for example, a filter such as edge extraction is used. In addition, since the object is captured in various sizes in the image, for example, the matching is repeatedly performed by changing the scale of the template or the image in various ways.

JP 2009-199575 A

  However, the above-described conventional technique has a problem that the amount of processing increases because matching is performed by changing the scale of the template or image in various ways.

  In one aspect, an object of the present invention is to provide a template matching program, a template matching method, and a template matching apparatus that can perform matching without changing the scale of a template or an image in various ways.

  In the first plan, the template matching program causes the computer to execute the following processing. The computer acquires captured images obtained by capturing a plurality of objects having the same shape. The computer receives the positions of the reference points for a plurality of objects included in the captured image, and calculates the scale of each pixel of the captured image based on the interval between the reference points of the objects. The computer converts the photographed image into a corrected image in which the scale of each pixel is constant based on the scale of each pixel of the photographed image. The computer detects an object by executing template matching on the corrected image.

  Matching can be performed without changing the scale of the template or image.

FIG. 1 is a diagram illustrating the configuration of the system according to the first embodiment. FIG. 2 is a functional block diagram of the configuration of the image processing apparatus according to the first embodiment. FIG. 3 is a diagram (1) illustrating an example of a captured image. FIG. 4 is a diagram (2) illustrating an example of a captured image. FIG. 5 is a diagram for explaining the positions of the reference points according to the first embodiment. FIG. 6 is a diagram illustrating an example of the data structure of the coordinate value table according to the first embodiment. FIG. 7 is a diagram illustrating an example of a scale table according to the first embodiment. FIG. 8 is a diagram illustrating a calculation example of the scale of each pixel. FIG. 9 is a diagram (1) for explaining the data addition processing according to the first embodiment. FIG. 10 is a diagram (2) for explaining the data addition processing according to the first embodiment. FIG. 11 is a diagram (3) for explaining the data addition processing according to the first embodiment. FIG. 12 is a diagram illustrating an example of the data structure of the correction parameter according to the first embodiment. FIG. 13 is a diagram illustrating an example of a corrected image. FIG. 14 is a diagram illustrating an example of the detection result. FIG. 15 is a flowchart showing a processing procedure of the image processing apparatus at the time of calibration. FIG. 16 is a flowchart illustrating a processing procedure of the image processing apparatus at the time of detection. FIG. 17 is a diagram for explaining the effect of the image processing apparatus according to the first embodiment. FIG. 18 is a functional block diagram of the configuration of the image processing apparatus according to the second embodiment. FIG. 19 is a diagram for explaining the reference point positions according to the second embodiment. FIG. 20 is a diagram illustrating an example of the data structure of the coordinate value table according to the second embodiment. FIG. 21 is a diagram illustrating a calculation example of the linear conversion parameter of each pixel. FIG. 22 is a diagram for explaining the data addition processing according to the second embodiment. FIG. 23 is a functional block diagram of the configuration of the image processing apparatus according to the third embodiment. FIG. 24 is a functional block diagram of the configuration of the image processing apparatus according to the fourth embodiment. FIG. 25 is a diagram illustrating an example of the data structure of the correction parameter according to the fourth embodiment. FIG. 26 is a diagram illustrating an example of a computer that executes an image processing program.

  Hereinafter, embodiments of a template matching program, a template matching method, and a template matching apparatus disclosed in the present application will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

  FIG. 1 is a diagram illustrating the configuration of the system according to the first embodiment. As shown in FIG. 1, this system includes a camera 50 and an image processing apparatus 100. The image processing apparatus 100 is an example of a template matching apparatus.

  The camera 50 is, for example, a device that is installed on the ceiling of a building and shoots an image in the shooting range with a wide-angle lens. For example, the camera 50 corresponds to a security camera, but is not limited thereto. The camera 50 transmits information on the captured image to the image processing apparatus 100. In the following description, information on an image photographed by the camera 50 is appropriately referred to as a photographed image.

  The image processing apparatus 100 is an apparatus that detects an object from a captured image by executing template matching. For example, the image processing apparatus 100 may output an alarm or the like when an object is detected.

  Here, since the captured image captured by the camera 50 is captured with a wide-angle lens, even if the captured object is the same object, for example, the object on the lower side of the captured image is larger than the upper side of the captured image. There is. For this reason, the image processing apparatus 100 examines the scale when the object appears in each pixel of the photographed image in advance, and enlarges the photographed image so as to cancel the difference in the size of the object at each position. Perform template matching after correction. For example, the image processing apparatus 100 reads a scale to be corrected at each pixel of the image, generates a corrected image by correcting the captured image so that the read scale is constant, and extracts feature points from the corrected image. Extract and execute template matching.

  For this system, calibration processing and detection processing are executed. First, processing during calibration will be described. First, the worker places a large number of calibration objects at various locations in the building. A camera 50 installed on the ceiling of the building takes an image of the calibration object.

  The image processing apparatus 100 displays a captured image acquired from the camera 50 on a display (not shown). Such a photographed image includes a plurality of calibration objects. The worker instructs two predetermined reference points on the displayed calibration object by operating a mouse (not shown). The image processing apparatus 100 reads the reference point designated by the worker. The image processing apparatus 100 calculates a scale to be corrected at each pixel of the captured image based on the instructed reference point. The image processing apparatus 100 determines a correction parameter for correcting the entire captured image, and stores the correction parameter.

  Next, processing at the time of detection will be described. The camera 50 installed on the building ceiling, for example, takes an image of a building including a person. The image processing apparatus 100 generates a corrected image by correcting the entire captured image based on the correction parameter stored in the calibration process. The image processing apparatus 100 extracts features from the corrected image, and performs template matching using a template held inside to detect a person's position. The image processing apparatus 100 outputs a detection result.

  Next, an example of the configuration of the image processing apparatus 100 will be described. FIG. 2 is a functional block diagram of the configuration of the image processing apparatus according to the first embodiment. As shown in FIG. 2, the image processing apparatus 100 is connected to a camera 50. The image processing apparatus 100 includes an acquisition unit 110, a calibration object display unit 120, a calibration information input unit 130, a local conversion calculation unit 140, a global conversion determination unit 150, and a correction parameter storage unit 160. The image processing apparatus 100 includes a corrected image generation unit 170, a template matching execution unit 180, and a detection result output unit 190.

  The camera 50 corresponds to the camera 50 shown in FIG. For example, the camera 50 is preferably a CMOS (Complementary MOS) camera or a CCD (Charge-Coupled Device) camera. The photographing result of the camera 50 is expressed as a photographed image.

  A captured image is a two-dimensional array composed of a large number of pixels, and each pixel has a value (pixel value) corresponding to the intensity of light. In the case of a color image, each pixel has three values corresponding to R, G, and B, for example. Each pixel is specified by its coordinate value. For example, the coordinate value is defined such that the upper left pixel is (0, 0), the first component increases toward the right side of the image, and the second component increases toward the bottom of the image. The coordinate value of the pixel takes a discrete value and is represented by, for example, a non-negative integer value.

  3 and 4 are diagrams illustrating an example of a captured image. A captured image 10 shown in FIG. 3 is an image captured by the camera 50 in the calibration process. For example, as illustrated in FIG. 3, the captured image 10 includes a plurality of calibration objects 10 a to 10 t. Although the calibration objects 10a to 10t have the same shape but are photographed with a wide-angle lens, the lower calibration object is larger than the upper calibration object.

  A captured image 20 illustrated in FIG. 4 is an image captured by the camera 50 in the detection time process. For example, as illustrated in FIG. 4, the captured image 20 includes a plurality of people 20 a to 20 d. For example, the size of the captured images 10 and 20 shown in FIGS. 3 and 4 is 640 × 480.

  Returning to the description of FIG. The acquisition unit 110 is a processing unit that acquires a captured image from the camera 50. The acquisition unit 110 outputs the captured image captured by the calibration process to the calibration object display unit 120. The acquisition unit 110 outputs the captured image captured by the detection process to the corrected image generation unit 170.

  The calibration object display unit 120 is a processing unit that displays a captured image captured by the process at the time of calibration on a display or the like. For example, the photographed image photographed in the calibration process is the photographed image 10 shown in FIG. The calibration object display unit 120 may execute processing such as changing the brightness of the captured image so that the calibration objects 10a to 10t can be easily seen.

  The worker refers to the captured image 10 displayed on the display and operates an input device such as a mouse connected to the image processing apparatus 100 to designate two reference points determined for each calibration object. FIG. 5 is a diagram for explaining the positions of the reference points according to the first embodiment. FIG. 5 shows a calibration object 10a as an example of a calibration object. The reference points are, for example, the upper left 1a of the calibration object 10a and the upper right 1b of the calibration object 10a. In the following description, the upper left reference point of the calibration object will be referred to as the first reference point, and the upper right reference point will be referred to as the second reference point as appropriate.

  The calibration information input unit 130 is a processing unit that reads the coordinate values of the first reference point and the second reference point designated by the worker for each calibration object, and generates a coordinate value table. FIG. 6 is a diagram illustrating an example of the data structure of the coordinate value table according to the first embodiment. As shown in FIG. 6, the coordinate value table 130a associates the calibration object number, the coordinates of the first reference point, and the coordinates of the second reference point. The calibration object number is a number that uniquely identifies the calibration object. The calibration information input unit 130 outputs the coordinate value table 130 a to the local conversion calculation unit 140.

  The local conversion calculation unit 140 is a processing unit that calculates a local scale on the captured image based on the coordinate value table 130a. Here, the local scale is the scale of the position where the calibration object exists on the captured image. For example, the local conversion calculation unit 140 executes the following process.

  The local conversion calculation unit 140 performs initialization by setting the target reference point as the head of the coordinate value table 130a. The local conversion calculation unit 140 calculates a scale at the position of the first reference point by dividing a predetermined value by the distance between the first reference point and the second reference point. The local conversion calculation unit 140 moves the reference point of interest down one position in the coordinate value table 130a, and repeatedly executes the process of calculating the scale. The local conversion calculation unit 140 ends the process when the scale with respect to the lowest reference point of the coordinate value table 130a is calculated.

  The local conversion calculation unit 140 registers the scale calculation result in the scale table. FIG. 7 is a diagram illustrating an example of a scale table according to the first embodiment. As shown in FIG. 7, the scale table 140a associates the coordinates of the point with the scale. The coordinates of the point correspond to the coordinates of the first reference point. The local conversion calculation unit 140 outputs the scale table 140a to the global conversion determination unit 150.

  Here, the local transformation calculation unit 140 uses, for example, the Euclidean distance as the point distance. The Euclidean distance is defined as the square root of the sum of the square of the difference between the first components and the square of the difference between the second components. Further, as the predetermined value, for example, a distance between reference points when the calibration object is placed at a standard position and orientation is used.

  The global conversion determination unit 150 is a processing unit that generates correction parameters for converting a captured image into a correction image based on the scale table 140a. The global conversion determination unit 150 stores the parameters in the correction parameter storage unit 160.

  For example, the global conversion determination unit 150 generates a correction parameter by sequentially executing a data interpolation process, a data addition process, and an inverse conversion calculation process.

Data interpolation processing executed by the global conversion determination unit 150 will be described. The global conversion determination unit 150 calculates a scale corresponding to all the pixels of the captured image by interpolating an unknown scale using a local scale registered in the scale table 140a. For example, the global conversion determination unit 150 calculates the scale s (u, v) at the pixel (u, v) of the captured image based on the equation (1). In the formula (1), the first reference point designated by the worker is ( _{pi, 0} , qi _{, 0} ). Also, let s _i be the scale corresponding to the first reference point ( _{pi, 0} , qi _{, 0} ). i = 1,..., n is the number of the calibration object. n represents the total number of calibration objects.

  In equation (1), dist represents the distance between two points. w (u, v) is a normalized weight and is defined by equation (2).

  The global conversion determination unit 150 calculates the scale of each pixel (u, v) of the captured image using Expressions (1) and (2). FIG. 8 is a diagram illustrating a calculation example of the scale of each pixel. For example, FIG. 8 shows the pixels 30a to 30f. Let the pixel at the origin be 30a. The scale of the pixel 30a is 0.8, the scale of the pixel 30b is 0.7, and the scale of the pixel 30c is 0.6. The scale of the pixel 30d is 0.8, the scale of the pixel 30e is 0.7, and the scale of the pixel 30f is 0.6.

  Next, the data addition process executed by the global conversion determination unit 150 will be described. 9, FIG. 10 and FIG. 11 are diagrams for explaining the data addition processing according to the first embodiment. The global conversion determination unit 150 determines the origin of the captured image and the origin of the corrected image, and associates the origin of the captured image with the origin of the corrected image. For example, as shown in FIG. 9, when the origin of the captured image is the pixel 30a on the captured image and the origin of the corrected image is the pixel 40a on the corrected image, the global conversion determination unit 150 sets the pixel 30a and the pixel 40a to each other. Make it correspond.

  Subsequently, the global transformation determination unit 150 associates adjacent points with only the scale at the origin. For example, as illustrated in FIG. 10, a pixel adjacent to one pixel away from the pixel 30 a of the captured image is a pixel 30 b, and the scale of the pixel 30 a is “0.8”. Further, as described in FIG. 9, the pixel 30a is assumed to be associated with the pixel 40a. In this case, the global conversion determination unit 150 associates the point corresponding to the pixel 30b with the point 40b that is “0.8” away from the pixel 40a. That is, the pixel 30b at the coordinate (1,0) on the captured image corresponds to the point 40b at the coordinate (0.8, 0) on the corrected image.

  In addition, the global conversion determination unit 150 repeatedly performs the above-described process of adding the scales corresponding to the pixels to obtain the correspondence of the points (pixels) in order. Thereby, for each pixel of the captured image, the coordinate value of the point of the corrected image corresponding to the pixel is determined. However, the coordinate value of the point of the corrected image is not necessarily an integer.

  FIG. 11 shows the relationship between the pixel on the captured image and the coordinate value of the corrected image associated with this pixel. For example, the coordinate value of the point of the corrected image corresponding to the pixel 30a of the captured image is (0, 0). The coordinate value of the point of the corrected image corresponding to the pixel 30b of the photographed image is (0.8, 0). The coordinate value of the point of the corrected image corresponding to the pixel 30c of the photographed image is (1.5, 0). The coordinate value of the point of the corrected image corresponding to the pixel 30d of the captured image is (0, 0.8). The coordinate value of the point of the corrected image corresponding to the pixel 30e of the captured image is (0.8, 0.7). The coordinate value of the point of the corrected image corresponding to the pixel 30f of the captured image is (1.5, 0.6).

  For example, the coordinate value of the corrected image corresponding to the pixel 30c is a value “1.5” obtained by adding the scale “0.8” of the pixel 30a and the scale “0.7” of the pixel 30b with respect to the x coordinate. Further, the coordinate value of the corrected image corresponding to the pixel 30d is a value “0.8” obtained by adding the scale “0.8” of the pixel 30a to the y coordinate.

  Next, the inverse conversion calculation process executed by the global conversion determination unit 150 will be described. In the above data addition process, the coordinate value of the point of the corrected image corresponding to the pixel of the captured image is obtained. On the other hand, in the inverse transformation calculation process, the global transformation determination unit 150 uses, for each pixel (x, y) of the corrected image, the coordinate value (u, v) of the point of the captured image corresponding to that pixel. calculate. This is because this correspondence is used when generating a corrected image.

The global conversion determination unit 150 obtains the coordinate values (u (x, y), v (x, y)) of the captured image corresponding to the pixel (x, y) of the corrected image using the equations (3) and (3). Calculate based on (4). In the expressions (3) and (4), (u _{i, j} , v _{i, j} ) represents each pixel of the captured image. x (u _{i, j} , v _{i, j} ), y (u _{i, j} , v _{i, j} ) is the point of the corrected image corresponding to the pixel (u _{i, j} , v _{i, j} ) of the captured image. Represents a coordinate value. w (x, y) represents weight normalization and is defined by equation (5).

  The global conversion determination unit 150 executes the calculations of Expressions (3) to (4) for all the pixels (x, y) of the corrected image, and generates the execution results as correction parameters.

FIG. 12 is a diagram illustrating an example of the data structure of the correction parameter. As shown in FIG. 12, the correction parameter 160a has the coordinate value of the pixel of the corrected image and the coordinate value of the point of the captured image. For example, it is indicated that the coordinate value of the point of the photographed image corresponding to the coordinate value (0, 0) of the pixel of the corrected image is (u _0,0 , v _0,0 ).

  Returning to the description of FIG. The correction parameter storage unit 160 is a storage unit that stores correction parameters.

  The corrected image generation unit 170 is a processing unit that generates a corrected image based on the captured image and the correction parameter 160a. The corrected image generation unit 170 generates a corrected image by moving the point (pixel) of the coordinate value of the captured image to the coordinate value of the pixel of the corrected image indicated by the correction parameter 160a. For example, the corrected image generation unit 170 executes the following process.

  The corrected image generation unit 170 performs initialization by setting the target pixel at the upper left of the corrected image and preparing a two-dimensional array for generating the corrected image. The corrected image generation unit 170 reads the correction parameter and obtains the coordinate value of the point of the captured image corresponding to the target pixel from the correction parameter. The corrected image generation unit 170 writes the pixel value of the pixel closest to the obtained coordinate value into the two-dimensional array as the pixel value of the corrected image. The corrected image generation unit 170 moves the target pixel. The corrected image generation unit 170 moves the image of interest to the right by one, moves it to the left end one step lower in the case of the right end, and ends it in the case of the lower right. The corrected image generation unit 170 reads the correction parameter again and repeats the above processing.

  When calculating the pixel value, the corrected image generation unit 170 uses the pixel value of the nearest pixel as it is in the above example, but uses the average value obtained by weighting the pixel values of a plurality of pixels by the distance. May be. For example, when the corrected image generation unit 170 generates a corrected image based on the captured image 20 illustrated in FIG. 4, a corrected image 25 illustrated in FIG. 13 is generated. FIG. 13 is a diagram illustrating an example of a corrected image. The corrected image generation unit 170 outputs the corrected image to the template matching execution unit 180.

  Note that this application mainly assumes shooting with a fixed camera in which the way an object is captured, but even with a camera having a zoom function, the correction parameters can be switched according to the zoom magnification. The present application can be applied.

  The template matching execution unit 180 is a processing unit that extracts features from the corrected image and scans the template to detect an object. The template matching execution unit 180 outputs the matching result to the detection result output unit 190. For example, the template matching execution unit 180 uses a filter when extracting features from the corrected image.

  The reason why a filter is used for feature extraction in the present application is as follows. The filter extracts features by the same calculation method for every pixel. Therefore, the same feature is extracted regardless of whether it is applied to the entire corrected image or a partial image. On the other hand, in general feature extraction that is not a filter, different calculations are performed depending on the pixel position. For this reason, when a feature is designed so as to extract a feature by cutting out a partial image that is a candidate for an object, feature extraction cannot be performed collectively for the entire corrected image.

  The detection result output unit 190 is a processing unit that outputs the number and positions of people based on the matching result. The detection result output unit 190 may use, for example, the center coordinates of the person's template on the corrected image as the person's position. FIG. 14 is a diagram illustrating an example of the detection result. For example, the detection result includes a serial number and a detected coordinate value. The serial number is a number for identifying the detected person. The detected coordinate value corresponds to the center coordinate of the human template on the corrected image.

  Next, an example of a processing procedure of the image processing apparatus 100 according to the first embodiment will be described. FIG. 15 is a flowchart showing a processing procedure of the image processing apparatus at the time of calibration. As illustrated in FIG. 15, the acquisition unit 110 of the image processing apparatus 100 acquires a captured image obtained by capturing a calibration object (step S101), and the calibration object display unit 120 of the image processing apparatus 100 displays the captured image ( Step S102).

  The calibration information input unit 130 of the image processing apparatus 100 reads the reference point (step S103). The local conversion calculation unit 140 and the global conversion determination unit 150 of the image processing apparatus 100 calculate a scale to be corrected (step S104), and the global conversion determination unit 150 determines a correction parameter 160a (step S105). The global conversion determination unit 150 stores the correction parameter 160a in the correction parameter storage unit 160 (step S106).

  FIG. 16 is a flowchart illustrating a processing procedure of the image processing apparatus at the time of detection. As illustrated in FIG. 16, the acquisition unit 110 of the image processing apparatus 100 acquires a captured image (step S201). The corrected image generation unit 170 of the image processing apparatus 100 generates a corrected image by correcting the captured image based on the correction parameter 160a (step S202).

  The template matching execution unit 180 of the image processing apparatus 100 executes template matching on the corrected image (step S203). The detection result output unit 190 of the image processing apparatus 100 outputs the result of template matching (step S204).

  Next, effects of the image processing apparatus 100 according to the present embodiment will be described. The image processing apparatus 100 receives the positions of reference points for a plurality of constituent objects included in the photographed image, and calculates the scale at each pixel of the photographed image based on the intervals between the reference points of the calibration objects. The image processing apparatus 100 generates a corrected image by correcting the captured image so that the scale of each pixel becomes constant, extracts feature points from the corrected image, and executes template matching. By executing such processing, the size of the same object becomes uniform regardless of position, so that it is not necessary to perform matching by changing the scale of the template or image in various ways, and the processing amount can be reduced. Can do.

  Here, as a conventional technique having versatility, it is described that the processing of the image processing apparatus 100 according to the first embodiment is faster than the case where the feature extraction is performed many times by changing the scale of the captured image. To do. FIG. 17 is a diagram for explaining the effect of the image processing apparatus according to the first embodiment.

  In the prior art, since a feature is extracted many times at different scales for one pixel, the processing amount increases. In the example shown in FIG. 17, the following processing is performed in order to perform processing at the same scale as the template. In the conventional technique, the partial image 5a of the photographed image 5 is enlarged 1.1 times to generate a corrected partial image 6a, and a feature extraction image 7a in which features are extracted from the corrected partial image 6a is generated. Further, in the related art, the partial image 5b of the photographed image 5 is enlarged 1.3 times to generate a corrected partial image 6b, and a feature extraction image 7b in which features are extracted from the corrected partial image 6b is generated. Then, feature extraction is performed twice for a certain pixel 8.

The required processing amount in the prior art is estimated as processing amount of feature extraction per pixel * number of pixels of the corrected partial image * number of repetitions. For example, the process of feature extraction per pixel is 100, the correction partial images the number of pixels is 100 * 100, if the repetition number is 1000 ^2, throughput ^is 100 * 100 ^{2 *} 1000 2 = ^{10 12,} and the large Value. This is insufficient for real-time processing because even a high-speed computer capable of 1 picosecond unit processing takes 1 second for all processing.

On the other hand, in the image processing apparatus 100 according to the present embodiment, the necessary processing amount is estimated as the feature extraction processing amount per pixel * the number of pixels of the corrected image. For the same numerical example as above, the amount of template matching processing is estimated to be 2 * 10 ¹⁰ for the same numerical example. For this reason, a high-speed computer capable of executing unit processing in 1 picosecond requires 20 ms as a whole, and enables real-time processing.

  Next, an image processing apparatus according to the second embodiment will be described. The system configuration is the same as that of the system shown in the first embodiment.

  FIG. 18 is a functional block diagram of the configuration of the image processing apparatus according to the second embodiment. As shown in FIG. 18, the image processing apparatus 200 is connected to the camera 50. The image processing apparatus 200 includes an acquisition unit 210, a calibration object display unit 220, a calibration information input unit 230, a local conversion calculation unit 240, a global conversion determination unit 250, and a correction parameter storage unit 260. The image processing apparatus 200 includes a corrected image generation unit 270, a template matching execution unit 280, and a detection result output unit 290.

  The description regarding the camera 50, the acquisition unit 210, and the calibration object display unit 220 is the same as the description regarding the camera 50, the acquisition unit 110, and the calibration object display unit 120 of FIG. The description regarding the corrected image generation unit 270, the template matching execution unit 280, and the detection result output unit 290 is the same as the description regarding the correction image generation unit 170, the template matching execution unit 180, and the detection result output unit 190 of FIG.

  The calibration information input unit 230 reads the coordinate values of the first reference point, the second reference point, and the third reference point designated by the worker for each calibration object, and generates a coordinate value table. Part. FIG. 19 is a diagram for explaining the reference point positions according to the second embodiment. In FIG. 19, as an example of the calibration object, the reference points are, for example, the upper left 1a of the calibration object 10a, the upper right 1b of the calibration object 10a, and the lower left 1c of the calibration object 10a. In the following description, the reference point at the upper left of the calibration object is referred to as the first reference point, the reference point at the upper right is referred to as the second reference point, and the reference point at the lower left is referred to as the third reference point. write. The worker refers to the captured image 10 displayed on the display and operates an input device such as a mouse connected to the image processing apparatus 200 to designate three reference points determined for each calibration object.

  FIG. 20 is a diagram illustrating an example of the data structure of the coordinate value table according to the second embodiment. As shown in FIG. 20, the coordinate value table 230a associates the calibration object number, the coordinates of the first reference point, the coordinates of the second reference point, and the coordinates of the third reference point. The calibration object number is a number that uniquely identifies the calibration object. The calibration information input unit 230 outputs the coordinate value table 230a to the local conversion calculation unit 240.

  The local conversion calculation unit 240 uses Expression (6) as local coordinate conversion. Although the local conversion calculation part 140 of Example 1 disclosed the structure corresponding to the change of the scale of a target object, it can respond also to the change of the direction of a target object using Formula (6).

In Expression (6), (δx, δy) ^T represents a minute change in the point of the corrected image, and (δu, δv) ^T represents a minute change in the point of the captured image. Note that the superscript T indicates transposition of a matrix or a vector. “a”, “b”, “c”, and “d” are linear transformation parameters, which differ depending on the point (u, v) of the photographed image. The case of b = c = 0 and a = d corresponds to the conversion with only the reduced scale (configuration of the first embodiment, etc.).

Three sets of reference points designated by the worker are expressed as ( _{pi, 0} , qi _{, 0} ), ( _{pi, x} , qi _{, x} ), ( _{pi, y} , qi _{, y} ) (i = 1,..., N). The standard width of the calibration object is w ₀ and the standard height is h ₀ . At this time, reverse conversion parameters a ′, b ′, c ′, and d ′ are determined by Expressions (7) to (10).

The linear conversion parameters a, b, c, and d in Equation (6) are defined as inverse matrices of a matrix composed of a ′, b ′, c ′, and d ′, and are expressed by Equations (11) to (14). The Here, the argument (p _{i, 0} , q _{i, 0} ) is omitted.

  The local conversion calculation unit 240 calculates local linear conversion parameters a, b, c, and d corresponding to each reference point. The local conversion calculation unit 240 outputs the local linear conversion parameters a, b, c, and d to the global conversion determination unit 250.

  The global conversion determination unit 250 is a processing unit that generates a correction parameter for converting a captured image into a corrected image based on a local linear conversion parameter. The global conversion determination unit 250 generates a correction parameter by sequentially executing, for example, a data interpolation process, a data addition process, and an inverse conversion calculation process. The global conversion determination unit 250 stores the correction parameter in the correction parameter storage unit 260.

  Data interpolation processing executed by the global conversion determination unit 250 will be described. The global conversion determination unit 250 calculates a linear conversion parameter at each pixel for all pixels of the captured image based on the local linear conversion parameter. This is obtained by interpolating a linear transformation parameter (local linear transformation parameter) obtained from a reference point designated by the worker.

The coordinates of the first reference point specified in the operator and _{(p i, 0, q i} , 0), the linear conversion parameters of such coordinates _{a i, b i, c i} , and _{d i.} i = 1,..., n represents the number of calibration objects, and n represents the total number of calibration objects. At this time, the linear conversion parameters for the pixel (u, v) of the captured image are defined by Expressions (15) to (18).

  In Expressions (15) to (18), dist represents the distance between two points. w (u, v) represents weight normalization and is defined by equation (19).

  The global conversion determination unit 250 calculates linear conversion parameters for each pixel (u, v) of the captured image based on Expressions (15) to (19). FIG. 21 is a diagram illustrating a calculation example of the linear conversion parameter of each pixel. In FIG. 21, pixels 30a to 30f are shown. Let the pixel at the origin be 30a. The linear parameters of the pixel 30a are a1, b1, c1, and d1. The linear parameters of the pixel 30b are a2, b2, c2, and d2. The linear parameters of the pixel 30c are a3, b3, c3, and d3. The linear parameters of the pixel 30d are a4, b4, c4, and d4. The linear parameters of the pixel 30e are a5, b5, c5, and d5. The linear parameters of the pixel 30f are a6, b6, c6, and d6.

  Next, the data addition process executed by the global conversion determination unit 250 will be described. FIG. 22 is a diagram for explaining the data addition processing according to the second embodiment. Similar to the global conversion determination unit 150 of the first embodiment, the global conversion determination unit 250 determines the origin of the captured image and the origin of the correction image, and associates the origin of the captured image with the origin of the correction image. . Then, the global transformation determination unit 250 determines the global coordinate transformation in the forward direction by adding linear transformation parameters in order from the pixel at the origin.

  FIG. 22 shows the relationship between the pixels on the captured image and the linear conversion parameters corresponding to the pixels. For example, the coordinate value of the point of the corrected image corresponding to the pixel 30a of the captured image is (0, 0). The coordinates of the image point of the corrected image corresponding to the pixel 30b of the captured image are (a1, c1). The coordinates of the point of the corrected image corresponding to the pixel 30c of the captured image are (a1 + a2, c1 + c2). The coordinates of the point of the corrected image corresponding to the pixel 30d of the photographed image are (b1, d1). The coordinates of the point of the corrected image corresponding to the pixel 30e of the photographed image are (a1 + b2, c1 + d2). The coordinates of the point of the corrected image corresponding to the pixel 30f of the photographed image are (a1 + a2 + b3, c1 + c2 + d3).

  Next, the inverse conversion calculation process executed by the global conversion determination unit 250 will be described. The inverse conversion calculation process executed by the global conversion determination unit 250 is the same as the inverse conversion calculation process executed by the global conversion determination unit 150 of the first embodiment.

  Next, effects of the image processing apparatus 200 according to the second embodiment will be described. The image processing apparatus 200 receives the positions of reference points for a plurality of calibration objects included in the captured image, calculates linear transformation parameters, and generates a corrected image by correcting the captured image so that each scale is constant. Then, feature points are extracted from the corrected image and template matching is executed. By executing such processing, the size of the same object becomes uniform regardless of position and orientation, so there is no need to perform matching by changing the scale of the template or image in various ways, reducing the amount of processing. can do.

  Next, an image processing apparatus according to the third embodiment will be described. The system configuration is the same as that of the system shown in the first embodiment.

  FIG. 23 is a functional block diagram of the configuration of the image processing apparatus according to the third embodiment. As shown in FIG. 23, the image processing apparatus 300 is connected to the camera 50. The image processing apparatus 300 includes an acquisition unit 310, a calibration object display unit 320, a calibration information input unit 330, a local conversion calculation unit 340, a global conversion determination unit 350, and a correction parameter storage unit 360. The image processing apparatus 300 includes a corrected image generation unit 370, a template matching execution unit 380, and a detection result output unit 390.

  The description of the camera 50, the acquisition unit 310, the calibration object display unit 320, the calibration information input unit 330, and the local transformation calculation unit 340 is as follows: the camera 50, the acquisition unit 210, the calibration object display unit 220, the calibration information input unit 230, FIG. This is the same as the description regarding the local conversion calculation unit 240. The description regarding the correction image generation unit 370, the template matching execution unit 380, and the detection result output unit 390 is the same as the description regarding the correction image generation unit 270, the template matching execution unit 280, and the detection result output unit 290 of FIG.

  In the second embodiment, it is necessary to determine the order of pixels when determining global coordinate transformation. Changing the order of the pixels generally changes the resulting conversion. Originally, the order is not given to the pixels, and a method of determining conversion without depending on the order is desirable. Therefore, this third embodiment discloses a conversion determination method that does not depend on the order of pixels.

  The global conversion determination unit 350 is a processing unit that generates a correction parameter for converting a captured image into a corrected image based on a local linear conversion parameter. The global conversion determination unit 350 stores the parameters in the correction parameter storage unit 360.

  For example, the global conversion determination unit 350 generates a correction parameter by sequentially executing a forward conversion determination process and an inverse conversion calculation process.

  The forward conversion determination process executed by the global conversion determination unit 350 will be described. In general, global coordinate transformation is approximated by linear transformation using a derivative as a transformation parameter in a narrow range. Based on this property, in the forward conversion determination process, the global conversion determination unit 350 predetermines a function (family) having an unknown coefficient, and the derivative thereof matches as much as possible with the given local coordinate conversion. The coefficient is determined by the least square method. Thereby, the forward global coordinate transformation that closely approximates the local coordinate transformation at each point of the image can be determined.

  The global transformation determination unit 350 uses, for example, Expression (20) and Expression (21) as the function family.

The evaluation function F of the least square method is expressed by Expression (22). In the equation (22), the subscript suffix represents the partial differentiation by the variable. Since the evaluation function F is a quadratic expression with respect to the unknown coefficients a ₀ ,..., A ₅ , b ₀ ,..., B ₅ , the minimization is reduced by solving the linear simultaneous equations. The linear simultaneous equations can be solved, for example, by a known Gaussian elimination method.

When the coefficients a ₀ ,..., A ₅ , b ₀ ,..., B ₅ are obtained, for each pixel (u, v) of the photographed image, the corresponding corrected image point (f ^x , f ^y ) can be calculated.

  Next, the inverse conversion calculation process executed by the global conversion determination unit 350 will be described. The inverse transformation calculation process executed by the global transformation determination unit 350 is the same as the inverse transformation calculation process executed by the global transformation determination unit 150 of the first embodiment.

  Next, effects of the image processing apparatus 300 according to the third embodiment will be described. The image processing apparatus 300 predetermines a family of functions having unknown coefficients based on the characteristic that global coordinate transformation is approximated by linear transformation using the derivative as a transformation parameter in a narrow range. The coefficient is determined by the least square method so that is as close as possible to the given local coordinate transformation. Thereby, the point of the corrected image corresponding to the pixel of the captured image can be specified without depending on the order of the pixels.

  Next, an image processing apparatus according to the fourth embodiment will be described. The system configuration is the same as that of the system shown in the first embodiment.

  In the first to third embodiments, both local coordinate conversion and global coordinate conversion indicate to which point of the corrected image each pixel of the captured image corresponds. Therefore, inverse transformation is calculated when generating a corrected image. On the other hand, in the fourth embodiment, conversion is determined in the opposite direction to the first to third embodiments. Thereby, the process which calculates reverse transformation becomes unnecessary. Since the number of conversions is reduced, the quality of the interpolated image can be improved. In addition, as will be described later, it is mathematically guaranteed that global coordinate transformation satisfies desirable properties.

  FIG. 24 is a functional block diagram of the configuration of the image processing apparatus according to the fourth embodiment. As shown in FIG. 24, the image processing apparatus 400 is connected to the camera 50. The image processing apparatus 400 includes an acquisition unit 410, a calibration object display unit 420, a calibration information input unit 430, a local conversion calculation unit 440, a global conversion determination unit 450, and a correction parameter storage unit 460. The image processing apparatus 400 includes a corrected image generation unit 470, a template matching execution unit 480, and a detection result output unit 490.

  The camera 50, the acquisition unit 410, the calibration object display unit 420, and the calibration information input unit 430 are described with reference to the camera 50, the acquisition unit 110, the calibration object display unit 120, and the calibration information input unit 130 (or the configuration information input unit in FIG. 2). 230). The description regarding the correction image generation unit 470, the template matching execution unit 480, and the detection result output unit 490 is the same as the description regarding the correction image generation unit 170, the template matching execution unit 180, and the detection result output unit 190 of FIG.

The local conversion calculation unit 440 uses Expression (23) as local coordinate conversion. In Expression (23), (δu, δv) ^T represents a minute change in the point of the captured image, and (δx, δy) ^T represents a minute change in the point of the corrected image. “a”, “b”, “c”, and “d” are linear conversion parameters, which differ depending on the point (u, v) of the photographed image.

Three sets of reference points designated by the worker are expressed as ( _{pi, 0} , qi _{, 0} ), ( _{pi, x} , qi _{, x} ), ( _{pi, y} , qi _{, y} ) (i = 1,..., N). The standard width of the calibration object is w ₀ and the standard height is h ₀ . At this time, parameters a, b, c, and d for local coordinate conversion are determined by equations (24) to (27).

  The local conversion calculation unit 440 outputs the local linear conversion parameters a, b, c, and d to the global conversion determination unit 450.

  The global conversion determination unit 450 is a processing unit that generates a correction parameter for converting a captured image into a corrected image based on a local linear conversion parameter. The global conversion determination unit 450 generates a correction parameter by sequentially executing, for example, data regression processing and integration processing. The global conversion determination unit 460 stores the correction parameter in the correction parameter storage unit 460.

  Data regression processing executed by the global conversion determination unit 450 will be described. The global transformation determination unit 450 predetermines a family of functions having unknown coefficients, and obtains the coefficients by the least square method so as to match the given local coordinate transformation as much as possible. Thereby, local coordinate transformation can be obtained for all points of the captured image.

  The global transformation determination unit 450 uses, for example, Expression (28) to Expression (31) as the function family.

  The evaluation function F of the least square method is expressed by Expression (32).

The global transformation determination unit 450 makes the coefficients a ₀ ,..., A ₅ , b ₀ ,..., B ₅ , c ₀ ,. , C ₅ , d ₀ ,..., D ₅ are determined. However, in order to execute the integration processing described later, as mathematical requests, f ^{u, x} , f ^{u, y} , f ^{v, x} , f ^{v, y} satisfy the conditions of the expressions (33) and (34). There is a need.

  In the case of the quadratic expression in the above example, the following constraint expressions (35) to (54) relating to the coefficients are obtained by calculating the integrable conditions of Expression (33) and Expression (34).

  The global transformation determination unit 450 searches for a combination that minimizes the evaluation function F from among the combinations of coefficients that satisfy the constraints (35) to (54). For example, the global transformation determination unit 450 searches for a coefficient that satisfies the constraint equations (35) to (54) and minimizes the evaluation function F based on the Lagrange undetermined multiplier method that is a known technique.

  Next, the integration process of the global conversion determination unit 450 will be described. The global conversion determination unit 450 solves the partial differential equation shown in Expression (55).

The global conversion determination unit 450 selects, as an initial condition of the partial differential equation, one point (x ₀ , y ₀ ) of the corrected image corresponds to one point (u ₀ , v ₀ ) of the captured image. For example, the global conversion determination unit 450 may select the center of the corrected image and the center of the captured image as the initial condition of the partial differential equation.

Expressions (35) to (54), which are the above-mentioned integration enable conditions, are necessary and sufficient conditions for the solution to exist in the partial differential equation (55). Specifically, the integrable condition represents that the value obtained by differentiating u _x by y matches the value obtained by differentiating u _y by x. The same applies to v. On the other hand, if the integrable condition is satisfied, Green's auxiliary formula indicates that the line integral does not depend on the path, so u and v defined by Equation (56) and Equation (57) are the solutions of the differential equation. .

As a method for the global transformation determination unit 450 to numerically solve the partial differential equation, for example, the Euler method which is a known technique is used. Since the line integration does not depend on the path, the global transformation determination unit 450 performs the integration along an appropriate path connecting one point (x ₀ , y ₀ ) selected in the initial condition and each point (x, y). Find the solution by approximating with the sum. The solved result is expressed by the coordinates (u, v) of the point of the captured image corresponding to each pixel (x, y) of the corrected image, and becomes a correction parameter.

FIG. 25 is a diagram illustrating an example of the data structure of the correction parameter according to the fourth embodiment. As shown in FIG. 25, the correction parameter 460a has the coordinate value of the pixel of the corrected image and the coordinate value of the point of the captured image. For example, it is indicated that the coordinate value of the point of the photographed image corresponding to the coordinate value (0, 0) of the pixel of the corrected image is (u _0,0 , v _0,0 ).

  Here, the property of global coordinate transformation will be described.

  In the present application, since the target object is detected from the corrected image, it is necessary that the corrected image obtained as a result of the correction process represents the entire captured image, and the correction is not performed so that the target object becomes a double copy. This means that the global coordinate transformation that generates the corrected image is bijective globally. That the transformation φ is bijective globally is that a point of the corrected image (u, v) = φ (x, y) with respect to an arbitrary point (u, v) of the captured image ( x, y) is present and such (x, y) represents only one.

Assuming that the reference point of the calibration object is visible in the photographed image and that the operator makes an appropriate designation, the three reference points do not ride on one straight line. In this case, a (u, v) d (u, v) -b (u, v) c (u, v) is u = _{pi, 0} , v = qi _{, 0} (i = 1,... , N) all take non-zero values. ^{Fu, x} , ^{fu, y} , ^{fv, x} , ^{fv, and y} are functions obtained by interpolating a, b, c, and d so as to have values at all points of the captured image. Therefore, it can be expected that f ^{u, x} (u, v) f ^{v, y} (u, v) −fu ^{, y} (u, v) f ^{v, x} (u, v) do not become zero at all points. . The above equation is equal to the Jacobian in the global transformation, that is, the determinant of the first derivative Jacobian matrix.

  In general, it is known that if the Jacobian of the conversion is not 0, the conversion is locally bijective. However, global bijectivity is not always guaranteed. For example, x (u, v) = u cosv or u * cos (v) etc., y (u, v) = u sin v or u * sin (v) etc. does not become Jacobian 0 when u> 0. Since the values match at v = 0, 2π, it is not bijective globally.

  However, the global transformation defined in this embodiment is mathematically guaranteed to be bijective globally. This is a desirable property. The following is a summary of mathematical theorems.

[Theorem] It is assumed that ^{fu, x} , ^{fu, y} , ^{fv, x} , ^{fv, and y} are first-order differentiable, the differentiation is continuous, and satisfy an integrable condition. At this time, the solutions u and v of the partial differential equations are bijective globally unless the Jacobian is 0. However, in the case of injectivity, the domain is assumed to be convex.

[Proof] Injectivity: Denies the conclusion and leads to contradiction. It is assumed that there are solutions u and v that take the same value (u ₁ , v ₁ ) for two different points (x ₁ , y ₁ ) and (x ₂ , y ₂ ). Consider a line segment L connecting these two points. The direction of the line segment is set as (a, b). Since the line segment L is included in the domain, u and v are closed curves C for points on L. Since the differentiation of u and v is determined only by the values of u and v, C becomes a simple closed curve without self-intersection. From the Jordan curve theorem, the closed curve C divides the plane into an inside and an outside. The inner side D becomes a bounded single connected component. Consider a vector field that correlates vectors (af ^{u, x} + bf ^{u, y} , af ^{v, x} + bf ^{v, y} ) at points C and D. This vector field is continuous. Further, since the Jacobian is not 0, the vector size does not become 0 at each point of C and D. On the other hand, in C, the above vector faces the tangential direction of C. This contradicts the Poincare hop theorem. Therefore, it is injectable globally.

Radiality: indicates that it is surjective for any bounded region (connected open set) Ω including (u ₀ , v ₀ ) specified in the initial condition. The following lemma can be obtained by slightly modifying the proof of the inverse mapping theorem, which is well known in analytics. Arbitrary (u ′) satisfying | (u ′, v ′) − (u, v) | <r with respect to arbitrary (u, v) that becomes a mapping image (belonging to Ω) , V ′) is also a mapping image. Here, r is a positive number and depends only on the distance d to the boundary of (u, v). r = r (d) is a monotonically increasing function. In the following, this lemma is used to show the bijectivity. Since Ω is connected, from the general theory of phase space theory, when Ω is represented by merging two open sets A and B having no common part, one open set is an empty set. Here, a subset A of Ω is defined as a collection of elements that become mapping images. Further, the subset B is defined as a collection of elements that do not become mapping images. Obviously, there is no common part between the two, and the merger of A and B matches Ω. Since (u ₀ , v ₀ ) is an element of A, A is not an empty set. It is clear that A is an open set from the local radiation. The fact that B becomes an open set is shown as follows. If the conclusion is negated, for some (u, v) and any positive number ε belonging to B, an element of A that satisfies | (u ′, v ′) − (u, v) | <ε (u ′, v ′) exists. First, let d be the distance to the boundary of (u, v). The r of the lemma is defined as r = r (d / 2). Let ε = min (d / 2, r). At this time, the distance d ′ to the boundary of the element (u ′, v ′) of A is not less than d / 2, and | (u ′, v ′) − (u, v) | <r (d / 2) ≦ r (d ′). From the above lemma, (u, v) becomes an element of A and contradicts. Therefore, it is derived that B becomes an empty set. This represents a global shot.

  Next, effects of the image processing apparatus 400 according to the fourth embodiment will be described. The image processing apparatus 400 predetermines a family of functions having unknown coefficients, and obtains the coefficients by the least square method so as to match the given local coordinate transformation as much as possible. Further, the image processing apparatus 400 specifies the coordinates (u, v) of the point of the captured image corresponding to each pixel (x, y) of the corrected image by solving the partial differential equation using the obtained coefficient. That is, the image processing apparatus 400 determines the conversion in the opposite direction to that of the first to third embodiments, so that the process for calculating the reverse conversion becomes unnecessary. For this reason, since the number of times of conversion is reduced, the quality of the interpolated image can be improved.

  Next, an example of a computer that executes an image processing program that implements the same functions as those of the image processing apparatuses 100 to 400 described in the above embodiment will be described. FIG. 26 is a diagram illustrating an example of a computer that executes an image processing program.

  As illustrated in FIG. 26, the computer 500 includes a CPU 501 that executes various arithmetic processes, an input device 502 that receives input of data from a user, and a display 503. The computer 500 includes a reading device 504 that reads a program and the like from a storage medium, and an interface device 505 that exchanges data with other computers via a network. The computer 500 also includes a RAM 506 that temporarily stores various types of information and a hard disk device 507. The devices 501 to 507 are connected to the bus 508.

  The hard disk device 507 has an acquisition program 507a, a calculation program 507b, and a detection program 507c. The CPU 501 reads out the acquisition program 507a, the calculation program 507b, and the detection program 507c and develops them in the RAM 506. The acquisition program 507a functions as an acquisition process 506a. The calculation program 507b functions as a calculation process 506b. The detection program 507c functions as a detection process 506c. The processing of the acquisition process 506a corresponds to the processing of the acquisition units 110, 210, 310, and 410. The processing of the calculation process 506b corresponds to the processing of the local conversion calculation units 140, 240, 340, 440 and the global conversion determination units 150, 250, 350, 450. The processing of the detection process 506c corresponds to the processing of the corrected image generation units 170, 270, 370, 470, the template matching execution units 180, 280, 380, 480, and the detection result output units 190, 290, 390, 490.

  Note that the acquisition program 507a, the calculation program 507b, and the detection program 507c are not necessarily stored in the hard disk device 507 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, and an IC card inserted into the computer 500. Then, the computer 500 may read and execute the acquisition program 507a, the calculation program 507b, and the detection program 507c.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Supplementary note 1)
Acquire a captured image of multiple objects of the same shape,
Receiving positions of reference points for the plurality of objects included in the photographed image, and calculating a scale at each pixel of the photographed image based on an interval between the reference points of the objects;
Based on the scale of each pixel of the photographed image, the photographed image is converted into a corrected image in which the scale of each pixel is constant,
A template matching program that executes processing for detecting an object by executing template matching on the corrected image.

(Supplementary Note 2) The process of calculating the scale at each pixel of the captured image calculates a local scale corresponding to the position of the object in the captured image based on the interval between the reference points of the object, The template matching program according to appendix 1, wherein a scale at each pixel of the captured image is calculated by performing interpolation using a local scale.

(Additional remark 3) The process which calculates the reduced scale in each pixel of the said picked-up image calculates the linear conversion parameter which converts the point of the said picked-up image into the point of the said correction | amendment image based on the space | interval of the reference point of the said object. The template matching program according to appendix 1, wherein the process of converting to the corrected image converts the captured image into a corrected image having a constant scale in each pixel based on the linear conversion parameter. .

(Additional remark 4) The process which calculates the reduced scale in each pixel of the said picked-up image calculates the local scale corresponding to the position of the said object of the said picked-up image based on the space | interval of the reference point of the said object, The relationship between the local scale and the scale of a certain point is shown, and the scale of each pixel of the captured image is determined by specifying the unknown coefficient based on a function obtained by differentiating a function having an unknown coefficient. The template matching program according to supplementary note 1, wherein the template matching program is calculated.

(Additional remark 5) The process which calculates the reduced scale in each pixel of the said picked-up image converts the local point of the said correction image into the local point of the said picked-up image based on the position of the reference point of the said object. The first function is specified, the unknown coefficient is specified so that a function obtained by differentiating the second function having the unknown coefficient matches the first function, and the specified unknown coefficient is set to the second function. The template matching program according to appendix 1, wherein a scale at each pixel of the captured image is calculated based on the template.

(Appendix 6) A template matching method executed by a computer,
Acquire a captured image of multiple objects of the same shape,
Receiving positions of reference points for the plurality of objects included in the photographed image, and calculating a scale at each pixel of the photographed image based on an interval between the reference points of the objects;
Based on the scale of each pixel of the captured image, generate a corrected image in which the scale of each pixel is constant,
The template matching method characterized by performing the process which detects a target object by performing template matching with respect to the said correction | amendment image.

(Supplementary Note 7) The process of calculating the scale at each pixel of the captured image calculates a local scale corresponding to the position of the object in the captured image based on the interval between the reference points of the object, The template matching method according to appendix 6, wherein a scale at each pixel of the captured image is calculated by performing interpolation using a local scale.

(Supplementary Note 8) The process of calculating the scale at each pixel of the captured image calculates linear conversion parameters for converting the points of the captured image into the points of the corrected image based on the interval between the reference points of the object. The template matching method according to appendix 6, wherein the process of converting to the corrected image converts the captured image into a corrected image in which the scale of each pixel is constant based on the linear conversion parameter. .

(Supplementary note 9) The process of calculating the scale at each pixel of the photographed image calculates a local scale corresponding to the position of the object in the photographed image based on the interval between the reference points of the object, The relationship between the local scale and the scale of a certain point is shown, and the scale of each pixel of the captured image is determined by specifying the unknown coefficient based on a function obtained by differentiating a function having an unknown coefficient. 8. The template matching method according to appendix 7, wherein the template matching method is calculated.

(Additional remark 10) The acquisition part which acquires the picked-up image by which the several object of the same shape was imaged,
Accepting positions of reference points for the plurality of objects included in the captured image, calculating a scale at each pixel of the captured image based on an interval between the reference points of the objects, and calculating a scale at each pixel of the captured image. Based on the correction image generation unit that generates a correction image in which the scale of each pixel is constant,
And a detection unit that detects an object by executing template matching on the corrected image.

(Additional remark 11) The said correction | amendment image generation part calculates the local scale corresponding to the position of the said object of the said picked-up image based on the space | interval of the reference point of the said object, and uses the said local scale. The template matching apparatus according to appendix 10, wherein a scale at each pixel of the captured image is calculated by performing interpolation.

(Supplementary Note 12) The correction image generation unit calculates a linear conversion parameter for converting a local point of the captured image into a local point of the correction image based on an interval between reference points of the object, The template matching apparatus according to appendix 10, wherein a scale at each pixel of the photographed image is calculated based on the linear conversion parameter.

(Additional remark 13) The said correction | amendment image generation part calculates the local scale corresponding to the position of the said object of the said picked-up image based on the space | interval of the reference point of the said object, and there exists the said local scale A scale at each pixel of the captured image is calculated by specifying the unknown coefficient based on a function obtained by differentiating a function having an unknown coefficient. The template matching apparatus according to appendix 10.

(Supplementary Note 14) The correction image generation unit specifies a first function for converting a local point of the correction image into a local point of the captured image based on the position of the reference point of the object, The unknown coefficient is specified such that a function obtained by differentiating the second function having an unknown coefficient matches the first function, and each of the captured images is determined based on the second function in which the specified unknown coefficient is set. The template matching apparatus according to appendix 10, wherein a scale of the pixel is calculated.

50 Camera 100, 200, 300, 400 Image processing device 110, 210, 310, 410 Acquisition unit 120, 220, 320, 420 Calibration object display unit 130, 230, 330, 430 Calibration information input unit 140, 240, 340, 440 Local transformation calculation unit 150, 250, 350, 450 Global transformation determination unit 160, 260, 360, 460 Correction parameter storage unit 170, 270, 370, 470 Correction image generation unit 180, 280, 380, 480 Template matching execution unit 190, 290, 390, 490 Detection result output unit

Claims (7)

On the computer,
Acquire a captured image of multiple objects of the same shape,
Receiving positions of a plurality of reference points for each of the plurality of objects included in the photographed image, and calculating a scale at each pixel of the photographed image based on an interval between the plurality of reference points;
Based on the scale of each pixel of the captured image, generate a corrected image in which the scale of each pixel is constant,
A template matching program that executes processing for detecting an object by executing template matching on the corrected image. The process of calculating the scale at each pixel of the captured image calculates a local scale corresponding to the position of the object of the captured image based on the interval between the plurality of reference points, and the local scale. The template matching program according to claim 1, wherein a scale at each pixel of the captured image is calculated by performing interpolation using. The process of calculating the scale at each pixel of the captured image includes linear conversion parameters for converting a local point of the captured image into a local point of the corrected image based on an interval between the plurality of reference points. The template matching program according to claim 1, wherein the template matching program is calculated and a scale of each pixel of the captured image is calculated based on the linear conversion parameter. The process of calculating the scale at each pixel of the captured image calculates a local scale corresponding to the position of the object of the captured image based on the interval between the plurality of reference points, and the local scale. Calculating the scale at each pixel of the captured image by identifying the unknown coefficient based on a function obtained by differentiating a function having an unknown coefficient The template matching program according to claim 1. The process of calculating the scale at each pixel of the captured image includes a first function for converting a local point of the corrected image into a local point of the captured image based on the positions of the plurality of reference points. Based on the second function that specifies the unknown coefficient and sets the specified unknown coefficient so that a function obtained by identifying and differentiating the second function having the unknown coefficient matches the first function. The template matching program according to claim 1, wherein a scale at each pixel of the image is calculated. A template matching method executed by a computer,
Acquire a captured image of multiple objects of the same shape,
Receiving positions of a plurality of reference points for each of the plurality of objects included in the photographed image, and calculating a scale at each pixel of the photographed image based on an interval between the plurality of reference points;
Based on the scale of each pixel of the captured image, generate a corrected image in which the scale of each pixel is constant,
The template matching method characterized by performing the process which detects a target object by performing template matching with respect to the said correction | amendment image. An acquisition unit for acquiring a captured image in which a plurality of objects having the same shape are captured;
Receiving positions of a plurality of reference points for each of the plurality of objects included in the photographed image, calculating a scale at each pixel of the photographed image based on an interval between the plurality of reference points, and each pixel of the photographed image A corrected image generation unit that generates a corrected image in which the scale of each pixel is constant based on the scale of
And a detection unit that detects an object by executing template matching on the corrected image.

Images