JP5471669B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

  As image recognition techniques for detecting a specific pattern from within an image, for example, techniques such as template matching and Hough transform exist. However, both techniques have a problem that a huge amount of processing is required in accordance with search processing in all parameter spaces such as horizontal, vertical position, rotation angle, and scale in the screen.

  Therefore, in recent years, a technique for estimating a scale parameter of an image feature using a scale space theory has been developed for the purpose of reducing a processing amount associated with a search process in a parameter space. This technology uses the response of a spatial filter such as a Laplacian of Gaussian filter with different scales to reduce the search processing amount in the parameter space and to extract the same image features from images with different scales. Can be extracted. In the following description, “filter” means a Laplacian / Gaussian filter.

  FIG. 16 is a diagram illustrating a characteristic example corresponding to the scale of the Laplacian / Gaussian filter. In FIG. 16, in order from the top, curves representing examples of filter characteristics when the filter scale is “σ = 1.0, 2.0, 3.0” are drawn. As shown in FIG. 16, the slope of the curve representing the filter characteristics becomes gentler as the scale increases.

  FIG. 17 is a diagram illustrating an example of a relationship between a response and a scale obtained by applying a Laplacian / Gaussian filter to an input image. The vertical axis in FIG. 17 represents the “LoG output value” that is the output value of the Laplacian-Gaussian filter, and the horizontal axis in FIG. 17 represents the scale “σ” of the Laplacian-Gaussian filter. As shown in FIG. 17, when the filter scale “σ” is 2.0, the “LoG output value” of the filter is the maximum. In the following, what is described as “output value” means “LoG output value”.

  The “LoG output value” obtained by applying the Laplacian / Gaussian filter to the image is calculated by the following equation (1) using the filter scale “σ” and the distance from the target pixel as parameters. Note that the distance from the target pixel is represented by the values of the X coordinate and the Y coordinate on the image.

JP 2003-22141 A JP 2007-227522 A

  However, the technique using the scale space theory described above has a problem that a specific pattern cannot be detected with high accuracy when the image is distorted.

  FIG. 18 is a diagram illustrating an example of local peak extraction from an input image using a Laplacian / Gaussian filter. In addition, a local peak means the pixel by which the response by a filter, ie, an output value, becomes a local peak among the pixels located in the vicinity of a certain pixel of interest in an image. The pixel having a local peak is a pixel in which the output value obtained by the filter between the target pixel and a pixel located in the vicinity of the target pixel is maximum or minimum. The pixels extracted in this way become image feature points.

  18-1 shown in FIG. 18 represents an input image. 18-2 shown in FIG. 18 represents a Laplacian-Gaussian filter having a scale “σ” of 1.0. 18-3 shown in FIG. 18 represents a Laplacian-Gaussian filter having a scale “σ” of 1.5. 18-4 shown in FIG. 18 represents a Laplacian-Gaussian filter having a scale “σ” of 2.0. 18-5 shown in FIG. 18 represents a Laplacian-Gaussian filter having a scale “σ” of 2.5. 18-6 shown in FIG. 18 represents a Laplacian-Gaussian filter having a scale “σ” of 3.0.

  18-2 ′ shown in FIG. 18 is a scale space image when the Laplacian / Gaussian filter 18-2 is applied to the input image 18-1. 18-3 'shown in FIG. 18 is a scale space image when the Laplacian / Gaussian filter 18-3 is applied to the input image 18-1. 18-4 ′ shown in FIG. 18 is a scale space image when the Laplacian / Gaussian filter 18-4 is applied to the input image 18-1. 18-5 ′ shown in FIG. 18 is a scale space image when the Laplacian-Gaussian filter 18-5 is applied to the input image 18-1. 18-6 ′ shown in FIG. 18 is a scale space image when the Laplacian-Gaussian filter 18-6 is applied to the input image 18-1.

  For example, as shown in FIG. 18, by applying Laplacian-Gaussian filters 18-2 to 18-6 to the input image data 18-1, scale space images 18-2 'to 18- formed from the responses are applied. 6 ′ is created. Next, out of the scale space images 18-2 ′ to 18-6 ′, pixels whose “LoG output value” obtained by the filter has a local peak are extracted from three scale space images having adjacent filter scales. To do. For example, as shown in FIG. 18, from the three scale space images 18-2 ′ to 18-4 ′ adjacent to each other, the “LoG output value” obtained by the filter has a local peak 18− 7 is extracted. Similarly, the pixel 18-8 in which the “LoG output value” obtained by the filter locally peaks is extracted from the scale space images 18-3 ′ to 18-5 ′. Similarly, the pixel 18-9 in which the “LoG output value” obtained by the filter locally peaks is extracted from the scale space images 18-4 ′ to 18-6 ′.

  FIG. 19 is a diagram used for explaining a local peak extraction method. 19-1 to 19-3 illustrated in FIG. 19 represent three scale space images with adjacent scales, for example, the scale “σ” is 1.0, 1.5, and 2.0. 19-4 shown in FIG. 19 represents a pixel of interest when a local peak is extracted. For example, as illustrated in FIG. 19, a filter is obtained from the three scale space images adjacent to each other with the target pixel 19-4 out of 26 pixels in the vicinity of the target pixel 19-4. A pixel having the maximum or minimum output value is extracted.

  FIG. 20 is a diagram illustrating the relationship between the filter response and the filter scale according to the image size. The upper part of FIG. 20 is a diagram illustrating an example of a relationship between a filter response and a filter scale when the filter is applied to an input image having a size of 200 × 200 pixels. The lower part of FIG. 20 is an image of the same subject as the image shown in the upper part of FIG. 20, and the relationship between the filter response and the filter scale when the filter is applied to an input image having a size of 400 × 400 pixels. It is a figure which shows an example. 20 conceptually represents a state in which a filter is applied to an input image having a size of 200 × 200 pixels. 20-3 of FIG. 20 conceptually represents a state in which a filter is applied to an input image having a size of 400 × 400 pixels.

As shown in the upper part of FIG. 20, for a target pixel of an input image having a size of 200 × 200 pixels, as shown by 20-2 in FIG. 20, when the filter scale “σ” is 1.5, “LoG "Output value" is the maximum. As shown in the lower part of FIG. 20, in the case of an input image having a size of 400 × 400 pixels, “LoG output value” when the filter scale “σ” is 3.0, as indicated by 20-4 in FIG. Is the maximum. The filter scale when the “LoG output value” of the input image having the size of 200 × 200 pixels is the maximum is “σ ₁ ”, and the “LoG output value” of the input image having the size of 400 × 400 pixels is the maximum. When the scale at this time is “σ ₂ ”, the relationship “σ ₂ ” = 2 × “σ ₁ ” is established. When such a relationship is established, a pixel extracted from the scale space image can be regarded as a feature point that is invariant to the scale of the image.

  FIG. 21 is a conceptual diagram showing how a Laplacian / Gaussian filter is applied to an undistorted input image. 21-1 shown in FIG. 21 is an input image model representing an undistorted input image. 21-2 shown in FIG. 21 is a filter model representing a Laplacian-Gaussian filter. 21-3 shown in FIG. 21 is a conceptual model representing a state in which a filter is applied to an input image.

  FIG. 22 is a diagram illustrating a planar model when the response result of the filter corresponding to 21-3 in FIG. 21 is viewed from the vertical direction with respect to the input image. FIG. 23 is a diagram illustrating a three-dimensional model when the response result of the filter corresponding to 21-3 in FIG. 21 is viewed from a predetermined viewpoint. As shown in FIG. 23, when the filter 21-2 is applied to the input image 21-1 shown in FIG. 21, the response result of the filter having a certain peak is obtained. That is, when the image is not distorted, it means that one pixel is extracted as a feature point from the image.

  FIG. 24 is a conceptual diagram showing how a Laplacian / Gaussian filter is applied to a distorted input image. 24-1 shown in FIG. 24 is a model of an input image representing a distorted input image. 24-2 and 24-3 shown in FIG. 24 are filter models representing Laplacian and Gaussian filters having different scales. 24-4 and 24-5 shown in FIG. 24 are conceptual models representing a state in which the filter 24-2 is applied to the input image 24-1. 24-6 and 24-7 shown in FIG. 24 are conceptual models representing a state in which the filter 24-3 is applied to the input image 24-1.

  FIG. 25 is a diagram illustrating a planar model when the response result of the filter corresponding to 24-4 in FIG. 24 is viewed from the vertical direction with respect to the input image. FIG. 26 is a diagram illustrating a three-dimensional model when the response result of the filter corresponding to 24-4 in FIG. 24 is viewed from a predetermined viewpoint. FIG. 27 is a diagram illustrating a planar model when the response result of the filter corresponding to 24-6 in FIG. 24 is viewed from the vertical direction with respect to the input image. FIG. 28 is a diagram illustrating a three-dimensional model when the response result of the filter corresponding to 24-6 in FIG. 24 is viewed from a predetermined viewpoint.

  As shown in FIG. 26, when the filter 24-2 is applied to the input image 24-1 shown in FIG. 24, the response result of the filter having two peaks is obtained. Further, as shown in FIG. 28, when a filter 23-3 having a scale different from that of the filter 24-2 is applied to the input image 24-1 shown in FIG. Is obtained. That is, when the image is distorted, it means that a plurality of pixels are extracted as feature points from the image.

  That is, as shown in FIGS. 26 and 28, for example, when distortion occurs in the image depending on the viewing direction, it is not possible to extract feature points that are invariant to the scale of the image. As a result, for example, the matching rate between the feature points of a certain pattern in the image and the feature points of the template may be reduced, and a specific pattern may not be detected accurately from the image.

  The disclosed technology has been made in view of the above, and provides an image processing apparatus, an image processing method, and an image processing program capable of accurately detecting a specific pattern from an input image even when the image is distorted. The purpose is to provide.

  In one aspect, an image processing apparatus disclosed in the present application includes a first image generation unit, a first extraction unit, a second image generation unit, a second extraction unit, and a pattern extraction unit. The first image generation unit operates the input image and the reference by applying a plurality of spatial filters having different scales to the input image and the reference image used when extracting a specific pattern from the input image. A scale space image sequence of the image is generated respectively. The first extraction unit includes, for the scale space image sequence of the input image generated by the first image generation unit and the scale space image sequence of the reference image, the scale space image of which the scale of the spatial filter is adjacent. Determine multiple combinations. Then, the first extraction unit extracts a predetermined pixel from a predetermined number of pixels located in the vicinity of the target pixel in the scale space image included in the combination for each determined combination. For example, the first extraction unit extracts a pixel whose output value of the spatial filter calculated based on the inter-pixel distance to the target pixel and the scale of the spatial filter is the maximum value or the minimum value. The second image generation unit is a combination of scale space images having different scales of the spatial filter for the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation unit. Determine more than one. Then, the second image generation unit generates, for each determined combination, an added image obtained by adding the scale space images included in the combination. The second extraction unit determines a plurality of combinations including a predetermined number of the addition images for the plurality of addition images generated by the second image generation unit. Then, for each determined combination, the second extraction unit extracts a predetermined pixel from a predetermined number of pixels located near the target pixel in the scale space image included in the combination. For example, the second extraction unit extracts a pixel whose output value of the spatial filter calculated based on the inter-pixel distance from the target pixel and the scale of the spatial filter is the maximum value or the minimum value. The pattern detection unit compares each pixel extracted for the reference image by the first extraction unit and the second extraction unit with each pixel extracted for the input image by the first extraction unit and the second extraction unit. Based on the result, a specific pattern is detected from the input image.

  According to one aspect of the technology disclosed in the present application, it is possible to accurately detect a specific pattern from an input image even if the image is distorted.

FIG. 1 is a diagram illustrating the image processing apparatus according to the first embodiment. FIG. 2 is a diagram illustrating the configuration of the image processing apparatus according to the second embodiment. FIG. 3 is a conceptual diagram illustrating a state in which a Laplacian / Gaussian filter according to the second embodiment is applied to a distorted input image. FIG. 4 is a conceptual diagram including a three-dimensional model illustrating a state in which a Laplacian / Gaussian filter according to the second embodiment is applied to a distorted input image. FIG. 5 is a diagram illustrating an example of local peak extraction from an input image using a Laplacian / Gaussian filter according to the second embodiment. FIG. 6 is a conceptual diagram for explaining the inertial spindle direction according to the second embodiment. FIG. 7 is a diagram illustrating a three-dimensional model when the response result of the filter corresponding to 6-4 in FIG. 6 is viewed from a predetermined viewpoint. FIG. 8 is a diagram illustrating a planar model when the response result of the filter corresponding to 6-4 in FIG. 6 is viewed from the vertical direction with respect to the input image. FIG. 9 is a diagram illustrating a three-dimensional model when the response result of the filter corresponding to 6-6 in FIG. 6 is viewed from a predetermined viewpoint. FIG. 10 is a diagram illustrating a planar model when the response result of the filter corresponding to 6-6 in FIG. 6 is viewed from the vertical direction with respect to the input image. FIG. 11 is a diagram for explaining a calculation method in the inertial spindle direction according to the second embodiment. FIG. 12 is a diagram for explaining a distortion correction method for an input image and a template image according to the second embodiment. FIG. 13 is a diagram illustrating a flow of processing performed by the image processing apparatus according to the second embodiment. FIG. 14 is a diagram used for explaining the effect of the second embodiment. FIG. 15 is a diagram illustrating an example of a computer that executes an image processing program. FIG. 16 is a diagram illustrating a characteristic example corresponding to the scale of the Laplacian / Gaussian filter. FIG. 17 is a diagram illustrating an example of a relationship between a response and a scale obtained by applying a Laplacian / Gaussian filter to an input image. FIG. 18 is a diagram illustrating an example of local peak extraction from an input image using a Laplacian / Gaussian filter. FIG. 19 is a diagram used for explaining a local peak extraction method. FIG. 20 is a diagram illustrating the relationship between the filter response and the filter scale according to the image size. FIG. 21 is a conceptual diagram showing how a Laplacian / Gaussian filter is applied to an undistorted input image. FIG. 22 is a diagram illustrating a planar model when the response result of the filter corresponding to 21-3 in FIG. 21 is viewed from the vertical direction with respect to the input image. FIG. 23 is a diagram illustrating a three-dimensional model when the response result of the filter corresponding to 21-3 in FIG. 21 is viewed from a predetermined viewpoint. FIG. 24 is a conceptual diagram showing how a Laplacian / Gaussian filter is applied to a distorted input image. FIG. 25 is a diagram illustrating a planar model when the response result of the filter corresponding to 24-4 in FIG. 24 is viewed from the vertical direction with respect to the input image. FIG. 26 is a diagram illustrating a three-dimensional model when the response result of the filter corresponding to 24-4 in FIG. 24 is viewed from a predetermined viewpoint. FIG. 27 is a diagram illustrating a planar model when the response result of the filter corresponding to 24-6 in FIG. 24 is viewed from the vertical direction with respect to the input image. FIG. 28 is a diagram illustrating a three-dimensional model when the response result of the filter corresponding to 24-6 in FIG. 24 is viewed from a predetermined viewpoint.

  Hereinafter, embodiments of an image processing apparatus, an image processing method, and an image processing program disclosed in the present application will be described in detail with reference to the drawings. In the following, the technology disclosed by the present application is not limited by an example described later as an embodiment of the image processing apparatus, the image processing method, and the image processing program disclosed by the present application.

  FIG. 1 is a diagram illustrating the image processing apparatus according to the first embodiment. As illustrated in FIG. 1, the image processing apparatus 10 includes a first image generation unit 11, a first extraction unit 12, a second image generation unit 13, a second extraction unit 14, and a pattern extraction unit 15.

  The first image generation unit 11 generates an input image and a scale space image sequence of a reference image used when a specific pattern is extracted from the input image. For example, the first image generation unit 11 generates a scale space image sequence of the input image and the reference image by applying a plurality of spatial filters having different scales to the input image and the reference image, respectively.

  The first extraction unit 12 has a scale space in which the scale of the spatial filter is adjacent to the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation unit 11. A plurality of image combinations are determined. Then, the first extraction unit 12 extracts a predetermined pixel from a predetermined number of pixels located near the target pixel in the scale space image included in the combination for each determined combination. For example, the first extraction unit 12 extracts a pixel whose output value of the spatial filter calculated based on the inter-pixel distance to the target pixel and the scale of the spatial filter is the maximum value or the minimum value. .

  The second image generation unit 13 is a scale space image having different scales of the spatial filter for the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation unit 11. A plurality of combinations are determined. Then, the second image generation unit 13 generates an addition image obtained by adding the scale space images included in the combination for each determined combination.

  The second extraction unit 14 determines a plurality of combinations including a predetermined number of the addition images for the plurality of addition images generated by the second image generation unit 13. Then, for each determined combination, the second extraction unit 14 extracts a predetermined pixel from a predetermined number of pixels located near the target pixel in the scale space image included in the combination. For example, the second extraction unit 14 extracts a pixel whose output value of the spatial filter calculated based on the inter-pixel distance from the target pixel and the scale of the spatial filter is the maximum value or the minimum value. .

  The pattern extraction unit 15 extracts each pixel extracted for the reference image by the first extraction unit 12 and the second extraction unit 14 and the input image by the first extraction unit 12 and the second extraction unit 14. A specific pattern is extracted from the input image based on the result of matching with each pixel.

  As described above, the image processing apparatus 10 extracts pixels that are feature points of the input image or the reference image not only from the scale space image but also from the addition image generated by adding the scale space images. For this reason, when the image processing apparatus 10 detects a specific pattern from the input image using the reference image, more feature points can be used. As a result, even if the image is distorted, the input image is accurate. A specific pattern can be detected from

[Configuration of Image Processing Apparatus (Example 2)]
FIG. 2 is a diagram illustrating the configuration of the image processing apparatus according to the second embodiment. As shown in FIG. 2, the image processing apparatus 100 is connected to the photographing apparatus 200 in a state where image data can be acquired. The photographing apparatus 200 corresponds to, for example, an optical device such as a camera that has a lens and shoots an image that enters from the lens in the form of a photograph or a moving image.

  As shown in FIG. 2, the image processing apparatus 100 includes an image input unit 101, an input image recording unit 102, a template image recording unit 103, a processed image switching unit 104, and a scale space image generating unit 105. Have.

  The image input unit 101 inputs image data of an image captured by the imaging device 200 from the imaging device 200. The image input unit 101 converts the input image data into digital data. The input image recording unit 102 stores the input image data converted into digital data by the image input unit 101 in an internal storage device. The template image recording unit 103 stores, in advance, image data of a pattern used for detecting a specific object from an image photographed by the photographing apparatus 200 in an internal storage device. The processed image switching unit 104 switches the processing target by the scale space image generating unit 105 to the input image data included in the input image recording unit 102 or the template image included in the template image recording unit 103.

  Note that the storage device included in the input image recording unit 102 and the template image recording unit 103 is, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory.

  The scale space image generation unit 105 generates a scale space image from the input image data input from the processed image switching unit 104 or the template image data. For example, the scale space image generation unit 105 generates a scale space image sequence by applying a plurality of Laplacian and Gaussian filters having different scales “σ” to the input image data or the template image data. Hereinafter, the Laplacian / Gaussian filter is referred to as a “filter”. Note that the scale space image generation unit 105 corresponds to the first image generation unit 11 illustrated in FIG. 1 described above.

  As illustrated in FIG. 2, the image processing apparatus 100 includes a scale space local peak extraction unit 106 and a scale corrected image generation unit 107. Note that the scale space local peak extraction unit 106 corresponds to the first extraction unit 12 illustrated in FIG. 1 described above.

  The scale space local peak extraction unit 106 is a pixel in which the response of the filter locally peaks from three scale space images in which the scale of the filter is adjacent in the scale space image example generated by the scale space image generation unit 105. Is output. Note that the scales are adjacent means that the filter scale “σ” when the scale space image example is generated is adjacent, such as 1.0, 1.5, and 2.0. The pixel having a local peak is a pixel whose output value obtained by the filter is maximum or minimum between the target pixel and a predetermined number of pixels located in the vicinity of the target pixel.

  The scale correction image generation unit 107 generates a scale correction image centered on the pixel extracted by the scale space local peak extraction unit 106 for the input image and the template image. For example, the scale-corrected image generation unit 107 converts the input image and the template image into a specific scale image of a rectangular area having a specific size centered on the pixel extracted by the scale space local peak extraction unit 106. In this way, the scale correction image generation unit 107 generates a scale correction image. The scale correction image functions as a feature pattern in which features are extracted from the input image and the template image, and is used for matching processing of an image matching unit 119 described later. Note that the scale-corrected image has a scale of 16 × 16 pixels, for example.

  As shown in FIG. 2, the image processing apparatus 100 includes a first scale space image addition unit 108, a first scale space addition image local peak extraction unit 109, a first distortion direction calculation unit 110, and a first distortion. A correction image generation unit 111. Furthermore, the image processing apparatus 100 includes a second scale space image addition unit 112, a second scale space addition image local peak extraction unit 113, a second distortion direction calculation unit 114, and a second distortion correction image generation unit 115. Have.

  The first scale space image addition unit 108 and the second scale space image addition unit 112 generate a plurality of addition images for each scale space image sequence generated by the scale space image generation unit 105. The first scale space image addition unit 108 and the second scale space image addition unit 112 correspond to the second image generation unit 13 illustrated in FIG. 1 described above.

  For example, the first scale space image adding unit 108 generates a plurality of added images by adding scale space images of adjacent scales. First, the first scale space image adding unit 108 determines a plurality of combinations of scale space images having adjacent filter scales for the scale space image sequence of the input image and the scale space image sequence of the reference image. Then, the first scale space image adding unit 108 generates a plurality of added images by adding the scale space images included in the combinations for each determined combination.

  For example, the second scale space image addition unit 112 generates a plurality of addition images by adding scale space images of every other scale. First, the second scale space image addition unit 112 determines a plurality of combinations of scale space images with every other filter scale for the scale space image sequence of the input image and the scale space image sequence of the reference image. Then, the second scale space image adding unit 112 generates, for each determined combination, an added image obtained by adding the scale space images included in the combination.

  FIG. 3 is a conceptual diagram illustrating a state in which a Laplacian / Gaussian filter according to the second embodiment is applied to a distorted input image. 3-1 shown in FIG. 3 is a model of an input image representing an input image distorted depending on, for example, the viewing direction or the shooting direction. 3-2 and 3-3 shown in FIG. 3 are models of filters with adjacent scales or filters with every other scale. 3-4 shown in FIG. 3 is a conceptual model representing a state in which the filter 3-2 is applied to the input image 3-1. 3-5 shown in FIG. 3 is a conceptual model representing a state in which the filter 3-3 is applied to the input image 3-1.

  For example, an example of generating an addition image by the first scale space image addition unit 108 or the second scale space image addition unit 112 will be described using FIG. 3 described above. As shown in 3-4 and 3-5 of FIG. 3, the first scale space image adding unit 108 scales the scale space image sequence of the input image as one of the combinations of the scale space images in which the scales of the filters are adjacent. A combination of the space images 3-4 and 3-5 is determined. Then, the first scale space image adding unit 108 generates an added image by adding the scale space image 3-4 and the scale space image 3-5. Similarly, the second scale space image addition unit 112 determines a combination of the scale space images 3-4 and 3-5 as one of the combinations of the scale space images with every other filter scale. Then, the second scale space image addition unit 112 generates an addition image by adding the scale space image 3-4 and the scale space image 3-5.

  The first scale space image adding unit 108 or the second scale space image adding unit 112 adds the scale space image with the filter scale adjacent to each other or the scale space image with every other filter scale to add the image. It is not limited to the case of generating. For example, the addition image may be generated by adding every third or fourth scale scale space image of the filter. That is, the image processing apparatus 100 may generate a plurality of added images by adding scale space images having different filter scales as much as possible.

  FIG. 4 is a conceptual diagram including a three-dimensional model illustrating a state in which a Laplacian / Gaussian filter according to the second embodiment is applied to a distorted input image. 4-1 shown in FIG. 4 represents a planar model when the response result of the filter corresponding to 3-4 in FIG. 3 is viewed from the vertical direction with respect to the input image, for example. 4-2 shown in FIG. 4 represents a three-dimensional model when the response result of the filter corresponding to 3-4 of FIG. 3 is seen from a predetermined viewpoint, for example. 4-3 illustrated in FIG. 4 represents, for example, a planar model when the response result of the filter corresponding to 3-5 in FIG. 3 is viewed from the vertical direction with respect to the input image. 4-4 shown in FIG. 4 represents, for example, a three-dimensional model when the response result of the filter corresponding to 3-5 in FIG. 3 is viewed from a predetermined viewpoint. 4-5 shown in FIG. 4 represents a plane model when the response result of the added image generated by adding 3-4 and 3-5 in FIG. 3 is viewed from the vertical direction with respect to the added image. . 4-6 shown in FIG. 4 represents a three-dimensional model when the response result of the added image generated by adding 3-4 and 3-5 in FIG. 3 is viewed from a predetermined viewpoint.

  For example, as shown in 4-2 and 4-4 in FIG. 4, a single peak is obtained from an addition image generated by adding a scale space image that is a response result of a filter having two peaks. The response result of the filter having is obtained.

  The first scale space addition image local peak extraction unit 109 extracts a pixel whose filter response is a local peak for each combination of the addition images generated by the first scale space image addition unit 108. First, the first scale space addition image local peak extraction unit 109 determines a plurality of combinations including a predetermined number of addition images for the plurality of addition images generated by the first scale space image addition unit 108. Then, the first scale space addition image local peak extraction unit 109 extracts pixels for which the filter response is a local peak for each determined combination. Specifically, the first scale space addition image local peak extraction unit 109 outputs from the filter to the target pixel from among a predetermined number of pixels located in the vicinity of the target pixel in the scale space image included in the combination. The pixel whose output value is the maximum value or the minimum value is extracted.

  Similarly, the second scale space addition image local peak extraction unit 113 extracts a pixel whose filter response is a local peak for each combination of the addition images generated by the second scale space image addition unit 112. The operation of the second scale space addition image local peak extraction unit 113 is the same as that of the first scale space addition image local peak extraction unit 109 described above. The first scale space addition image local peak extraction unit 109 and the second scale space addition image local peak extraction unit 113 correspond to the second extraction unit 14 shown in FIG. 1 described above.

  The operation of the image processing apparatus 100 described above will be generally described with reference to FIG. Specifically, the scale space local peak extraction unit 106, the first scale space image addition unit 108, the first scale space addition image local peak extraction unit 109, the second scale space image addition unit 112, the second scale space addition image local This is an operation by the peak extraction unit 113. FIG. 5 is a diagram illustrating an example of local peak extraction from an input image using a Laplacian / Gaussian filter according to the second embodiment.

  5-1 shown in FIG. 5 represents input image data. 5-2 to 5-6 shown in FIG. 5 represent scale space image sequences generated by applying Laplacian and Gaussian filters having different scales to the input image data 5-1. Note that the scale of the filter is one scale larger in the order of 5-2 to 5-6 shown in FIG. 5-7 to 5-9 illustrated in FIG. 5 represent pixels extracted by the scale space local peak extraction unit 106. 5-10 to 5-13 illustrated in FIG. 5 represent addition images generated by the first scale space image addition unit 108. 5-14 and 5-15 shown in FIG. 5 represent pixels extracted by the first scale space addition image local peak extraction unit 109. 5-16 to 5-18 illustrated in FIG. 5 represent addition images generated by the second scale space image addition unit 112. 5-19 shown in FIG. 5 represents a pixel extracted by the second scale space addition image local peak extraction unit 113.

  The scale space local peak extraction unit 106 outputs pixels whose filter response is locally peaked from three scale space images with adjacent scales. That is, as illustrated in FIG. 5, the scale space local peak extraction unit 106 determines that the “LoG output value” obtained by the filter is locally obtained from the three scale space images 5-2 to 5-4 in which the filter scale is adjacent. Pixels 5-7 having a peak are extracted. Further, as shown in FIG. 5, the scale space local peak extraction unit 106 obtains a response obtained by a filter from three scale space images 5-3 to 5-5 having adjacent filter scales, that is, a “LoG output value”. Extract a pixel 5-8 having a local peak. Further, as shown in FIG. 5, the scale space local peak extraction unit 106 obtains a response obtained by a filter from three scale space images 5-4 to 5-6 having adjacent filter scales, that is, a “LoG output value”. Extract a pixel 5-8 having a local peak.

  The first scale space image adding unit 108 generates a plurality of added images by adding scale space images of adjacent scales. That is, as illustrated in FIG. 5, the first scale space image adding unit 108 selects, from the scale space image sequences 5-2 to 5-6, a combination of images with adjacent filter scales (5-2, 5-3). ), (5-3, 5-4), (5-4, 5-5), (5-5, 5-6). Then, as shown in FIG. 5, the first scale space image adding unit 108 adds the scale space images included in the combination, thereby adding the added images 5-10, 5-11, 5-12, and 5-13. Generate each.

  The first scale space addition image local peak extraction unit 109 extracts a pixel whose filter response is a local peak for each combination of the addition images generated by the first scale space image addition unit 108. That is, as illustrated in FIG. 5, the first scale space addition image local peak extraction unit 109 performs combinations (5-10, 5-11, 5-12) and (5-11) of addition images including three addition images. , 5-12, 5-13) respectively. Then, as shown in FIG. 5, the first scale space addition image local peak extraction unit 109 locally receives the response obtained by the filter from the combination of the addition images (5-10, 5-11, 5-12). The peak pixel 5-14 is extracted. Further, as shown in FIG. 5, the first scale space addition image local peak extraction unit 109 locally receives a response obtained by the filter from the combination (5-11, 5-12, 5-13) of the addition images. The peak pixel 5-15 is extracted.

  The second scale space image adding unit 112 generates a plurality of added images by adding the scale space images of every other scale. That is, as illustrated in FIG. 5, the second scale space image adding unit 112 selects, from the scale space image sequences 5-2 to 5-6, a combination of images in which filter scales are adjacent, (5-2, 5-4). ), (5-3, 5-5) and (5-4, 5-6). Then, as illustrated in FIG. 5, the second scale space image adding unit 112 generates the added images 5-16, 5-17, and 5-18 by adding the scale space images included in the combination.

  The second scale space addition image local peak extraction unit 113 extracts a pixel whose filter response is a local peak for each combination of the addition images generated by the second scale space image addition unit 112. That is, as illustrated in FIG. 5, the second scale space addition image local peak extraction unit 113 determines a combination (5-16, 5-17, 5-18) of addition images including three addition images. Then, as illustrated in FIG. 5, the second scale space addition image local peak extraction unit 113 receives a response obtained by the filter locally from the combination of the addition images (5-16, 5-17, 5-18). The peak pixel 5-19 is extracted.

  Thus, the scale space local peak extraction unit 106, the first scale space image addition unit 108, the first scale space addition image local peak extraction unit 109, the second scale space image addition unit 112, and the second scale space addition image local peak extraction The operation summary by the unit 113 is finished.

  The first distortion direction calculation unit 110 calculates the scale of the input image and the template image based on the response of the filter in the vicinity of the pixel extracted by the first scale space addition image local peak extraction unit 109, that is, the distribution of output values from the filter. Detect the direction of distortion for each. For example, the first distortion direction calculation unit 110 calculates the direction in which the spread of the response distribution is maximized, that is, the inertial principal axis direction, based on the response distribution of the filter near the pixel extracted from the above-described addition image. To calculate the direction of distortion of the input image or template image.

  FIG. 6 is a conceptual diagram for explaining the inertial spindle direction according to the second embodiment. 6-1 shown in FIG. 6 is a model of an input image representing an input image distorted depending on, for example, the viewing direction or the shooting direction. 6-2 and 6-3 shown in FIG. 6 are models of filters having different scales. 6-4 shown in FIG. 6 is a conceptual model representing a state in which the filter 6-2 is applied to the input image 6-1. 6-5 shown in FIG. 6 is a dotted line indicating the direction of the principal axis of inertia in 6-4 shown in FIG. 6-6 shown in FIG. 6 is a conceptual model representing a state in which the filter 6-3 is applied to the input image 6-1. 6-7 shown in FIG. 6 is a dotted line indicating the direction of the principal axis of inertia in 6-6 shown in FIG.

  FIG. 7 is a diagram illustrating a three-dimensional model when the response result of the filter corresponding to 6-4 in FIG. 6 is viewed from a predetermined viewpoint. FIG. 8 is a diagram illustrating a planar model when the response result of the filter corresponding to 6-4 in FIG. 6 is viewed from the vertical direction with respect to the input image. FIG. 9 is a diagram illustrating a three-dimensional model when the response result of the filter corresponding to 6-6 in FIG. 6 is viewed from a predetermined viewpoint. FIG. 10 is a diagram illustrating a planar model when the response result of the filter corresponding to 6-6 in FIG. 6 is viewed from the vertical direction with respect to the input image.

  Since the calculation method of the inertia principal axis direction by the first strain direction calculation unit 110 is the same for the input image and the template image, the calculation method for the inertia principal axis direction of the input image will be described below as an example. FIG. 11 is a diagram for explaining a calculation method in the inertial spindle direction according to the second embodiment. 11-1 shown in FIG. 11 represents an addition image generated by adding 6-4 and 6-6 shown in FIG. 6, for example. 11-2 shown in FIG. 11 represents the position of a pixel extracted from the addition image represented by 11-1 in FIG. 11 as the output value of the filter locally peaks. 11-3 shown in FIG. 11 represents the position of a pixel in the added image represented by 11-1 in FIG. A vertical axis “j” shown in FIG. 11 is a coordinate axis representing the vertical position of a pixel in the scale space image. “I” shown in FIG. 11 is a coordinate axis representing the horizontal position of the pixel in the scale space image. “Θ” shown in FIG. 11 represents the direction of the principal axis of inertia.

The first strain direction calculation unit 110 sets (i _G, j _G ) as the pixel position where the output value of the filter corresponding to 11-2 in FIG. 11 locally peaks, and corresponds to 11-3 in FIG. Let f (i, j) be the output value of the filter at the pixel to be processed. Then, the first strain direction calculation unit 110 substitutes (i _G, j _G ) and f (i, j) into the following expression (2), thereby adding an image corresponding to 11-1 in FIG. The inertial principal axis direction “θ” is calculated.

  Similarly to the first distortion direction calculation unit 110, the second distortion direction calculation unit 114 also uses the input image and the filter response in the vicinity of the pixel extracted by the second scale space addition image local peak extraction unit 113. The distortion direction for each scale of the template image is detected.

  The first distortion corrected image generation unit 111 corrects the distortion of the input image and the template image using the inertial principal axis direction calculated by the first distortion direction calculation unit 110. FIG. 12 is a diagram for explaining a distortion correction method for an input image and a template image according to the second embodiment. 12-1 shown in FIG. 12 represents an input image, for example. 12-2 shown in FIG. 12 represents a template image, for example.

  First, the distortion correction of the input image by the first distortion correction image generation unit 111 will be described. As shown in 12-3 of FIG. 12, the first distortion-corrected image generation unit 111 generates an input image having a specific scale centered on a pixel extracted by the first scale space addition image local peak extraction unit 109. One strain direction calculation unit 110 rotates in the strain direction calculated. The strain direction corresponds to the inertial axis direction described above. Next, the first distortion corrected image generation unit 111 corrects the scale of the input image after rotation indicated by 12-4 in FIG. 12 in the horizontal direction and the vertical direction as indicated by 12-5 in FIG. Thus, a corrected image is generated. For example, the first distortion corrected image generation unit 111 corrects in the horizontal direction and the vertical direction so as to have a scale of 16 × 16 pixels.

  The above-described corrected image for the input image functions as a feature pattern in which features are extracted from the input image, and is used for matching processing of an image matching unit 119 described later. The corrected image has a scale of 16 × 16 pixels, for example.

  Next, template image distortion correction by the first distortion correction image generation unit 111 will be described. First, as shown in 12-6 of FIG. 12, the first distortion-corrected image generation unit 111 generates a template image having a specific scale centered on the pixel extracted by the first scale space addition image local peak extraction unit 109. The first strain direction calculation unit 110 rotates in the strain direction. Next, the first distortion-corrected image generation unit 111 corrects the scale of the rotated template image indicated by 12-7 in FIG. 12 in the horizontal direction and the vertical direction as indicated by 12-8 in FIG. Thus, a corrected image is generated.

  The above-described correction image for the template image functions as a feature pattern in which features are extracted from the template image, and is used for matching processing of an image matching unit 119 described later. The corrected image has a scale of 16 × 16 pixels, for example.

  Note that the second distortion corrected image generation unit 115 also uses the inertial principal axis direction calculated by the second distortion direction calculation unit 114 in the same manner as the first distortion correction image generation unit 111 described above, to distort the input image and the template image. A corrected image is generated by correcting.

  As shown in FIG. 2, the image processing apparatus 100 includes a recorded image pattern switching unit 116, an input image feature pattern recording unit 117, a template image feature pattern recording unit 118, and an image matching unit 119.

  The recorded image pattern switching unit 116 switches the recording destination of the feature image pattern generated for the template image and the feature image pattern generated for the input image. For example, the recorded image pattern switching unit 116 records the input image whose characteristic is corrected by the scale correction image generation unit 107, the first distortion correction image generation unit 111, and the second distortion correction image generation unit 115. It operates so as to be recorded in the unit 117. Further, the recorded image pattern switching unit 116 records a template image whose distortion and scale have been corrected by the scale corrected image generating unit 107, the first distortion corrected image generating unit 111, and the second distortion corrected image generating unit 115 as template image feature pattern recording. The unit 118 operates so as to be recorded.

  The input image feature pattern recording unit 117 includes an internal storage device in which distortion and scale are corrected by the scale correction image generation unit 107, the first distortion correction image generation unit 111, and the second distortion correction image generation unit 115. Store the image. This corrected image functions as a feature image pattern of the input image.

  The template image feature pattern recording unit 118 includes an internal storage device in which distortion and scale are corrected by the scale correction image generation unit 107, the first distortion correction image generation unit 111, and the second distortion correction image generation unit 115. Store the image. This corrected image functions as a feature image pattern of the template image.

  The storage device included in the input image feature pattern recording unit 117 and the template image feature pattern recording unit 118 is, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory.

  The image matching unit 119 collates the feature image pattern of the input image that the input image feature pattern recording unit 117 has with the feature image pattern of the template image recorded in the template image feature pattern recording unit 118. Next, the image matching unit 119 determines whether the similarity between the feature image pattern of the input image and the feature image pattern of the template image exceeds a predetermined threshold. Then, when the degree of similarity exceeds a predetermined threshold as a result of the determination, the image matching unit 119 stores the set of the feature image pattern of the input image and the feature image pattern of the template image at that time in an internal storage device. save. The image matching unit 119 corresponds to the pattern extraction unit 15 illustrated in FIG. 1 described above.

  Each functional unit of the image processing apparatus 100 described above is, for example, an electronic circuit or an integrated circuit. Examples of the electronic circuit include a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). Examples of the integrated circuit include an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array).

[Processing by Image Processing Device (Example 2)]
FIG. 13 is a diagram illustrating a flow of processing performed by the image processing apparatus according to the second embodiment. Note that the processing by the image processing apparatus 100 shown in FIG. 13 is executed at a predetermined timing. As illustrated in FIG. 13, the image processing apparatus 100 generates a scale space image from the template image and extracts local peak pixels (step S1301). Step S1301 corresponds to the operation content by the scale space local peak extraction unit 106 described above.

  Next, the image processing apparatus 100 generates and records a scale correction feature pattern around the local peak pixel (step S1302). Step S1302 corresponds to a series of operations by the scale correction image generation unit 107, the input image feature pattern recording unit 117, the template image feature pattern recording unit 118, and the like described above.

  Subsequently, the image processing apparatus 100 generates an added image sequence from the scale space image and extracts local peak pixels (step S1303). Step S1303 corresponds to the operation contents of the first scale space addition image local peak extraction unit 109 and the second scale space addition image local peak extraction unit 113 described above.

  The image processing apparatus 100 generates and records a feature pattern in which distortion and scale around the local peak pixel are corrected (step S1304). Note that step S1304 corresponds to a series of operations by the first distortion correction image generation unit 111, the second distortion correction image generation unit 115, the input image feature pattern recording unit 117, the template image feature pattern recording unit 118, and the like.

  Next, the image processing apparatus 100 inputs and records an image from the photographing apparatus 200 (step S1305). Step S1305 corresponds to a series of operation contents by the image input unit 101 and the input image recording unit 102 described above.

  Subsequently, the image processing apparatus 100 generates a scale space image from the input image and extracts local peak pixels (step S1306). Step S1306 corresponds to the operation content of the scale space local peak extraction unit 106 described above.

  Then, the image processing apparatus 100 generates and records a scale correction feature pattern around the local peak pixel (step S1307). Step S1302 corresponds to a series of operations by the scale correction image generation unit 107, the input image feature pattern recording unit 117, the template image feature pattern recording unit 118, and the like described above.

  Next, the image processing apparatus 100 generates an added image sequence from the scale space image and extracts local peak pixels (step S1308). Step S1308 corresponds to the operation contents of the first scale space addition image local peak extraction unit 109 and the second scale space addition image local peak extraction unit 113 described above.

  Subsequently, the image processing apparatus 100 generates and records a feature pattern in which distortion and scale around the local peak pixel are corrected (step S1309). Note that step S1309 corresponds to a series of operations by the first distortion correction image generation unit 111, the second distortion correction image generation unit 115, the input image feature pattern recording unit 117, the template image feature pattern recording unit 118, and the like.

  The image processing apparatus 100 reads the feature pattern of the input image from the input image feature pattern recording unit 117 (step S1310). Then, the image processing unit 100 determines whether there is an unprocessed feature image pattern of the input image (step S1311). If there is an unprocessed pattern as a result of the determination (step S1311, YES), the image processing apparatus 100 reads the template image feature pattern from the template image feature pattern recording unit 118 (step S1312).

  Then, the image processing unit 100 determines whether there is an unprocessed feature pattern of the template image (step S1313). If there is an unprocessed pattern as a result of the determination (step S1313, YES), the image processing unit 100 determines whether the similarity between the feature pattern of the input image and the feature pattern of the template image exceeds a predetermined threshold value. Is determined (step S1314). As a result of the determination, if the similarity exceeds a predetermined threshold (step S1314, YES), the image processing unit 100 records a set of the feature pattern of the input image and the feature pattern of the template image at that time (step S1314). S1315).

  Here, the description returns to step S1311. If there is no unprocessed pattern as a result of the determination (step S1311, No), the image processing apparatus 100 ends the process as it is. Returning to the description of step S1313. If there is no unprocessed pattern as a result of the determination (step S1313, No), the image processing apparatus 100 returns to step S1310 described above and reads the next feature pattern of the input image. Also, the description returns to step S1314. If the degree of similarity exceeds a predetermined threshold as a result of the determination (step S1314, No), the image processing apparatus 100 returns to step S1312, and reads the feature pattern of the next template image. Steps S1310 to S1315 correspond to the operation content of the image matching unit 119 described above.

  Note that the image processing apparatus 100 does not necessarily perform processing in the order shown in FIG. For example, the image processing apparatus 100 may complete the processes in steps S1301 to S1304 described above before executing the processes in steps S1305 to S1315.

[Effects of Example 2]
As described above, the image processing apparatus 100 extracts pixels as feature points of the input image and the template image not only from the scale space image but also from the addition image generated by adding the scale space images. For this reason, according to the second embodiment, when a specific pattern is detected from an input image using a template image, matching can be performed using more feature points. As a result, the image is distorted. Even if it exists, a specific pattern can be accurately detected from the input image.

  The image processing apparatus 100 detects a specific pattern using a corrected image generated by correcting the distortion and scale of the input image and the template image. For this reason, according to the second embodiment, a specific pattern can be detected from an input image with higher accuracy even if the image is distorted.

  FIG. 14 is a diagram used for explaining the effect of the second embodiment. 14-1 shown in FIG. 14 represents a matching result between the feature points of the input image and the feature points of the template image in the related art. 14-2 illustrated in FIG. 14 represents a matching result between the feature points of the input image and the feature points of the template image by the image processing apparatus 100 according to the second embodiment. As illustrated in 14-2 of FIG. 14, the image processing apparatus 100 according to the second embodiment has a higher matching score between the feature point of the input image and the feature point of the template image than the conventional one. From the results shown in FIG. 14, it can be seen that according to the second embodiment, a specific pattern can be detected from an input image with high accuracy even if the image is distorted.

  Hereinafter, other embodiments of the image processing apparatus, the image processing method, and the image processing program disclosed in the present application will be described.

(1) Device Configuration, etc. For example, the configuration of the image processing apparatus 100 shown in FIG. 2 is functionally conceptual and does not necessarily need to be physically configured as illustrated. For example, the first scale space image adding unit 108 and the second scale space image adding unit 112 shown in FIG. 2 may be integrated functionally or physically. Further, for example, the first scale space addition image local peak extraction unit 109 and the second scale space addition image local peak extraction unit 113 illustrated in FIG. 2 may be integrated functionally or physically. Further, for example, the first strain direction calculation unit 110 and the second strain direction calculation unit 114 may be integrated functionally or physically. Further, for example, the first distortion correction image generation unit 111 and the second distortion correction image generation unit 115 may be integrated functionally or physically. As described above, all or a part of the image processing apparatus 100 can be configured to be functionally or physically distributed / integrated in arbitrary units according to various loads or usage conditions.

(2) Image Processing Method The image processing method described below is realized by the above-described embodiments. This image processing method is applied to the image processing apparatus 100 described above. The image processing method includes a first image generation step, a first extraction step, a second image generation step, a second extraction step, and a pattern detection step.

  The first image generation step generates a scale space image sequence of the input image and the image by applying a plurality of filters having different scales to the input image and the template image, respectively.

  In the first extraction step, for the scale space image sequence generated by the first image generation step, a plurality of combinations of scale space images with which the scale of the spatial filter is adjacent are determined from the scale space image sequence. Then, in the first extraction step, for each determined combination, a predetermined pixel is extracted from a predetermined number of pixels located in the vicinity of the target pixel in the scale space image included in the combination. That is, in the first extraction step, the output value of the filter calculated based on the inter-pixel distance to the target pixel and the filter scale from the predetermined number of pixels located in the vicinity of the target pixel is the maximum value or The pixel having the minimum value is extracted.

  In the second image generation step, for each scale space image sequence generated in the first image generation step, a combination of scale space images having different filter scales from the scale space images included in the scale space image sequence is obtained. Make multiple decisions. In the second image generation step, an added image obtained by adding the scale space images included in the combination is generated for each determined combination.

  In the second extraction step, a plurality of combinations including a predetermined number of addition images are determined for the plurality of addition images generated in the second image generation step. Then, in the second extraction step, for each determined combination, a predetermined pixel is extracted from a predetermined number of pixels located near the target pixel in the scale space image included in the combination. In other words, in the second extraction step, a pixel having a maximum or minimum output value of the filter calculated based on the inter-pixel distance from the target pixel and the filter scale is extracted.

  In the pattern detection step, each pixel extracted for the template image by the first extraction step and the second extraction step is compared with each pixel extracted for the input image by the first extraction step and the second extraction step. Based on the result, a specific pattern is detected from the input image.

(3) Image processing program In addition, for example, various processes executed by the image processing apparatus 100 described in the above-described embodiments are executed by a computer system such as a personal computer or a workstation. Can also be realized. For various processes of the image processing apparatus 100, see, for example, FIG.

  Therefore, in the following, an example of a computer that executes an image processing program that realizes the same function as the processing executed by the image processing apparatus 100 described in the above embodiment will be described with reference to FIG. FIG. 15 is a diagram illustrating an example of a computer that executes an image processing program.

  As shown in FIG. 15, a computer 300 functioning as the image processing apparatus 100 includes a CPU (Central Processing Unit) 310 that executes various arithmetic processes, and an input apparatus that receives input of image data from a camera or the like not shown in the figure. 320. Further, the computer 300 includes a RAM (Random Access Memory) 330 that temporarily stores various types of information and a hard disk device 340. Each device 310 to 340 is connected to a bus 350.

  Instead of the CPU 310, for example, an electronic circuit such as an MPU (Micro Processing Unit) or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array) can be used. Further, instead of the RAM 330, a semiconductor memory device such as a flash memory can be used.

  The hard disk device 340 stores an image processing program 341 and image processing data 342 that exhibit functions similar to the functions of the image processing apparatus 100 described above. Note that the image processing program 341 can be appropriately distributed and stored in a storage unit of another computer that is communicably connected via a network.

  Then, the CPU 310 reads the image processing program 341 from the hard disk device 340 and develops it in the RAM 330, whereby the image processing program 341 functions as an image processing process 331 as shown in FIG. The image processing process 331 expands the image processing data 342 and the like read from the hard disk device 340 in an area allocated to itself on the RAM 330 as appropriate, and executes various processes based on the expanded various data.

  Note that the image processing process 341 includes, for example, processing executed by the processing function unit of the image processing apparatus 100 illustrated in FIG. Note that the processing executed by the processing function unit of the image processing apparatus 100 corresponds to, for example, the processing shown in FIG. 13 described above in the second embodiment.

  Note that the image processing program 341 is not necessarily stored in the hard disk 340 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, and an IC card inserted into the computer 300. Then, the computer 300 may read and execute each program from these.

  Further, each program is stored in “another computer (or server)” connected to the computer 300 via a public line, the Internet, a LAN, a WAN, or the like. Then, the computer 300 may read and execute each program from these.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Supplementary Note 1) By applying a plurality of spatial filters having different scales to the input image and the reference image used when a specific pattern is extracted from the input image, the scale space of the input image and the reference image A first image generation unit for generating each image sequence;
Regarding the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation unit, the scale of the operated spatial filter is adjacent from the scale space image sequence. A plurality of scale space image combinations, and for each of the determined combinations, a predetermined number of pixels located in the vicinity of the target pixel in the scale space image included in the combination, and the target pixel. A first extraction unit that extracts a pixel having a maximum or minimum output value of the spatial filter calculated based on the inter-pixel distance and the scale of the spatial filter;
For the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation unit, the operated space from the scale space images included in the scale space image sequence A second image generation unit that determines a plurality of combinations of scale space images having different filter scales and generates an added image obtained by adding the scale space images included in the combination for each of the determined combinations;
For a plurality of addition images generated by the second image generation unit, a plurality of combinations including a predetermined number of the addition images are determined, and for each determined combination, a target pixel in a scale space image included in the combination Among the predetermined number of pixels located in the vicinity, a pixel whose output value of the spatial filter calculated based on the inter-pixel distance to the target pixel and the scale of the spatial filter is the maximum value or the minimum value is selected. A second extraction unit for extracting;
Each pixel extracted for the reference image by the first extraction unit and the second extraction unit, and each pixel extracted for the input image by the first extraction unit and the second extraction unit An image processing apparatus comprising: a pattern detection unit configured to detect a specific pattern from the input image based on a collation result.

(Additional remark 2) Based on distribution of the said output value for every said scale in the pixel vicinity extracted by the said 2nd extraction part, the detection part which detects the distortion direction for every said scale of the said input image and the said reference image, ,
The image processing apparatus according to claim 1, further comprising: a correction unit that corrects distortion of the input image and the reference image based on a distortion direction for each scale detected by the detection unit.

(Appendix 3) An image processing method applied to the image processing apparatus,
By applying a plurality of spatial filters having different scales to each of the input image and the reference image used when extracting a specific pattern from the input image, the scale space image sequence of the input image and the reference image respectively A first image generation step to generate;
Regarding the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation step, the scale of the applied spatial filter is adjacent from the scale space image sequence. A plurality of scale space image combinations, and for each of the determined combinations, a predetermined number of pixels located in the vicinity of the target pixel in the scale space image included in the combination, and the target pixel. A first extraction step of extracting a pixel whose output value of the spatial filter calculated based on the inter-pixel distance and the scale of the spatial filter is a maximum value or a minimum value;
For the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation step, the operated space from the scale space images included in the scale space image sequence A second image generation step of determining a plurality of combinations of scale space images having different filter scales and generating an added image obtained by adding the scale space images included in the combination for each determined combination;
For a plurality of addition images generated by the second image generation step, a plurality of combinations including a predetermined number of the addition images are determined, and for each determined combination, a target pixel in a scale space image included in the combination Among the predetermined number of pixels located in the vicinity, a pixel whose output value of the spatial filter calculated based on the inter-pixel distance to the target pixel and the scale of the spatial filter is the maximum value or the minimum value is selected. A second extraction step to extract;
Each pixel extracted for the reference image by the first extraction step and the second extraction step, and each pixel extracted for the input image by the first extraction step and the second extraction step And a pattern detection step of detecting a specific pattern from the input image based on a result of collation.

(Additional remark 4) The detection step which detects the distortion direction for every said scale of the said input image and the said reference image based on distribution of the said output value for every said scale in the pixel vicinity extracted by the said 2nd extraction step; ,
The image processing method according to claim 3, further comprising a correction step of correcting distortion of the input image and the reference image based on a distortion direction for each scale detected by the detection step.

(Supplementary Note 5) An image processing program to be executed by a computer as the image processing apparatus,
By applying a plurality of spatial filters having different scales to each of the input image and the reference image used when extracting a specific pattern from the input image, the scale space image sequence of the input image and the reference image respectively A first image generation procedure to generate;
Regarding the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation procedure, the scale of the applied spatial filter is adjacent from the scale space image sequence. A plurality of scale space image combinations, and for each of the determined combinations, a predetermined number of pixels located in the vicinity of the target pixel in the scale space image included in the combination, and the target pixel. A first extraction procedure for extracting a pixel whose output value of the spatial filter calculated based on the inter-pixel distance and the scale of the spatial filter is a maximum value or a minimum value;
For the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation procedure, the operated space from the scale space images included in the scale space image sequence A second image generation procedure for determining a plurality of combinations of scale space images having different filter scales, and generating, for each determined combination, an added image obtained by adding the scale space images included in the combination;
For a plurality of added images generated by the second image generation procedure, a plurality of combinations including a predetermined number of the added images are determined, and for each determined combination, a target pixel in a scale space image included in the combination Among the predetermined number of pixels located in the vicinity, a pixel whose output value of the spatial filter calculated based on the inter-pixel distance to the target pixel and the scale of the spatial filter is the maximum value or the minimum value is selected. A second extraction procedure to extract;
Each pixel extracted for the reference image by the first extraction procedure and the second extraction procedure, and each pixel extracted for the input image by the first extraction procedure and the second extraction procedure An image processing program that causes a computer to execute a pattern detection procedure for detecting a specific pattern from the input image based on a collation result.

(Additional remark 6) Based on the distribution of the said output value for every said scale in the pixel vicinity extracted by the said 2nd extraction procedure, the detection procedure which detects the distortion direction for every said scale of the said input image and the said reference image, ,
The image processing program according to claim 5, further comprising a correction procedure for correcting distortion of the input image and the reference image based on the distortion direction for each scale detected by the detection procedure.

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 11 1st image generation part 12 1st extraction part 13 2nd image generation part 14 2nd extraction part 15 Pattern extraction part 100 Image processing apparatus 101 Image input part 102 Input image recording part 103 Template image Recording unit 104 Processed image switching unit 105 Scale space image generation unit 106 Scale space local peak extraction unit 107 Scale correction image generation unit 108 First scale space image addition unit 109 First scale space addition image local peak extraction unit 110 First distortion direction Calculation unit 111 First distortion correction image generation unit 112 Second scale space image addition unit 113 Second scale space addition image local peak extraction unit 114 Second distortion direction calculation unit 115 Second distortion correction image generation unit 116 Recording image pattern switching unit 117 Input image feature pattern Recording unit 118 template image feature pattern recording unit 119 the image matching unit 200 imaging device 300 computer 310 CPU
320 Input device 330 RAM
331 Image processing process 340 Hard disk device 341 Image processing program 342 Data for image processing

Claims (4)

By applying a plurality of spatial filters having different scales to each of the input image and the reference image used when extracting a specific pattern from the input image, the scale space image sequence of the input image and the reference image respectively A first image generation unit for generating;
Regarding the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation unit, the scale of the operated spatial filter is adjacent from the scale space image sequence. A plurality of scale space image combinations, and for each of the determined combinations, a predetermined number of pixels located in the vicinity of the target pixel in the scale space image included in the combination, and the target pixel. A first extraction unit that extracts a pixel having a maximum or minimum output value of the spatial filter calculated based on the inter-pixel distance and the scale of the spatial filter;
For the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation unit, the operated space from the scale space images included in the scale space image sequence A second image generation unit that determines a plurality of combinations of scale space images having different filter scales and generates an added image obtained by adding the scale space images included in the combination for each of the determined combinations;
For a plurality of addition images generated by the second image generation unit, a plurality of combinations including a predetermined number of the addition images are determined, and for each determined combination, a target pixel in a scale space image included in the combination Among the predetermined number of pixels located in the vicinity, a pixel whose output value of the spatial filter calculated based on the inter-pixel distance to the target pixel and the scale of the spatial filter is the maximum value or the minimum value is selected. A second extraction unit for extracting;
Each pixel extracted for the reference image by the first extraction unit and the second extraction unit, and each pixel extracted for the input image by the first extraction unit and the second extraction unit An image processing apparatus comprising: a pattern detection unit configured to detect a specific pattern from the input image based on a collation result. A detection unit for detecting a distortion direction for each of the addition images of the input image and the reference image based on a distribution of the output values for each of the addition images in the vicinity of the pixel extracted by the second extraction unit;
The correction unit according to claim 1, further comprising: a correction unit that corrects distortion of the addition image corresponding to the input image and the reference image based on a distortion direction for each of the addition images detected by the detection unit. The image processing apparatus described. An image processing method applied to the image processing apparatus,
By applying a plurality of spatial filters having different scales to each of the input image and the reference image used when extracting a specific pattern from the input image, the scale space image sequence of the input image and the reference image respectively A first image generation step to generate;
Regarding the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation step, the scale of the applied spatial filter is adjacent from the scale space image sequence. A plurality of scale space image combinations, and for each of the determined combinations, a predetermined number of pixels located in the vicinity of the target pixel in the scale space image included in the combination, and the target pixel. A first extraction step of extracting a pixel whose output value of the spatial filter calculated based on the inter-pixel distance and the scale of the spatial filter is a maximum value or a minimum value;
For the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation step, the operated space from the scale space images included in the scale space image sequence A second image generation step of determining a plurality of combinations of scale space images having different filter scales and generating an added image obtained by adding the scale space images included in the combination for each determined combination;
For a plurality of addition images generated by the second image generation step, a plurality of combinations including a predetermined number of the addition images are determined, and for each determined combination, a target pixel in a scale space image included in the combination Among the predetermined number of pixels located in the vicinity, a pixel whose output value of the spatial filter calculated based on the inter-pixel distance to the target pixel and the scale of the spatial filter is the maximum value or the minimum value is selected. A second extraction step to extract;
Each pixel extracted for the reference image by the first extraction step and the second extraction step, and each pixel extracted for the input image by the first extraction step and the second extraction step And a pattern detection step of detecting a specific pattern from the input image based on a result of collation. An image processing program to be executed by a computer as the image processing apparatus,
By applying a plurality of spatial filters having different scales to each of the input image and the reference image used when extracting a specific pattern from the input image, the scale space image sequence of the input image and the reference image respectively A first image generation procedure to generate;
Regarding the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation procedure, the scale of the applied spatial filter is adjacent from the scale space image sequence. A plurality of scale space image combinations, and for each of the determined combinations, a predetermined number of pixels located in the vicinity of the target pixel in the scale space image included in the combination, and the target pixel. A first extraction procedure for extracting a pixel whose output value of the spatial filter calculated based on the inter-pixel distance and the scale of the spatial filter is a maximum value or a minimum value;
For the scale space image sequence of the input image and the scale space image sequence of the reference image generated by the first image generation procedure, the operated space from the scale space images included in the scale space image sequence A second image generation procedure for determining a plurality of combinations of scale space images having different filter scales, and generating, for each determined combination, an added image obtained by adding the scale space images included in the combination;
For a plurality of added images generated by the second image generation procedure, a plurality of combinations including a predetermined number of the added images are determined, and for each determined combination, a target pixel in a scale space image included in the combination Among the predetermined number of pixels located in the vicinity, a pixel whose output value of the spatial filter calculated based on the inter-pixel distance to the target pixel and the scale of the spatial filter is the maximum value or the minimum value is selected. A second extraction procedure to extract;
Each pixel extracted for the reference image by the first extraction procedure and the second extraction procedure, and each pixel extracted for the input image by the first extraction procedure and the second extraction procedure An image processing program that causes a computer to execute a pattern detection procedure for detecting a specific pattern from the input image based on a collation result.