WO2014030399A1 - 物体識別装置、物体識別方法、及びプログラム - Google Patents
物体識別装置、物体識別方法、及びプログラム Download PDFInfo
- Publication number
- WO2014030399A1 WO2014030399A1 PCT/JP2013/063982 JP2013063982W WO2014030399A1 WO 2014030399 A1 WO2014030399 A1 WO 2014030399A1 JP 2013063982 W JP2013063982 W JP 2013063982W WO 2014030399 A1 WO2014030399 A1 WO 2014030399A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- feature amount
- feature
- region
- unit
- input image
- Prior art date
Links
- 238000012850 discrimination method Methods 0.000 title 1
- 230000009466 transformation Effects 0.000 claims abstract description 29
- 238000000605 extraction Methods 0.000 claims description 114
- 238000006243 chemical reaction Methods 0.000 claims description 58
- 238000000034 method Methods 0.000 claims description 25
- 238000004458 analytical method Methods 0.000 abstract description 2
- 230000001131 transforming effect Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 63
- 238000012545 processing Methods 0.000 description 32
- 238000004364 calculation method Methods 0.000 description 24
- 230000010354 integration Effects 0.000 description 21
- 238000001514 detection method Methods 0.000 description 17
- 239000000284 extract Substances 0.000 description 8
- 238000003709 image segmentation Methods 0.000 description 8
- 238000012795 verification Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 239000000470 constituent Substances 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 239000004278 EU approved seasoning Substances 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 235000009508 confectionery Nutrition 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 235000011194 food seasoning agent Nutrition 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000004804 winding Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/10—Selection of transformation methods according to the characteristics of the input images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/759—Region-based matching
Definitions
- the present invention relates to an apparatus, method, and program for accurately identifying an object in an image.
- Patent Literature 1 and Non-Patent Literature 1 disclose local feature quantity extraction devices using SIFT (Scale Invariant Feature Transform) feature quantities.
- a local feature quantity extraction device first, only information relating to luminance is extracted from each pixel of an image, a large number of characteristic points (feature points) are detected from the extracted luminance information, and feature points that are information relating to each feature point Output information.
- the feature point information indicates, for example, the coordinate position and scale of the detected local feature point, the orientation of the feature point, and the like.
- a local region for extracting the feature amount is acquired from the feature point information such as the coordinate value, scale, and orientation of each detected feature point, and a local feature amount is generated (description).
- a local feature amount extracted from a captured image that is, an input image to identify an image in which the same subject as the subject in the captured image is shown. 1 is compared with a local feature amount 2 generated from a reference image. Specifically, first, for all combinations of the feature quantities for the plurality of feature points constituting the local feature quantity 1 and the feature quantities for the plurality of feature points constituting the local feature quantity 2, first, The distance calculation is performed to determine the closest feature quantity as the corresponding feature quantity. As for the corresponding feature amount, it is determined that the feature point that is the source of the feature amount generation is also supported.
- the coordinate position when the coordinate position of the feature point in the input image is moved according to a specific geometric transformation matches the coordinate position of the feature point in the reference image. Whether or not the corresponding feature point is correct is determined based on whether or not it is.
- the number of feature points determined to correspond correctly is equal to or greater than a certain value, it is determined that the same subject is captured (that is, the subject in the input image matches the subject in the reference image).
- the conventional object identification method using local feature amounts identifies an object based on the correspondence between the local feature amount extracted from the luminance information of the input image and the local feature amount extracted from the luminance information of the reference image. Is going.
- the object shown in the input image and the object shown in the reference image are different, but if there is only a slight difference between them, there are many corresponding feature points. Therefore, there is a problem that the images are mistakenly identified as being images of the same object.
- the present invention has been made in view of the above, and it is an object of the present invention to provide a technique for more accurately identifying an image in which the same object as that in the image is captured.
- the object identification device performs local feature amount collation for determining whether or not each feature amount of a feature point extracted from an input image and each feature amount of a feature point extracted from a reference image correspond correctly.
- the number of combinations of features determined to be correctly supported by the local feature amount extraction means, the input image divided region feature amount extraction means for extracting feature amounts from each divided region obtained by dividing the input image, and the local feature amount matching means When a score based on the reference image is equal to or greater than a predetermined value, an image region obtained by performing geometric transformation for correcting a geometric shift between the input image and the reference image on the predetermined region of the reference image.
- the input image divided region feature value converting means for acquiring the feature value of the divided region included in the region in the input image corresponding to the position, and the input image divided region feature value converting means. And the obtained feature amount, and compares the feature amount extracted from the predetermined area of the reference image, characterized by comprising a feature checker means for outputting a verification result.
- local feature amount collation for determining whether or not each feature amount of feature points extracted from an input image and each feature amount of feature points extracted from a reference image correspond correctly Step, an input image divided region feature amount extraction step for extracting a feature amount from each divided region obtained by dividing the input image, and a combination number of feature amounts determined to correspond correctly by the local feature amount matching step.
- a score based on the reference image is equal to or greater than a predetermined value, an image region obtained by performing geometric transformation for correcting a geometric shift between the input image and the reference image on the predetermined region of the reference image.
- An input image divided region feature amount converting step for obtaining a feature amount of the divided region included in the region in the input image corresponding to the position; and the input image divided region feature A feature quantity obtained by the conversion step, collates the feature amount extracted from the predetermined area of the reference image, characterized by comprising a feature checker step of outputting a collation result.
- the program according to the present invention allows a computer to determine whether or not each feature amount of a feature point extracted from an input image and each feature amount of a feature point extracted from a reference image correctly correspond to each other. Based on the number of combinations of feature quantities determined to be correctly supported by the collating means, the input image segmented area feature quantity extracting means for extracting feature quantities from each divided area obtained by dividing the input image, and the local feature quantity collating means The position of the image region obtained by performing geometric transformation for correcting a geometric shift between the input image and the reference image on the predetermined region of the reference image when the score is a predetermined value or more
- the input image divided region feature value converting means for acquiring the feature value of the divided region included in the region of the input image corresponding to the input image divided region feature value converting means A feature quantity obtained by collates the feature amount extracted from the predetermined area of the reference image, characterized in that function as the feature checker means for outputting a verification result.
- FIG. 5 is a flowchart illustrating an operation example of an input image different area determination unit 14; 3 is a block diagram illustrating a configuration example of a local feature quantity extraction unit 11.
- FIG. 3 is a block diagram illustrating a configuration example of a local feature amount matching unit 12.
- FIG. 4 is a block diagram illustrating a configuration example of an input image divided region feature amount extraction unit 13.
- FIG. It is the figure which showed the example of the production
- 3 is a block diagram illustrating a configuration example of an input image divided region feature value conversion unit 15.
- FIG. 3 is a block diagram illustrating a configuration example of a feature amount matching unit 16.
- FIG. 4 is a block diagram illustrating a configuration example of a local feature amount matching unit 17.
- FIG. 3 is a block diagram illustrating a configuration example of a different area estimation unit 18.
- FIG. 3 is a block diagram illustrating a configuration example of a different area estimation unit 18.
- FIG. 6 is a block diagram illustrating a configuration example of a different area feature amount extraction unit 19. It is a block diagram which shows the structural example of the object identification apparatus in 3rd Embodiment. 3 is a block diagram illustrating a configuration example of a different area estimation unit 20. FIG. It is a block diagram which shows the structural example of the object identification apparatus in 4th Embodiment. 3 is a block diagram illustrating a configuration example of a different area estimation unit 21. FIG. 3 is a block diagram illustrating a configuration example of a different area estimation unit 21. FIG. It is a block diagram which shows the structural example of the object identification apparatus in 5th Embodiment. 3 is a block diagram illustrating a configuration example of a different area estimation unit 22. FIG.
- FIG. 3 is a block diagram illustrating a configuration example of a different area estimation unit 22.
- FIG. 3 is a block diagram illustrating a configuration example of a different area estimation unit 22.
- FIG. It is a block diagram which shows the structural example of the object identification apparatus in 6th Embodiment.
- 4 is a block diagram illustrating a configuration example of a divided region feature amount extraction unit 23.
- FIG. 4 is a block diagram illustrating an example of a configuration of a divided region feature value conversion unit 24.
- FIG. It is a block diagram which shows the structural example of the object identification apparatus in 7th Embodiment. 4 is a block diagram illustrating a configuration example of an input image divided region feature value conversion unit 25.
- FIG. 4 is a block diagram illustrating a configuration example of a selected divided region feature amount extraction unit 26. It is the figure which showed the relationship between the input image selection division area in an input image, and the selection division area in a reference image. It is a block diagram which shows the structural example of the object identification apparatus in 8th Embodiment. 4 is a block diagram illustrating a configuration example of a local feature amount matching unit 27.
- FIG. 3 is a block diagram illustrating a configuration example of a feature amount matching unit 28.
- FIG. 5 is a block diagram illustrating a configuration example of an identification score integration determination unit 29.
- FIG. 1 is a block diagram showing a configuration of an object identification apparatus according to a first embodiment of the present invention.
- the object identification device includes a local feature amount extraction unit 11, a local feature amount collation unit 12, an input image divided region feature amount extraction unit 13, an input image different region determination unit 14, an input image divided region feature amount conversion unit 15, and a feature amount.
- a verification unit 16 is provided.
- the object identification device can be configured using an information processing device such as a personal computer or a portable information terminal, for example.
- the function of each part which comprises an object identification device is implement
- the local feature amount extraction unit 11 detects a feature point from the input image, and extracts the detected feature point and a feature amount of a local region that is a region in the vicinity thereof as a local feature amount. Details of the processing by the local feature quantity extraction unit 11 will be described later.
- the local feature amount collation unit 12 collates the local feature amount 1 extracted from the input image by the local feature amount extraction unit 11 with the local feature amount 2 extracted from the reference image, and specifies the corresponding local feature amount. . Details of the method for specifying the corresponding local feature will be described later with reference to FIG.
- the local feature amount matching unit 12 specifies the corresponding local region between the input image and the reference image according to the position when the local region corresponding to the local feature amount is geometrically transformed. For example, when the coordinate position of the local area when the local area in the input image is rotated by a predetermined angle about the center of the image matches the coordinate position of the corresponding local area in the reference area, A local region whose coordinate position matches in the input image and the reference image is specified as a corresponding local region.
- the geometric transformation is performed so that the geometric shift between the reference image and the input image is corrected.
- the local feature-value collation part 12 is the image ID of the reference image determined that the information (geometric transformation information) about the used geometric transformation corresponds to the local region when the corresponding local region is specified.
- the local feature identification image ID is output.
- the local feature quantity 2 extracted from the reference image may be extracted in advance from a plurality of reference images and stored in a database such as the local feature quantity DB shown in FIG. You may extract using the feature-value extraction part 11. FIG. When stored in the database, local feature amounts extracted from a reference image including similar objects (similar objects as subjects) may be registered in association with each other. Details of the local feature amount matching unit 12 will be described later.
- the input image divided region feature quantity extraction unit 13 receives the input image and outputs the input image divided region feature quantity and the input image divided region information based on the input image. Details of the processing by the input image divided region feature amount extraction unit 13 will be described later.
- the input image different region determination unit 14 performs local processing on the reference image corresponding to the local feature identification image ID output from the local feature amount matching unit 12 or the difference region of the reference image group associated with the local feature identification image ID.
- the geometric transformation indicated by the geometric transformation information output from the feature amount matching unit 12 is performed, and the input image different area information is output.
- the reference image difference area is a reference image when it is predicted that a slight difference may occur between an object shown in the input image and an object shown in the reference image. This is a region in which a portion where the difference can occur is copied.
- the difference area information of the reference image may be coordinate value information of four corners of the rectangle. Alternatively, it may be information indicating the coordinate value of the pixel group in the reference image constituting the different area.
- the input image different area information can be a coordinate value in the input image obtained by performing geometric transformation on each of the four corner coordinate values of the different area in the reference image.
- the information of the difference area of the reference image is the coordinate value information of the pixel group constituting the difference area
- the geometric conversion corresponding to the geometric conversion information is performed on each of the pixel groups, and the input image
- the coordinate value information of the pixel group constituting the different area can be used as the input image different area information.
- the difference area information of the reference image is stored in advance in the database.
- the difference area information of the reference image may be stored in the local feature amount DB together with the local feature amount 2.
- the input image divided region feature value conversion unit 15 includes the input image different region information output from the input image different region determination unit 14, the input image divided region feature value output from the input image divided region feature value extraction unit 13, and the input The image segmentation area information is received, and a feature quantity 1 that is a feature quantity extracted from a different area in the input image is output based on the received information. Details of the processing performed by the input image divided region feature value conversion unit 15 will be described later.
- the feature amount collation unit 16 collates the feature amount 1 output from the input image divided region feature amount conversion unit 15 with the feature amount 2 that is the feature amount extracted from the difference area in the reference image, and the collation result Is output.
- the feature amount matching unit 16 determines whether the object included in the input image is the same as the object included in the reference image (whether the input image and the reference image are the same object as the subject) in the matching. When it is determined that they are the same, the feature amount matching unit 16 outputs the image ID of the input image determined to be the same as the different area identification image ID.
- the feature amount 2 may be extracted from a plurality of reference images in advance and stored in a database, or may be extracted from a reference image on the fly. When stored in the database, similar objects may be registered in association with each other. Details of the feature amount matching unit 16 will be described later.
- FIG. 2 is a flowchart showing a flow of processing by the input image different area determination unit 14 shown in FIG. As shown in FIG. 2, first, a variable i for controlling the process is initialized in S141.
- the geometric transformation information output from the local feature amount matching unit 12 is acquired.
- the difference area information of the reference image is acquired from the local feature DB.
- the difference area information acquired here may be coordinate value information of the four corners of the rectangle, or the coordinate value of the pixel group in the reference image constituting the difference area It may be information indicating.
- the geometric transformation information acquired in S142 is performed on the difference area information acquired in S143.
- the different area information is the coordinate value information at the four corners
- geometric transformation is performed on one of the four coordinate values.
- the different area information is coordinate value information of a pixel group in a reference image constituting the different area
- geometric transformation is performed on one pixel of the pixel group. If the variable i is less than the predetermined number N at this time, the value of the variable i is updated in S145, and the processing of S143 and S144 is continued until the value of the variable i becomes N or more.
- the difference area information acquired in S143 is the coordinate value information in the reference images at the four corners
- the difference area information is the coordinate value information of the pixel group in the reference image constituting the difference area.
- the number of pixel groups in the reference image constituting the different area is set as a value of N.
- FIG. 3 is a block diagram illustrating a configuration example of the local feature quantity extraction unit 11.
- the local feature amount extraction unit 11 includes a luminance information extraction unit 101, a local feature point detection unit 102, and a local feature amount generation unit 103.
- the luminance information extraction unit 101 receives an input image, extracts only information relating to luminance from each pixel of the input image, and outputs it.
- the input image received here is an image taken by an imaging device such as a digital camera, a digital video camera, or a mobile phone, or an image captured through a scanner or the like.
- the image may be a compressed image such as JPEG (Joint Photographic Experts Group), or may be an uncompressed image such as TIFF (Tagged Image File Format).
- the local feature point detection unit 102 detects a large number of characteristic points (feature points) from the image, and outputs feature point information that is information about each feature point.
- the feature point information is, for example, the coordinate position and scale of the detected local feature point, the orientation of the feature point, the “feature point number” that is a unique ID (Identification) assigned to the feature point, etc. Is shown.
- the local feature point detection unit 102 may output the feature point information as separate feature point information for each orientation direction of each feature point. For example, the local feature point detection unit 102 may output feature point information only for the most main orientation direction at each feature point, and may also output feature point information for the second and subsequent main orientation directions. It is good.
- the local feature point detection unit 102 can assign different feature point numbers for each orientation direction at each feature point.
- the local feature point detection unit 102 can use, for example, DoG (Difference-of-Gaussian) processing when detecting feature points from an image and extracting feature point information.
- DoG Difference-of-Gaussian
- the local feature point detection unit 102 can determine the position and scale of the feature point by searching for an extreme value in the scale space using DoG processing.
- the local feature point detection unit 102 can calculate the orientation of each feature point using the determined position and scale of the feature point and the gradient information of the surrounding area.
- the local feature point detection unit 102 may use another method such as Fast-Hessian Detector instead of DoG when detecting feature points from an image and extracting feature point information.
- the local feature point detection unit 102 may select only important feature points from among the feature points detected therein, and output only information relating to the feature points as feature point information.
- the local feature amount generation unit 103 receives the feature point information output from the local feature point detection unit 102 and generates a local feature amount that is a feature amount of a local region (a feature point and its surrounding region) for each feature point ( Describe).
- generation part 103 may output a local feature-value in the format compressed by reversible compression, such as ZIP and LZH.
- the local feature point detection unit 102 determines the importance of the feature point to be detected
- the local feature amount generation unit 103 can generate and output the local feature amount in the order of importance of the feature point.
- the local feature value generation unit 103 may generate and output local feature values in the order of the coordinate positions of the feature points.
- the local feature quantity generation unit 103 acquires a local area for feature quantity extraction from the coordinate value, scale, and orientation of each detected feature point based on the feature point information.
- a local region can be acquired for each feature point information.
- the local region is rotated and normalized according to the orientation direction of the feature points, and then divided into sub-regions.
- the local area can be divided into 16 blocks (4 ⁇ 4 blocks).
- a feature vector is generated for each sub-region of the local region.
- a gradient direction histogram can be used as the feature vector of the sub-region.
- the gradient direction is calculated for each pixel in each sub-region, quantized in eight directions, and the frequency in the eight directions quantized for each sub-region is tabulated to generate a gradient direction histogram.
- a feature vector constituted by a gradient direction histogram of 16 blocks ⁇ 8 directions generated for each feature point is output as a local feature amount.
- the output local feature amount includes the coordinate position information of the feature point.
- FIG. 4 is a block diagram illustrating a configuration example of the local feature amount matching unit 12.
- the local feature amount matching unit 12 includes a corresponding feature point determination unit 201, an erroneous corresponding point removal unit 202, an identification score calculation unit 203, and a threshold determination unit 204.
- the corresponding feature point determination unit 201 receives the local feature amount 1 extracted from the input image by the local feature amount extraction unit 11 and the local feature amount 2 extracted from the reference image. The corresponding feature point determination unit 201 determines whether or not the local feature quantity 1 and the local feature quantity 2 correspond to each other, and in the case of correspondence, the local feature quantity 1 and the local feature quantity 2 are assumed to correspond to each other. Output point information. For example, when the local feature value 1 and the local feature value 2 are each a set of feature values describing gradient histograms around feature points, first, distance calculation in the feature value space is performed for all combinations of local feature values.
- the local feature and the local feature region of the local feature are related to the combination of the local feature having the minimum distance value.
- the position information of the local feature area corresponding to the position information of the local feature area is output as the corresponding feature point information.
- the miscorresponding point removal unit 202 receives the corresponding feature point information from the corresponding feature point determination unit 201, and discriminates the feature point erroneously corresponding to the correctly corresponding feature point from among the corresponding feature points. Thus, the determined feature point information is output, and the geometric transformation information used for the determination is also output. For example, by applying a technique such as RANSAC to the corresponding feature point information received from the corresponding feature point determination unit 201, a geometric transformation for moving the coordinates in the reference image to the coordinates in the input image is estimated, and the geometric transformation is performed. Information about is assumed to be geometric transformation information.
- the geometric transformation estimated here is applied to each feature point on the reference image side of the corresponding feature point, and if they almost coincide with the feature point on the input image side, it is determined that the feature point corresponds correctly. On the other hand, if it does not coincide with the feature point on the input side, it is determined that the feature point is erroneously associated.
- the identification score calculation unit 203 receives the corresponding feature point information from the erroneous corresponding point removal unit 202 and outputs an identification score.
- the identification score For the identification score to be output, for example, the number of combinations of feature points that correspond correctly is counted from the corresponding feature point information received from the incorrect corresponding point removal unit 202, and the number is mapped to a score between 0 and 1
- a table may be prepared in advance, and the identification score may be output with reference to the table.
- m / (c + m) may be calculated as an identification score, where m is the minimum number of feature points that is determined in advance.
- the threshold determination unit 204 performs threshold processing on the identification score output from the identification score calculation unit 203.
- the threshold determination unit 204 determines that the images are the same object, and identifies the ID of the reference image as a local feature identification. Output as image ID.
- the threshold set by the threshold determination unit 204 may be a value determined in advance and held inside, or a value given from the outside.
- FIG. 5 is a block diagram illustrating a configuration example of the input image divided region feature amount extraction unit 13.
- the input image divided region feature amount extraction unit 13 includes a divided region image generation unit 301 and a feature amount extraction unit 302.
- the divided region image generation unit 301 receives the input image and divides the input image according to a predetermined rule.
- the divided area image generation unit 301 outputs a divided area image that is an image of each divided area, and input image divided area information that is information including the coordinate information of each area and the area ID of each area. For example, when each region is rectangular, the coordinate information of each region can be coordinate information of four corners of each region.
- FIG. 6 is a diagram showing an example of a divided area generation pattern.
- the input image may be divided into regions of a uniform size in the vertical and horizontal directions or in the diagonal direction to generate divided regions.
- a divided region may be generated by dividing the region into non-uniformly sized regions.
- divided regions may be generated hierarchically using a plurality of divided region generation patterns.
- the feature amount extraction unit 302 extracts feature amounts from each of the divided region images generated by the divided region image generation unit 301, and outputs the extracted feature amounts as input image divided region feature amounts.
- feature amounts such as “color arrangement” and “color histogram” are used for analysis regarding color information of the difference region of the input image and the difference region of the reference image. It may be extracted. Alternatively, in order to analyze a fine character difference between the difference area of the input image and the difference area of the reference image, a feature amount capable of expressing “characteristic” may be extracted.
- FIG. 7 is a block diagram illustrating a configuration example of the input image divided region feature amount conversion unit 15. As illustrated in FIG. 7, the input image divided region feature value conversion unit 15 includes a different region divided region feature value integration unit 501.
- the within-different-region divided region feature amount integration unit 501 receives the input image different region information from the input image different region determination unit 14 and receives the input image divided region feature amount and the input image divided region information from the input image divided region feature amount extraction unit 13. Receive. Based on the received information, the different region divided region feature amount integration unit 501 outputs a feature amount 1 that is a feature amount in the different region of the input image.
- the in-different-area divided area feature amount integration unit 501 first specifies the divided areas included in the different area based on the input image different area information and the input image divided area information.
- the input image divided region feature values which are the feature values extracted from each of the identified divided regions, are integrated to determine the value of feature value 1 representing the feature value in the different region.
- the input image divided region feature amount output from the input image divided region feature amount extraction unit 13 is a “color histogram”
- a value obtained by adding and normalizing the color histograms for the divided regions included in the different regions. May be output as the feature amount 1.
- the weight for integrating the feature values may be increased, and for the divided areas that are only partially included in the different areas, the weight may be decreased.
- the weight may be set based on, for example, “a ratio of an area included in a different area in the divided area to an area of each divided area”.
- the input image divided region feature value output from the input image divided region feature value extraction unit 13 is a feature value extracted from divided regions generated hierarchically using a plurality of divided region generation patterns
- the input image divided region feature amount integration unit 501 the input image divided region is used by using only the pattern having the largest number of divided regions completely included in the different region of the input image among the hierarchical divided region generation patterns. You may integrate feature-value.
- FIG. 8 is a block diagram illustrating a configuration example of the feature amount matching unit 16.
- the feature amount matching unit 16 includes a different area identification score calculation unit 601 and a threshold determination unit 602.
- the different area identification score calculation unit 601 receives the feature quantity extracted from the different area of the input image as the feature quantity 1 and the feature quantity extracted from the different area of the reference image as the feature quantity 2, respectively.
- the different area identification score calculation unit 601 outputs an identification score determined from the received two feature quantities as a different area identification score.
- the different area identification score is a scale that increases as the feature quantity 1 and the feature quantity 2 are similar. For example, the distance between the feature quantity 1 and the feature quantity 2 in the feature quantity space may be calculated, and the reciprocal thereof may be output as the different area identification score.
- the minimum value of the distance on the feature amount space is found in all the combinations of the feature amounts, and the minimum value is The reciprocal of the value obtained by dividing the distance in the feature amount space in all combinations of feature amounts may be output as the different area identification score.
- a table for mapping the distance values in the feature amount space of the feature amount 1 and the feature amount 2 to a score between 0 and 1 is prepared in advance, and the difference area identification score is referenced with reference to the table. May be output.
- the threshold value determination unit 602 compares the different region identification score output from the different region identification score calculation unit 601 with a threshold value. If the difference region identification score is equal to or greater than the threshold value, the threshold value determination unit 602 determines that the image is the same object. The ID is output as a different area identification image ID.
- the threshold set by the threshold determination unit 602 may be a value determined in advance and held inside, or may be a value given from the outside.
- FIG. 9 shows the configuration of the object identification device in a modification of the present embodiment.
- the object identification apparatus shown in FIG. 9 is different from the object identification apparatus shown in FIG. 1 in that a different area information DB that is a database storing only different area information is provided. If the local feature amount 2 is not stored as a database and is extracted on the fly from the reference image, the present embodiment can be realized with the configuration of FIG.
- 10 to 13 are image diagrams showing patterns that can be considered as the relationship between the difference area in the reference image and the input image difference area in the input image, respectively.
- FIG. 10 shows an example in which an object is displayed on the entire reference image and a different area is set for the entire reference image.
- the characters and patterns engraved on the object are almost the same, but the color of the object is different. That is, in this example, since the entire input image may be different from the entire reference image, the entire reference image is set as the difference area.
- FIG. 11 shows an example in which an object is displayed on the entire reference image and a different area is set for a part of the reference image.
- the object is almost the same, but only a part of the object has a different color, character, or pattern. That is, in this example, since it is considered that a part of the input image may be different from a part of the reference image, a part of the reference image is set as the difference area.
- the examples shown in FIGS. 12 and 13 are substantially the same as the examples shown in FIGS. 10 and 11, respectively, except that the reference image is not shown in the whole image but only in a part thereof.
- the difference area in the registered reference image and the difference area in the input image are the minimum necessary areas in which a difference can be predicted from the reference image and the input image, respectively. Extracted. Therefore, when comparing the entire local feature amount of the reference image and the entire local feature amount of the input image, even if it is determined that they are the same image because the difference is slight, By comparing again only the feature values of the difference image between the reference image and the input image, it is possible to distinguish a fine difference caused by the difference in the article. As a result, it is possible to suppress misjudgment that has been a problem when only local feature values are used.
- FIG. 14 to 16 show an example in which the object identification device according to the present embodiment is configured as a server client system via a network.
- 14 is a configuration example on the client side
- FIG. 15 is a configuration example on the server side
- FIG. 16 is a block diagram showing an overall configuration example.
- the client side includes a local feature quantity extraction unit 11 and an input image divided region feature quantity extraction unit 13.
- the local feature quantity 1 output from the local feature quantity extraction unit 11 and the input image segmentation area feature quantity and input image segmentation area information output from the input image segmentation area feature quantity extraction unit 13 are transferred from the client side to the server side. Is transmitted.
- the server side includes a local feature amount matching unit 12, an input image different region determination unit 14, an input image divided region feature amount conversion unit 15, and a feature amount matching unit 16.
- a local feature amount matching unit 12 On the server side, an object shown in the input image is identified using the local feature value 1, the input image divided region feature value, and the input image divided region information transmitted from the client side.
- a server client system is configured with the object identification device of the present embodiment by connecting the configurations shown in FIGS. 14 and 15 through a network.
- this configuration it is not necessary to directly transmit the input image to the server side, and it is only necessary to transmit local feature amounts, divided region feature amounts, and divided region information that are lighter than the input image. Therefore, in this embodiment, compared with a configuration in which the client side transmits only an input image to the server side without performing processing, the server side performs identification processing, and the client side obtains the identification result. It is possible to shorten the time.
- the local feature amount matching unit 12 determines whether the feature amounts of the feature points extracted from the input image correspond to the feature amounts of the feature points extracted from the reference image. Determine.
- the input image divided region feature amount extraction unit 13 extracts a feature amount from each divided region obtained by dividing the input image.
- the input image segmented region feature amount conversion unit 15 performs geometrical processing between the input image and the reference image.
- the feature amount of the divided area included in the area in the input image corresponding to the position of the image area obtained by performing geometric transformation for correcting the misalignment on a predetermined area (difference area) of the reference image is acquired.
- the difference area of the reference image is determined by the input image difference area determination unit 14.
- the feature amount collation unit 16 collates the feature amount acquired by the input image divided region feature amount conversion unit 15 with the feature amount extracted from the difference area of the reference image, and outputs a collation result. As a result, it is possible to more accurately identify an input image in which the same object as that in the reference image is captured.
- FIG. 17 is a block diagram showing a configuration of the object identification device according to the second exemplary embodiment of the present invention.
- the object identification device according to the second exemplary embodiment includes a local feature amount extraction unit 11, a local feature amount collation unit 17, an input image divided region feature amount extraction unit 13, and an input image different region determination unit 14.
- the local feature amount matching unit 12 of the object identification device of the first embodiment is changed to the local feature amount matching unit 17 and stores the difference area information.
- the difference area information DB which is a database is different from the first embodiment in that a difference area estimation unit 18 and a difference area feature amount extraction unit 19 are changed. Details of the processing by the local feature amount matching unit 17, the different region estimation unit 18, and the different region feature amount extraction unit 19 will be described later. Since other components are the same as those in the first embodiment, the same reference numerals as those in the first embodiment are given and detailed description thereof is omitted.
- FIG. 18 is a block diagram illustrating a configuration example of the local feature amount matching unit 17.
- the local feature amount matching unit 17 includes a corresponding feature point determination unit 201, an erroneous corresponding point removal unit 202, an identification score calculation unit 203, and a threshold determination unit 204. That is, the constituent elements of the local feature amount matching unit 17 shown in FIG. 18 are the same as the constituent elements of the local feature amount matching unit 12 shown in FIG. However, the local feature amount matching unit 17 shown in FIG. 18 outputs the geometric conversion information from the erroneous corresponding point removing unit 202 and outputs the local feature identification image ID from the threshold determining unit 204, in addition to the erroneous corresponding point.
- the corresponding feature point information output from the removal unit 202 is different from the local feature amount matching unit 17 in FIG. 18 in that the corresponding feature point information is output from the local feature amount matching unit 17.
- FIG. 19 is a block diagram showing the configuration of the different area estimation unit 18.
- the different region estimation unit 18 includes an erroneous correspondence feature point density search unit 801.
- the miscorresponding feature point density search unit 801 receives the corresponding feature point information from the local feature amount matching unit 17 and outputs different area information that is information about the different area in the reference image.
- Corresponding feature point information received from the local feature amount matching unit 17 includes feature point information that corresponds correctly and feature point information that corresponds incorrectly. Therefore, by using the feature point information that is included in the corresponding feature point information, the reference point is searched by searching the reference image for a region where the feature points that correspond incorrectly are dense. It is possible to estimate the difference area in the image.
- a search for a region where feature points that correspond in error are densely defined is a feature that defines a rectangular window of a certain size, moves this rectangular window in the difference image, and corresponds incorrectly.
- the area of the rectangular window can be set as the difference area.
- the method of estimating the difference area in the reference image is not limited to this, and any method can be used as long as it is an estimation method based on an area where feature points corresponding to each other are erroneously gathered. . That is, the difference area (predetermined area) of the reference image is an area including an area where the local feature amount matching unit 17 determines that feature points corresponding to the reference image in error are dense.
- FIG. 20 is a block diagram illustrating a configuration example of the different area feature quantity extraction unit 19. As illustrated in FIG. 20, the different area feature quantity extraction unit 19 includes a different area image generation unit 901 and a different area feature quantity calculation unit 902.
- the different area image generation unit 901 receives the reference image from the reference image DB and receives the different area information from the different area estimation unit 18.
- the received difference area information is coordinate value information of the four corners of the difference area in the reference image
- the reference image is read by sequentially reading pixels located on a line connecting two adjacent corners with a straight line among the four corners. By determining the pixel from which the value is read and its order, a difference area image in the input image is generated and output.
- the different area information received from the different area estimation unit 18 is information representing the coordinate values of the pixel group constituting the different area in the reference image
- the different area image generation unit 901 reads the reference image in that order, Output as a difference area image in the input image.
- the different area feature quantity calculation unit 902 extracts a feature quantity from the different area image generated by the different area image generation unit 901 and outputs the feature quantity.
- a feature quantity such as “color arrangement” or “color histogram” may be extracted, or a feature quantity that can express “characteristic” is extracted. You may do it.
- the feature amount extracted here must be a feature amount extracted by the same process as the feature amount extraction unit 302 which is a component of the input image divided region feature amount extraction unit 13 shown in FIG.
- the difference area in the reference image can be estimated without registering the difference area in the reference image in advance in the database. , Effective when it is not possible to register in advance the area where a difference is expected to occur as a difference area (for example, to identify only a product with a scratch somewhere among many products) It is. Since the difference area in the reference image performed in the present embodiment can be estimated for the entire object or a part of the object, FIG. 10 to FIG. The present embodiment is effective for any of the examples shown in FIG.
- the present embodiment when configured as a server client system via a network, it is not necessary to transmit the input image directly to the server side as in the first embodiment, and it is lighter than the input image. Only the local feature amount, the divided region feature amount, and the divided region information need be transmitted. For this reason, in this embodiment, compared to a case where the client side does not perform processing but only transmits an image to the server side, identification processing is performed on the server side, and the identification result is obtained until the client side obtains the identification result. Time can be shortened.
- FIG. 21 is a block diagram showing a configuration of an object identification device according to the third exemplary embodiment of the present invention.
- the object identification device according to the third exemplary embodiment includes a local feature amount extraction unit 11, a local feature amount collation unit 17, an input image divided region feature amount extraction unit 13, and an input image different region determination unit 14.
- the difference area estimation unit 18 of the object identification device of the second embodiment is changed to the difference region estimation unit 20. Different from form. Details of the processing by the different area estimation unit 20 will be described later. Other components are the same as those in the second embodiment, and the same reference numerals are given and detailed description thereof is omitted.
- FIG. 22 is a block diagram illustrating a configuration example of the different area estimation unit 20.
- the different area estimation unit 20 includes an object area estimation unit 2001 and an erroneous correspondence feature point density search unit 2002.
- the object region estimation unit 2001 receives a reference image corresponding to the local feature identification image ID output from the local feature amount matching unit 17 or a reference image group associated with the local feature identification image ID, and includes an object in the reference image.
- the object area information which is information representing the area where is present, is output.
- the reference image received here may be stored in a database in advance as shown in FIG. 21, or may be acquired from outside the object identification device.
- the processing in the object region estimation unit 2001 includes, for example, a method of roughly estimating the object region by analyzing the edge strength in the reference image, or learning the image pattern of the background region in advance and using the object as a region other than the background. A method for roughly estimating the area is conceivable.
- the miscorresponding feature point density search unit 2002 is similar to the miscorresponding feature point density search unit 801, which is a component of the different area estimation unit 18 shown in FIG. However, the miscorresponding feature point density search unit 2002 has a second point that the object region information output from the object region estimation unit 2001 is input in addition to the corresponding feature point information received from the local feature amount matching unit 17. This is different from the embodiment.
- the miscorresponding feature point density search unit 2002 focuses on only the points that exist inside the object region among the corresponding feature points, and searches for regions where feature points that correspond incorrectly are dense.
- the miscorresponding feature point density search unit 2002 estimates a different area from the inside of the object area in the reference image. Therefore, in the present embodiment, it is possible to estimate the difference area in the reference image without being affected by an erroneous corresponding feature point appearing from an area other than the object. That is, the difference area (predetermined area) in the reference image is an area that is determined by the local feature amount matching unit 17 that feature points that correspond to each other in the reference image are densely collected. It becomes the area including.
- the miscorresponding feature point density search unit 2002 limits the range in which the feature points that are erroneously associated in the reference image are densely searched. Compared with the corresponding feature point density search unit 801, high-speed processing is possible.
- the object identification is used because the difference area in the reference image can be estimated even if the difference area in the reference image is not registered in the database in advance. This is effective when it is not possible to register information regarding the difference area in advance in the inspection system (for example, when identifying only a product having a scratch somewhere among many products).
- the estimation of the difference area in the reference image performed in the present embodiment is possible regardless of whether the difference area is the entire object or a part of the object. This is particularly effective in the case of the example shown in FIGS. 12 and 13 because it is possible to estimate the difference area with high accuracy without being affected by the erroneously corresponding feature points appearing from.
- this embodiment When this embodiment is configured as a server client system via a network, it is not necessary to transmit the input image directly to the server side as in the first and second embodiments. Only light local feature values, divided region feature values, and divided region information need be transmitted. For this reason, in this embodiment, compared to a case where the client side does not perform processing but only transmits an image to the server side, identification processing is performed on the server side, and the identification result is obtained until the client side obtains the identification result. Time can be shortened.
- FIG. 23 is a block diagram showing a configuration of the object identification device in the fourth exemplary embodiment of the present invention.
- the object identification device according to the fourth exemplary embodiment includes a local feature amount extraction unit 11, a local feature amount collation unit 12, an input image divided region feature amount extraction unit 13, and an input image different region determination unit 14.
- the local feature amount matching unit 17 and the different region estimation unit 20 of the object identification device according to the third embodiment include the local feature amount matching unit 12 and the difference region.
- the difference from the third embodiment is that the estimation unit 21 is changed.
- the local feature amount matching unit 12 is the same as the local feature amount matching unit 12 of the object identification device according to the first embodiment, and detailed description thereof is omitted. Details of the different area estimation unit 21 will be described later.
- Other components are the same as those in the third embodiment, and the same reference numerals are given and detailed descriptions thereof are omitted.
- the different region estimation unit 21 illustrated in FIG. 24 includes a template matching unit 2101.
- the template matching unit 2101 receives a reference image corresponding to the local feature identification image ID output from the local feature amount matching unit 12 (or a reference image group associated with the local feature identification image ID), and performs local feature amount matching.
- a template image of a different area corresponding to the local feature identification image ID output from the unit 12 (or a template image group associated with the local feature identification image ID) is also received, and the difference area information is obtained based on the received image group.
- this template image is an image pattern typically seen around the difference area.
- the template matching unit 2101 estimates each difference area in the reference image by matching each area in the reference image with this template image and performing a template matching process to search for an area most similar to the template image.
- an area in the reference image in which the similarity with the predetermined pattern image is greater than or equal to a predetermined value is set as a difference area in the reference image.
- the template image may be stored in advance in a database as shown in FIG. 23, or may be acquired from outside the object identification device.
- the different region estimation unit 21 may include an object region estimation unit 2001 and a template matching unit 2102 as shown in FIG.
- the object region estimation unit 2001 in FIG. 25 is the same as the object region estimation unit 2001 that is a component of the different region estimation unit 20 shown in FIG.
- the template matching unit 2102 is similar to the template matching unit 2101 that is a component of the different area estimation unit 21 shown in FIG.
- the template matching unit 2102 includes a reference image corresponding to the local feature identification image ID output from the local feature amount matching unit 12 and a template of a difference area corresponding to the local feature identification image ID output from the local feature amount matching unit 12.
- the object area information output from the object area estimation unit 2001 is input.
- the template matching unit 2102 can estimate the difference area in the reference image by performing template matching using the template image only for the object area in the reference image.
- the template matching unit 2102 is faster than the template matching unit 2101 in which the range of the region that matches the template image is the entire reference image because the range of the region in the reference image that matches the template image is limited. Is possible.
- the difference area in the reference image can be estimated by using the image pattern as a template image.
- the area where the address is described is a character string layout such as a postal code, address, and address.
- the difference area in the reference image performed in the present embodiment can be estimated regardless of whether the difference area is the whole object or a part of the object.
- FIG. 21 When the configuration of FIG. 21 is FIG. 25, it is possible to estimate the difference area again after removing the influence of the background by estimating the object area first, as in the third embodiment. It can be estimated with high accuracy and is particularly effective in the case of the example shown in FIGS.
- this embodiment is configured as a server client system via a network, it is not necessary to transmit the input image directly to the server side as in the first to third embodiments. Only light local feature values, divided region feature values, and divided region information need be transmitted. For this reason, in this embodiment, compared to a case where the client side does not perform processing but only transmits an image to the server side, identification processing is performed on the server side, and the identification result is obtained until the client side obtains the identification result. Time can be shortened.
- FIG. 26 is a block diagram illustrating a configuration of the object identification device according to the fifth exemplary embodiment of the present invention.
- the object identification device according to the fifth exemplary embodiment includes a local feature amount extraction unit 11, a local feature amount collation unit 17, an input image divided region feature amount extraction unit 13, and an input image different region determination unit 14.
- the object identification device of the fifth embodiment the object identification device of the second embodiment and the object identification device of the fourth embodiment are combined.
- the comparison with the object identification device of the second embodiment is different in that the different area estimation unit 18 is changed to a different area estimation unit 22. Details of the processing by the different area estimation unit 22 will be described later.
- Other components are the same as those in the second embodiment, and the same reference numerals are given and detailed description thereof is omitted.
- FIG. 27 to FIG. 29 are block diagrams showing configuration examples of the different area estimation unit 22, and each figure will be described below.
- the different area estimation unit 22 illustrated in FIG. 27 includes a miscorresponding feature point density search unit 2201 and a template matching unit 2202.
- the miscorresponding feature point density search unit 2201 in FIG. 27 is substantially the same as the miscorresponding feature point density search unit 801, which is a component of the different area estimation unit 18 shown in FIG. 19, but is not different area information.
- the difference is that the difference candidate area information is output.
- the difference candidate area information output from the miscorresponding feature point density search unit 2201 may be the same as the difference area information output from the miscorresponding feature point density search unit 801, or may be more than the difference area information. It can be regarded as a slightly expanded area and can be used as the area information.
- the template matching unit 2202 in FIG. 27 is similar to the template matching unit 2102 that is a component of the different region estimation unit 21 shown in FIG. 25, but the difference candidate region information is input instead of the object region information. Is different. That is, the template matching unit 2202 in FIG. 27 performs template matching using the template image only for the difference candidate region in the reference image estimated by the miscorresponding feature point density search unit 2201, and performs the difference region in the reference image. Is estimated.
- the difference area information output from the template matching unit 2202 is highly reliable because the template matching unit 2202 further narrows down the difference area from the difference candidate areas estimated by the miscorresponding feature point density search unit 2201. Difference area information is output.
- the template matching unit 2203 in FIG. 28 is substantially the same as the template matching unit 2101 that is a component of the different region estimation unit 21 shown in FIG. 24, except that it outputs difference candidate region information instead of difference region information. ing.
- the difference candidate area information output by the template matching unit 2203 may be the same as the difference area information output by the template matching unit 2101 or may be regarded as an area slightly wider than the difference area information. It may be information.
- the miscorresponding feature point density search unit 2204 in FIG. 28 is similar to the miscorresponding feature point density search unit 2002, which is a component of the different area estimation unit 20 shown in FIG.
- the difference is that difference candidate area information is input instead of.
- the difference area information output from the miscorresponding feature point density search unit 2204 is used to further narrow down the difference area from the difference candidate areas estimated by the template matching unit 2203 by the error corresponding feature point density search unit 2204.
- the difference area information with high reliability is output.
- the different area estimation unit 22 shown in FIG. 29 includes a miscorresponding feature point density search unit 2201, a template matching unit 2203, and a difference candidate area duplication detection unit 2205.
- the miscorresponding feature point density search unit 2201 in FIG. 29 is the same as the miscorresponding feature point density search unit 2201 that is a component of the different area estimation unit 22 shown in FIG. 27, and a detailed description thereof will be omitted.
- the template matching unit 2203 in FIG. 29 is the same as the template matching unit 2203 that is a component of the different area estimation unit 22 shown in FIG. 28, and a detailed description thereof will be omitted.
- the difference candidate region duplication detection unit 2205 receives the difference candidate region information output from the miscorresponding feature point density search unit 2201 and the difference candidate region information output from the template matching unit 2203, and receives these two difference candidates. An area where the areas overlap is determined as a difference area, and the difference area information is output.
- the difference area information output from the difference candidate area overlap detection unit 2205 is information on an area determined as a difference candidate area by the miscorresponding feature point density search unit 2201 and the template matching unit 2203. Highly different area information is output.
- the difference area can be estimated by using the image pattern as a template image. For example, when trying to identify only a specific mail from a plurality of postal images with the same envelope but differing only in the address, the area where the address is described is a character string layout such as a postal code, address, and address. Can be defined as an image pattern determined to some extent, which is effective.
- the difference area in the reference image performed in the present embodiment can be estimated whether the difference area is the whole object or a part of the object.
- FIG. 27 can also be regarded as a configuration in which the object identification device of the third embodiment and the object identification device of the fourth embodiment are combined. That is, when the different region estimation unit 22 has the configuration shown in FIGS. 27, 28, and 29, an object region estimation unit can be added before the mis-corresponding feature point density search unit 2201 and the template matching unit 2203. Since the difference area is estimated from the object area after the influence of the background is removed, the configuration is particularly effective in the case of FIGS.
- this embodiment is configured as a server client system via a network, it is not necessary to transmit the input image directly to the server side, as in the first to fourth embodiments. Only light local feature values, divided region feature values, and divided region information need be transmitted. For this reason, in this embodiment, compared to a case where the client side does not perform processing but only transmits an image to the server side, identification processing is performed on the server side, and the identification result is obtained until the client side obtains the identification result. Time can be shortened.
- FIG. 30 is a block diagram illustrating a configuration example of the object identification device according to the sixth exemplary embodiment of the present invention.
- the object identification device according to the sixth exemplary embodiment includes a local feature amount extraction unit 11, a local feature amount collation unit 17, an input image divided region feature amount extraction unit 13, and an input image different region determination unit 14.
- An input image divided region feature value conversion unit 15 a feature value matching unit 16, a different region estimation unit 18, a divided region feature value extraction unit 23, and a divided region feature value conversion unit 24.
- the different region feature value extraction unit 19 of the object identification device according to the second embodiment includes the divided region feature value extraction unit 23 and the divided region feature value conversion unit. The difference is that it is changed to 24. Details of the divided area feature quantity extraction unit 23 and the divided area feature quantity conversion unit 24 will be described later. Since other components are the same as those in the second embodiment, the same reference numerals as those in the second embodiment are given and detailed description thereof is omitted.
- FIG. 31 is a block diagram illustrating a configuration example of the divided region feature amount extraction unit 23.
- the divided region feature amount extraction unit 23 includes a divided region image generation unit 2301 and a feature amount extraction unit 2302.
- the divided region image generation unit 2301 is substantially the same as the divided region image generation unit 301 that is a component of the input image divided region feature amount extraction unit 13 shown in FIG.
- the divided region image generation unit 2301 is different from the divided region image generation unit 301 in that it receives a reference image instead of an input image and that it outputs divided region information instead of input image divided region information.
- the feature quantity extraction unit 2302 is also substantially the same as the feature quantity extraction unit 302 that is a component of the input image divided region feature quantity extraction unit 13 shown in FIG.
- the feature quantity extraction unit 2302 is different from the feature quantity extraction unit 302 in that it outputs a segment area feature quantity instead of the input image segment area feature quantity.
- FIG. 32 is a block diagram illustrating a configuration example of the divided region feature amount conversion unit 24. As illustrated in FIG. 32, the divided region feature value conversion unit 24 includes a different region divided region feature value integration unit 2401.
- the different region divided region feature value integration unit 2401 is substantially the same as the different region divided region feature value integration unit 501 which is a component of the input image divided region feature value conversion unit 15 shown in FIG.
- the different region divided region feature amount integration unit 2401 receives the different region information instead of the input image different region information, receives the divided region feature amount instead of the input image divided region feature, and receives the divided region information instead of the input image divided region information.
- the feature quantity 2 is not directly extracted from the region in the reference image indicated by the different region information as in the first to fifth embodiments, but from each region obtained by dividing the image once.
- the feature quantity is extracted, and the feature quantities of the divided areas included in the different areas are integrated to determine the value of the feature quantity 2.
- the configuration shown in FIG. 30 as a configuration example of the present embodiment is a configuration based on the second embodiment. Similarly, based on the configurations in the third to fifth embodiments, respectively. It is also possible to have a configuration as described above. That is, in the configuration examples of the third to fifth embodiments, the different area feature quantity extraction unit can be replaced with the divided area feature quantity extraction unit 23 and the divided area feature quantity conversion unit 24. . Also in the first embodiment, when the feature quantity 2 is extracted from a reference image on the fly rather than extracted from a plurality of reference images and stored in a database in advance as shown in FIG. A configuration including the area feature quantity extraction unit 23 and the divided area feature quantity conversion unit 24 is possible.
- this embodiment is configured as a server client system via a network, it is not necessary to transmit the input image directly to the server side as in the first to fifth embodiments. Only light local feature values, divided region feature values, and divided region information need be transmitted. For this reason, in this embodiment, compared to a case where the client side does not perform processing but only transmits an image to the server side, identification processing is performed on the server side, and the identification result is obtained until the client side obtains the identification result. Time can be shortened.
- FIG. 33 is a block diagram illustrating a configuration example of the object identification device according to the seventh exemplary embodiment of the present invention.
- the object identification device according to the seventh exemplary embodiment includes a local feature amount extraction unit 11, a local feature amount collation unit 17, an input image divided region feature amount extraction unit 13, and an input image different region determination unit 14.
- An input image divided region feature value conversion unit 25 a feature value matching unit 16, a different region estimation unit 18, and a selected divided region feature value extraction unit 26.
- the input image segmentation region feature value conversion unit 15 and the difference region feature value extraction unit 19 of the object identification device according to the second embodiment include the input image segmentation region.
- the difference is that the feature amount conversion unit 25 and the selected divided region feature amount extraction unit 26 are changed. Details of the input image divided region feature value conversion unit 25 and the selected divided region feature value extraction unit 26 will be described later.
- Other components are the same as those in the second embodiment, and the same reference numerals are given and detailed descriptions thereof are omitted.
- FIG. 34 is a block diagram illustrating a configuration example of the input image divided region feature value conversion unit 25. As shown in FIG. 34, the input image divided region feature value conversion unit 25 includes a divided region feature value selection unit 2501.
- the divided region feature value selection unit 2501 is similar to the different region divided region feature value integration unit 501 which is a component of the input image divided region feature value conversion unit 15 illustrated in FIG.
- the divided area feature quantity selection unit 2501 is different from the divided area feature quantity integration unit 501 in that the input image selection divided area information is output in addition to the feature quantity 1.
- the divided region feature value selection unit 2501 outputs information on divided regions that are completely within the different regions of the input image as input image selection divided region information, and at the same time, among the input image divided region feature values, Only the feature quantity of the divided area that is completely within the different area is selected and output as the feature quantity 1.
- This input image selection divided region information is information including coordinate information and region IDs at the four corners of the divided region that is completely within the different region of the input image.
- FIG. 35 is a block diagram illustrating a configuration example of the selected divided region feature amount extraction unit 26.
- the selected divided region feature amount extraction unit 26 includes a selected divided region determination unit 2601, a selected divided region image generation unit 2602, and a feature amount extraction unit 2603.
- the selected divided region determining unit 2601 performs geometric conversion indicated by the geometric conversion information received from the local feature amount matching unit 17 on the input image selected divided region information received from the input image divided region feature amount converting unit 25, and includes the reference image in the reference image.
- the selected divided region information indicating the information of the region corresponding to the input image selected divided region in the input image is output.
- the geometric conversion information used here must be information for converting from the coordinates in the input image to the coordinates in the reference image.
- FIG. 36 is a diagram showing a relationship between an input image selection divided area in the input image and a selected division area in the reference image.
- the selected divided region determination unit 2601 includes four divided regions that are completely within the difference region of the input image indicated by the input image selected divided region information received from the input image divided region feature amount conversion unit 25. Geometric transformation indicated by the geometric transformation information is performed on the corner coordinate values, and an area from which a feature amount is to be extracted in the reference image is determined as a selected divided area.
- the selected divided region image generation unit 2602 is similar to the different region image generation unit that is a component of the different region feature amount extraction unit 19 shown in FIG.
- the selected divided area image generation unit 2602 is different in that the selected divided area information is input instead of the different area information, and the selected divided area image is output instead of the different area image.
- the feature quantity extraction unit 2603 is similar to the different area feature quantity calculation unit 902 that is a component of the different area feature quantity extraction unit 19 shown in FIG.
- the feature amount extraction unit 2603 is different in that a selected divided region image is input instead of a different region image.
- the feature amount extracted by the feature amount extraction unit 2603 is similar to the different region feature amount calculation unit 902, and the feature amount extraction unit 302 that is a component of the input image divided region feature amount extraction unit 13 shown in FIG. It must be a feature quantity extracted by the same process.
- the feature amount 2 in the difference region of the reference image is not extracted independently from the input image, but is a divided region from which the feature amount 1 is generated.
- the information about is converted by the geometric conversion information, and a region for generating a feature amount in the reference image is determined, and then a feature amount 2 is generated from the region.
- FIG. 33 which has been described in order as a configuration example of the present embodiment, is a configuration based on the second embodiment, and similarly, based on the third to fifth embodiments.
- the different area feature quantity extraction unit is changed to the selected divided area feature quantity extraction unit, and the selected divided area information is output from the input image divided area feature quantity conversion unit.
- This can be configured by inputting the selected divided region information to the selected divided region feature amount extraction unit.
- the feature amount 2 is extracted from a reference image on the fly instead of being extracted from a plurality of reference images in advance and stored in a database as shown in FIG. It can be configured by including an image segmentation area feature amount conversion unit 25 and a selection segmentation area feature amount extraction unit 26.
- this embodiment is configured as a server client system via a network, it is not necessary to transmit the input image directly to the server side as in the first to sixth embodiments. Only light local feature values, divided region feature values, and divided region information need be transmitted. For this reason, in this embodiment, compared to a case where the client side does not perform processing but only transmits an image to the server side, identification processing is performed on the server side, and the identification result is obtained until the client side obtains the identification result. Time can be shortened.
- FIG. 37 is a block diagram illustrating a configuration example of the object identification device according to the eighth exemplary embodiment of the present invention.
- the object identification device according to the eighth embodiment includes a local feature quantity extraction unit 11, a local feature quantity verification unit 27, an input image divided region feature quantity extraction unit 13, and an input image different region determination unit 14. , An input image divided region feature value conversion unit 15, a feature value verification unit 28, and an identification score integrated determination unit 29.
- the local feature amount matching unit 12 and the feature amount matching unit 16 of the object identification device according to the first embodiment include the local feature amount matching unit 27 and the feature amount.
- the difference from the first embodiment is that the identification score integration determination unit 29 is added as a new component, being changed to the verification unit 28. Details of the local feature amount matching unit 27, the feature amount matching unit 28, and the identification score integrated determination unit 29 will be described later.
- Other components are the same as those in the first embodiment, and the same reference numerals are given and detailed description thereof is omitted.
- FIG. 38 is a block diagram illustrating a configuration example of the local feature amount matching unit 27.
- the local feature amount matching unit 27 includes a corresponding feature point determination unit 201, an erroneous corresponding point removal unit 202, an identification score calculation unit 203, and a threshold determination unit 2701.
- the corresponding feature point determination unit 201, the incorrect corresponding point removal unit 202, and the identification score calculation unit 203 in FIG. 38 are the corresponding feature point determination unit 201, the erroneous corresponding point, which are components of the local feature amount matching unit 12 illustrated in FIG. This is the same as the removal unit 202 and the identification score calculation unit 203, and detailed description thereof is omitted.
- the threshold determination unit 2701 in FIG. 38 is substantially the same as the threshold determination unit 204 that is a component of the local feature amount matching unit 12 illustrated in FIG. 4, but includes not only the local feature identification image ID but also the local feature identification image ID. The difference is that an identification score is output from a local feature extracted from a corresponding reference image (or a group of reference images associated therewith).
- the threshold value set by the threshold value determination unit 2701 may be made looser than the threshold value set by the threshold value determination unit 204 so that a large number of local feature identification image IDs and identification scores are output.
- FIG. 39 is a block diagram illustrating a configuration example of the feature amount matching unit 28.
- the feature amount matching unit 28 includes a different area identification score calculation unit 601 and a threshold determination unit 2801.
- the difference area identification score calculation unit 601 in FIG. 39 is the same as the difference area identification score calculation unit 601 that is a component of the feature amount matching unit 16 illustrated in FIG. 8, and detailed description thereof is omitted.
- the threshold determination unit 2801 in FIG. 39 is almost the same as the threshold determination unit 602 that is a component of the feature amount matching unit 16 shown in FIG. 8, but corresponds to not only the different region identification image ID but also the different region identification image ID.
- the difference is that a difference area identification score is output from a feature amount extracted from a difference area of a reference image or a reference image group associated therewith.
- the threshold value set by the threshold value determination unit 2801 may be set to be looser than the threshold value set by the threshold value determination unit 602 so that a large number of different region identification image IDs and different region identification scores are output.
- FIG. 40 is a block diagram illustrating a configuration example of the identification score integration determination unit 29. As illustrated in FIG. 40, the identification score integration determination unit 29 includes an identification score integration unit 2901 and a threshold determination unit 2902.
- the identification score integrating unit 2901 receives the identification score output from the local feature amount matching unit 27 and the different area identification score output from the feature amount matching unit 28, and calculates an integrated score based on the received score. Output. At this time, for example, a product of an identification score corresponding to the same image ID and a different area identification score may be obtained, and the value may be output as an integrated score.
- the threshold determination unit 2902 in FIG. 40 includes a threshold determination unit 204 that is a component of the local feature amount matching unit 12 illustrated in FIG. 4 and a threshold determination unit 602 that is a component of the feature amount verification unit 16 illustrated in FIG. Almost identical.
- the threshold value determination unit 2902 compares the integrated score output from the identification score integration unit 2901 with a predetermined threshold value.
- the image is determined to be an image having the same object as the subject, and the image ID of the input image is output as the identification image ID. If the image is less than the threshold, the input image and the reference image are the same object as the subject. It is determined that the image is not to be performed.
- the threshold determination unit 2902 is an image in which the input object and the reference image are the same object as the subject based on the result of matching by the local feature value matching unit 27 and the result of matching by the feature value matching unit 28. It is determined whether or not there is.
- the final discrimination result is not determined only by the different area discrimination score, but is finally determined from the score integrated with the discrimination score based on the local feature amount.
- the identification result is determined.
- images of the same object are captured in a bad environment (for example, a dark environment) and other similar objects are shot in an ideal environment, the similar objects have similar colors as well as textures. If this is the case, correct identification cannot be performed only with the feature amount extracted from the different area, but the identification result for the same object can be relatively increased by combining it with the identification result based on the local feature amount.
- the identification score is output from the local feature amount matching unit and the different area identification score is output from the feature amount matching unit, and these are output to the identification score integrated determination unit. It can be configured by inputting.
- this embodiment is configured as a server client system via a network, it is not necessary to transmit the input image directly to the server side as in the first to seventh embodiments. Only light local feature values, divided region feature values, and divided region information need be transmitted. For this reason, in this embodiment, compared to a case where the client side does not perform processing but only transmits an image to the server side, identification processing is performed on the server side, and the identification result is obtained until the client side obtains the identification result. Time can be shortened
- the local feature-value collation means which determines whether each feature-value of the feature point extracted from the input image and each feature-value of the feature point extracted from the reference image correspond correctly, Input image divided region feature amount extraction means for extracting a feature amount from each divided region obtained by dividing the input image; When a score based on the number of combinations of feature amounts determined to be correctly handled by the local feature amount matching unit is equal to or greater than a predetermined value, a geometrical deviation between the input image and the reference image is corrected.
- Input image divided region feature value conversion for acquiring the feature value of the divided region included in the region in the input image corresponding to the position of the image region obtained by performing geometric transformation on the predetermined region of the reference image Means,
- a feature amount collating unit that collates the feature amount acquired by the input image divided region feature amount converting unit with the feature amount extracted from the predetermined region of the reference image and outputs a collation result.
- the object identification device according to supplementary note 1, further comprising storage means for storing information about the predetermined region of the reference image.
- region of the said reference image is an area
- the predetermined region of the reference image includes a feature point in a reference image in which a feature amount is erroneously determined by the local feature amount matching unit among regions where an article is captured.
- the predetermined area of the reference image is determined in the reference image by the local feature amount matching unit to be erroneously associated with the feature amount, and the similarity to the predetermined pattern image
- the object identification device according to appendix 1, wherein the object identification region is a region including a region having a predetermined value or more.
- the division area feature-value extraction means which extracts a feature-value from each division area which divided
- a divided region feature amount converting means for acquiring a feature amount of the divided region included in the predetermined region of the reference image; Additional features 1 to 6 wherein the feature amount extracted from the predetermined region of the reference image used for collation by the feature amount collating unit is a feature amount acquired by the divided region feature amount converting unit.
- the selection division area feature-value extraction means which extracts the feature-value of the area
- the feature quantity extracted from the predetermined area of the reference image used for collation by the feature quantity collating means is a feature quantity extracted by the selected divided area feature quantity extracting means.
- the input image and the reference image are images having the same object as a subject.
- the object identification device according to any one of appendices 1 to 8, further comprising an integrated determination unit that determines whether or not there is.
- the local feature-value collation step which determines whether each feature-value of the feature point extracted from the input image and each feature-value of the feature point extracted from the reference image correspond correctly,
- An input image divided region feature amount extracting step for extracting a feature amount from each divided region obtained by dividing the input image; When the score based on the number of combinations of feature amounts determined to be correctly matched by the local feature amount matching step is equal to or greater than a predetermined value, a geometrical deviation between the input image and the reference image is corrected.
- Input image divided region feature value conversion for acquiring the feature value of the divided region included in the region in the input image corresponding to the position of the image region obtained by performing geometric transformation on the predetermined region of the reference image Steps, A feature amount collating step of collating the feature amount acquired by the input image divided region feature amount converting step with the feature amount extracted from the predetermined region of the reference image and outputting a collation result.
- An object identification method characterized by the above.
- Local feature amount matching means for determining whether or not each feature amount of the feature point extracted from the input image and each feature amount of the feature point extracted from the reference image correspond correctly,
- Input image divided region feature amount extraction means for extracting a feature amount from each divided region obtained by dividing the input image; When a score based on the number of combinations of feature amounts determined to be correctly handled by the local feature amount matching unit is equal to or greater than a predetermined value, a geometrical deviation between the input image and the reference image is corrected.
- Input image divided region feature value conversion for acquiring the feature value of the divided region included in the region in the input image corresponding to the position of the image region obtained by performing geometric transformation on the predetermined region of the reference image means, A function for collating the feature amount acquired by the input image divided region feature amount conversion unit with the feature amount extracted from the predetermined region of the reference image, and for functioning as a feature amount collation unit that outputs a collation result program.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
Description
本発明の第1の実施の形態について図面を参照して説明する。
図1は本発明の第1の実施の形態である物体識別装置の構成を示すブロック図である。物体識別装置は、局所特徴量抽出部11、局所特徴量照合部12、入力画像分割領域特徴量抽出部13、入力画像差異領域決定部14、入力画像分割領域特徴量変換部15、及び特徴量照合部16を備える。物体識別装置は、例えば、パーソナルコンピュータや携帯情報端末等の情報処理装置を用いて構成することができる。そして、物体識別装置を構成する各部の機能は、例えば、プロセッサが記憶部に格納されているプログラムをメモリに展開して実行することによって実現される。なお、後述する他の実施形態における構成要素についても同様に実現することができる。
本発明の第2の実施の形態について図面を参照して説明する。第2の実施の形態では、参照画像中の差異領域を事前にデータベース登録せずに、参照画像中の差異領域を推定して識別を行う。
本発明の第3の実施の形態について図面を参照して説明する。
図21は、本発明の第3の実施の形態における物体識別装置の構成を示すブロック図である。図21に示すように、第3の実施の形態の物体識別装置は、局所特徴量抽出部11、局所特徴量照合部17、入力画像分割領域特徴量抽出部13、入力画像差異領域決定部14、入力画像分割領域特徴量変換部15、特徴量照合部16、差異領域推定部20、及び差異領域特徴量抽出部19を備える。
本発明の第4の実施の形態について図面を参照して説明する。第4の実施の形態では、差異領域の推定方法として他の方法を用いる。
図24に示す差異領域推定部21は、テンプレートマッチング部2101を備える。テンプレートマッチング部2101は、局所特徴量照合部12から出力される局所特徴識別画像IDに対応する参照画像(あるいはその局所特徴識別画像IDに関連付いた参照画像群)を受け取ると共に、局所特徴量照合部12から出力される局所特徴識別画像IDに対応する差異領域のテンプレート画像(あるいはその局所特徴識別画像IDに関連付いたテンプレート画像群)も受け取り、受け取った画像群に基づいて、差異領域情報を出力する。具体的には、このテンプレート画像は、差異領域周辺で典型的に見られる画像パターンである。テンプレートマッチング部2101は、参照画像中の各領域とこのテンプレート画像とをマッチングし、テンプレート画像に最も類似する領域を探すテンプレートマッチング処理を行うことによって、参照画像中の差異領域を推定する。すなわち、参照画像中において所定のパターン画像との類似度が所定値以上である領域が、参照画像中の差異領域として設定される。テンプレート画像は、図23のように、あらかじめデータベースに記憶しておいても良いし、物体識別装置の外部から取得しても良い。
本発明の第5の実施の形態について図面を参照して説明する。
図26は、本発明の第5の実施の形態における物体識別装置の構成を示すブロック図である。図26に示すように、第5の実施の形態の物体識別装置は、局所特徴量抽出部11、局所特徴量照合部17、入力画像分割領域特徴量抽出部13、入力画像差異領域決定部14、入力画像分割領域特徴量変換部15、特徴量照合部16、差異領域推定部22、及び差異領域特徴量抽出部19を備える。
図27に示す差異領域推定部22は、誤対応特徴点密集度探索部2201、及びテンプレートマッチング部2202を備える。図27の誤対応特徴点密集度探索部2201は、図19に示した差異領域推定部18の構成要素である誤対応特徴点密集度探索部801とほぼ同一であるが、差異領域情報ではなく差異候補領域情報を出力する点が異なっている。この誤対応特徴点密集度探索部2201が出力する差異候補領域情報は、誤対応特徴点密集度探索部801で出力される差異領域情報と同一であっても良いし、その差異領域情報よりもわずかに広げた領域として捉え、その領域情報としても良い。
本発明の第6の実施の形態について図面を参照して説明する。
図30は、本発明の第6の実施の形態における物体識別装置の構成例を示すブロック図である。図30に示すように、第6の実施の形態の物体識別装置は、局所特徴量抽出部11、局所特徴量照合部17、入力画像分割領域特徴量抽出部13、入力画像差異領域決定部14、入力画像分割領域特徴量変換部15、特徴量照合部16、差異領域推定部18、分割領域特徴量抽出部23、及び分割領域特徴量変換部24を備える。
本発明の第7の実施の形態について図面を参照して説明する。
図33は、本発明の第7の実施の形態である物体識別装置の構成例を示すブロック図である。図33に示すように、第7の実施の形態の物体識別装置は、局所特徴量抽出部11、局所特徴量照合部17、入力画像分割領域特徴量抽出部13、入力画像差異領域決定部14、入力画像分割領域特徴量変換部25、特徴量照合部16、差異領域推定部18、及び選択分割領域特徴量抽出部26を備える。
本発明の第8の実施の形態について図面を参照して説明する。
図37は、本発明の第8の実施の形態における物体識別装置の構成例を示すブロック図である。図37に示すように、第8の実施の形態の物体識別装置は、局所特徴量抽出部11、局所特徴量照合部27、入力画像分割領域特徴量抽出部13、入力画像差異領域決定部14、入力画像分割領域特徴量変換部15、特徴量照合部28、及び識別スコア統合判定部29を備える。
前記入力画像を分割した各分割領域から特徴量を抽出する入力画像分割領域特徴量抽出手段と、
前記局所特徴量照合手段によって正しく対応していると判定された特徴量の組み合わせ数に基づくスコアが所定値以上である場合、前記入力画像と前記参照画像との間の幾何的なずれを補正する幾何変換を前記参照画像の所定の領域に対して行うことによって得られた画像領域の位置に対応する前記入力画像における領域に含まれる前記分割領域の特徴量を取得する入力画像分割領域特徴量変換手段と、
前記入力画像分割領域特徴量変換手段によって取得された特徴量と、前記参照画像の前記所定の領域から抽出された特徴量とを照合し、照合結果を出力する特徴量照合手段と
を備えたことを特徴とする物体識別装置。
前記参照画像の所定の領域に含まれる前記分割領域の特徴量を取得する分割領域特徴量変換手段と
を備え、
前記特徴量照合手段による照合に用いられる前記参照画像の前記所定の領域から抽出された特徴量は、前記分割領域特徴量変換手段により取得された特徴量であることを特徴とする付記1から6のいずれか1つに記載の物体識別装置。
前記特徴量照合手段による照合に用いられる前記参照画像の前記所定の領域から抽出された特徴量は、前記選択分割領域特徴量抽出手段により抽出された特徴量であることを特徴とする付記3から6のいずれか1項に記載の物体識別装置。
前記入力画像を分割した各分割領域から特徴量を抽出する入力画像分割領域特徴量抽出ステップと、
前記局所特徴量照合ステップによって正しく対応していると判定された特徴量の組み合わせ数に基づくスコアが所定値以上である場合、前記入力画像と前記参照画像との間の幾何的なずれを補正する幾何変換を前記参照画像の所定の領域に対して行うことによって得られた画像領域の位置に対応する前記入力画像における領域に含まれる前記分割領域の特徴量を取得する入力画像分割領域特徴量変換ステップと、
前記入力画像分割領域特徴量変換ステップによって取得された特徴量と、前記参照画像の前記所定の領域から抽出された特徴量とを照合し、照合結果を出力する特徴量照合ステップと
を備えたことを特徴とする物体識別方法。
入力画像から抽出した特徴点のそれぞれの特徴量と、参照画像から抽出した特徴点のそれぞれの特徴量とが正しく対応しているか否かを判定する局所特徴量照合手段、
前記入力画像を分割した各分割領域から特徴量を抽出する入力画像分割領域特徴量抽出手段、
前記局所特徴量照合手段によって正しく対応していると判定された特徴量の組み合わせ数に基づくスコアが所定値以上である場合、前記入力画像と前記参照画像との間の幾何的なずれを補正する幾何変換を前記参照画像の所定の領域に対して行うことによって得られた画像領域の位置に対応する前記入力画像における領域に含まれる前記分割領域の特徴量を取得する入力画像分割領域特徴量変換手段、
前記入力画像分割領域特徴量変換手段によって取得された特徴量と、前記参照画像の前記所定の領域から抽出された特徴量とを照合し、照合結果を出力する特徴量照合手段
として機能させるためのプログラム。
12 局所特徴量照合部
13 入力画像分割領域特徴量抽出部
14 入力画像差異領域決定部
15 入力画像分割領域特徴量変換部
16 特徴量照合部
17 局所特徴量照合部
18 差異領域推定部
19 差異領域特徴量抽出部
20、21、22 差異領域推定部
23、24 分割領域特徴量抽出部
25 入力画像分割領域特徴量変換部
26 選択分割領域特徴量抽出部
27 局所特徴量照合部
28 特徴量照合部
29 識別スコア統合判定部
101 輝度情報抽出部
102 局所特徴点検出部
103 局所特徴量生成部
201 対応特徴点決定部
202 誤対応点除去部
203 識別スコア算出部
204 閾値判定部
301 分割領域画像生成部
302 特徴量抽出部
501 差異領域内分割領域特徴量統合部
601 差異領域識別スコア算出部
602 閾値判定部
801 誤対応特徴点密集度探索部
901 差異領域画像生成部
902 差異領域特徴量算出部
2001 物体領域推定部
2002 誤対応特徴点密集度探索部
2101、2102 テンプレートマッチング部
2201 誤対応特徴点密集度探索部
2202、2203 テンプレートマッチング部
2204 誤対応特徴点密集度探索部
2205 差異候補領域重複検出部
2301 分割領域画像生成部
2302 特徴量抽出部
2401 差異領域内分割領域特徴量統合部
2501 分割領域特徴量選択部
2601 選択分割領域決定部
2602 選択分割領域画像生成部
2603 特徴量抽出部
2701、2801 閾値判定部
2901 識別スコア統合部
2902 閾値判定部
Claims (10)
- 入力画像から抽出した特徴点のそれぞれの特徴量と、参照画像から抽出した特徴点のそれぞれの特徴量とが正しく対応しているか否かを判定する局所特徴量照合手段と、
前記入力画像を分割した各分割領域から特徴量を抽出する入力画像分割領域特徴量抽出手段と、
前記局所特徴量照合手段によって正しく対応していると判定された特徴量の組み合わせ数に基づくスコアが所定値以上である場合、前記入力画像と前記参照画像との間の幾何的なずれを補正する幾何変換を前記参照画像の所定の領域に対して行うことによって得られた画像領域の位置に対応する前記入力画像における領域に含まれる前記分割領域の特徴量を取得する入力画像分割領域特徴量変換手段と、
前記入力画像分割領域特徴量変換手段によって取得された特徴量と、前記参照画像の前記所定の領域から抽出された特徴量とを照合し、照合結果を出力する特徴量照合手段と
を備えたことを特徴とする物体識別装置。 - 前記参照画像の前記所定の領域についての情報を記憶した記憶手段を備えることを特徴とする請求項1に記載の物体識別装置。
- 前記参照画像の前記所定の領域は、前記局所特徴量照合手段によって特徴量が誤って対応していると判定された参照画像における特徴点を含む領域であることを特徴とする請求項1に記載の物体識別装置。
- 前記参照画像の前記所定の領域は、物品が写った領域のうち、前記局所特徴量照合手段によって特徴量が誤って対応していると判定された参照画像における特徴点を含む領域であることを特徴とする請求項1に記載の物体識別装置。
- 前記参照画像の前記所定の領域は、前記参照画像中において所定のパターン画像との類似度が所定値以上である領域を含む領域であることを特徴とする請求項1に記載の物体識別装置。
- 前記参照画像の前記所定の領域は、前記参照画像中において、前記局所特徴量照合手段によって特徴量が誤って対応していると判定され、かつ、所定のパターン画像との類似度が所定値以上である領域を含む領域であることを特徴とする請求項1に記載の物体識別装置。
- 前記参照画像を分割した各分割領域から特徴量を抽出する分割領域特徴量抽出手段と、
前記参照画像の所定の領域に含まれる前記分割領域の特徴量を取得する分割領域特徴量変換手段と
を備え、
前記特徴量照合手段による照合に用いられる前記参照画像の前記所定の領域から抽出された特徴量は、前記分割領域特徴量変換手段により取得された特徴量であることを特徴とする請求項1から6のいずれか1項に記載の物体識別装置。 - 前記入力画像分割領域特徴量変換手段により特徴量が取得された前記入力画像における前記分割領域に対応する前記参照画像における領域の特徴量を抽出する選択分割領域特徴量抽出手段を備え、
前記特徴量照合手段による照合に用いられる前記参照画像の前記所定の領域から抽出された特徴量は、前記選択分割領域特徴量抽出手段により抽出された特徴量であることを特徴とする請求項3から6のいずれか1項に記載の物体識別装置。 - 入力画像から抽出した特徴点のそれぞれの特徴量と、参照画像から抽出した特徴点のそれぞれの特徴量とが正しく対応しているか否かを判定する局所特徴量照合ステップと、
前記入力画像を分割した各分割領域から特徴量を抽出する入力画像分割領域特徴量抽出ステップと、
前記局所特徴量照合ステップによって正しく対応していると判定された特徴量の組み合わせ数に基づくスコアが所定値以上である場合、前記入力画像と前記参照画像との間の幾何的なずれを補正する幾何変換を前記参照画像の所定の領域に対して行うことによって得られた画像領域の位置に対応する前記入力画像における領域に含まれる前記分割領域の特徴量を取得する入力画像分割領域特徴量変換ステップと、
前記入力画像分割領域特徴量変換ステップによって取得された特徴量と、前記参照画像の前記所定の領域から抽出された特徴量とを照合し、照合結果を出力する特徴量照合ステップと
を備えたことを特徴とする物体識別方法。 - コンピュータを、
入力画像から抽出した特徴点のそれぞれの特徴量と、参照画像から抽出した特徴点のそれぞれの特徴量とが正しく対応しているか否かを判定する局所特徴量照合手段、
前記入力画像を分割した各分割領域から特徴量を抽出する入力画像分割領域特徴量抽出手段、
前記局所特徴量照合手段によって正しく対応していると判定された特徴量の組み合わせ数に基づくスコアが所定値以上である場合、前記入力画像と前記参照画像との間の幾何的なずれを補正する幾何変換を前記参照画像の所定の領域に対して行うことによって得られた画像領域の位置に対応する前記入力画像における領域に含まれる前記分割領域の特徴量を取得する入力画像分割領域特徴量変換手段、
前記入力画像分割領域特徴量変換手段によって取得された特徴量と、前記参照画像の前記所定の領域から抽出された特徴量とを照合し、照合結果を出力する特徴量照合手段
として機能させるためのプログラム。
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/423,051 US20150213328A1 (en) | 2012-08-23 | 2013-05-20 | Object identification apparatus, object identification method, and program |
CN201380044205.3A CN104584073B (zh) | 2012-08-23 | 2013-05-20 | 物体识别装置和物体识别方法 |
JP2014531522A JP6143111B2 (ja) | 2012-08-23 | 2013-05-20 | 物体識別装置、物体識別方法、及びプログラム |
EP13831171.7A EP2889835A4 (en) | 2012-08-23 | 2013-05-20 | DEVICE, METHOD AND PROGRAM FOR DIFFERENTIATING AN OBJECT |
EP20159526.1A EP3693893A1 (en) | 2012-08-23 | 2013-05-20 | Object identification apparatus, object identification method, and program |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2012-184536 | 2012-08-23 | ||
JP2012184536 | 2012-08-23 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2014030399A1 true WO2014030399A1 (ja) | 2014-02-27 |
Family
ID=50149717
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2013/063982 WO2014030399A1 (ja) | 2012-08-23 | 2013-05-20 | 物体識別装置、物体識別方法、及びプログラム |
Country Status (6)
Country | Link |
---|---|
US (1) | US20150213328A1 (ja) |
EP (2) | EP2889835A4 (ja) |
JP (1) | JP6143111B2 (ja) |
CN (1) | CN104584073B (ja) |
TW (1) | TW201419169A (ja) |
WO (1) | WO2014030399A1 (ja) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2020101969A (ja) * | 2018-12-21 | 2020-07-02 | 株式会社モルフォ | 画像処理装置、画像処理方法およびプログラム |
JP2021114223A (ja) * | 2020-01-21 | 2021-08-05 | セコム株式会社 | 認識装置、学習装置、認識システム、認識方法、認識プログラム、学習方法、及び学習プログラム |
JP2021117565A (ja) * | 2020-01-23 | 2021-08-10 | セコム株式会社 | 認識装置、学習装置、認識システム、認識方法、及び認識プログラム |
Families Citing this family (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6168303B2 (ja) * | 2012-01-30 | 2017-07-26 | 日本電気株式会社 | 情報処理システム、情報処理方法、情報処理装置およびその制御方法と制御プログラム、通信端末およびその制御方法と制御プログラム |
WO2014030400A1 (ja) * | 2012-08-23 | 2014-02-27 | 日本電気株式会社 | 物体識別装置、物体識別方法、及びプログラム |
KR20150015680A (ko) * | 2013-08-01 | 2015-02-11 | 씨제이씨지브이 주식회사 | 특징점의 생성을 이용한 이미지 보정 방법 및 장치 |
US11615460B1 (en) | 2013-11-26 | 2023-03-28 | Amazon Technologies, Inc. | User path development |
WO2015197029A1 (zh) * | 2014-06-27 | 2015-12-30 | 北京奇虎科技有限公司 | 一种人脸相似度识别方法和系统 |
US10679177B1 (en) | 2015-03-25 | 2020-06-09 | Amazon Technologies, Inc. | Using depth sensing cameras positioned overhead to detect and track a movement of a user within a materials handling facility |
US10810539B1 (en) | 2015-03-25 | 2020-10-20 | Amazon Technologies, Inc. | Re-establishing tracking of a user within a materials handling facility |
US10586203B1 (en) | 2015-03-25 | 2020-03-10 | Amazon Technologies, Inc. | Segmenting a user pattern into descriptor regions for tracking and re-establishing tracking of a user within a materials handling facility |
US11205270B1 (en) * | 2015-03-25 | 2021-12-21 | Amazon Technologies, Inc. | Collecting user pattern descriptors for use in tracking a movement of a user within a materials handling facility |
JP6333871B2 (ja) * | 2016-02-25 | 2018-05-30 | ファナック株式会社 | 入力画像から検出した対象物を表示する画像処理装置 |
US20170323149A1 (en) * | 2016-05-05 | 2017-11-09 | International Business Machines Corporation | Rotation invariant object detection |
US10861184B1 (en) * | 2017-01-19 | 2020-12-08 | X Development Llc | Object pose neural network system |
US10242291B2 (en) * | 2017-02-08 | 2019-03-26 | Idemia Identity & Security | Device for processing images of people, the device seeking to sort these images as a function of contextual information |
US11328513B1 (en) | 2017-11-07 | 2022-05-10 | Amazon Technologies, Inc. | Agent re-verification and resolution using imaging |
US10846562B2 (en) * | 2018-01-12 | 2020-11-24 | Arizona Board Of Regents On Behalf Of Arizona State University | Systems and methods for image matching |
KR102484353B1 (ko) * | 2018-02-09 | 2023-01-04 | 한화정밀기계 주식회사 | 대형 부품 인식 방법 |
US11386306B1 (en) | 2018-12-13 | 2022-07-12 | Amazon Technologies, Inc. | Re-identification of agents using image analysis and machine learning |
JP7229795B2 (ja) * | 2019-02-01 | 2023-02-28 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | アノテーション装置、アノテーション方法、及び、プログラム |
US11270415B2 (en) * | 2019-08-22 | 2022-03-08 | Adobe Inc. | Image inpainting with geometric and photometric transformations |
US20220262263A1 (en) * | 2021-02-16 | 2022-08-18 | Flir Unmanned Aerial Systems Ulc | Unmanned aerial vehicle search and rescue systems and methods |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6711293B1 (en) | 1999-03-08 | 2004-03-23 | The University Of British Columbia | Method and apparatus for identifying scale invariant features in an image and use of same for locating an object in an image |
JP2010079545A (ja) | 2008-09-25 | 2010-04-08 | Canon Inc | 画像処理装置、画像処理方法およびプログラム |
JP2011113197A (ja) * | 2009-11-25 | 2011-06-09 | Kddi Corp | 画像検索方法およびシステム |
JP2012033022A (ja) * | 2010-07-30 | 2012-02-16 | Panasonic Corp | 空間における変化領域検出装置及び方法 |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4196302B2 (ja) * | 2006-06-19 | 2008-12-17 | ソニー株式会社 | 情報処理装置および方法、並びにプログラム |
CA2668941C (en) * | 2006-11-17 | 2015-12-29 | Thomson Licensing | System and method for model fitting and registration of objects for 2d-to-3d conversion |
WO2008111452A1 (ja) * | 2007-03-09 | 2008-09-18 | Omron Corporation | 認識処理方法およびこの方法を用いた画像処理装置 |
CN101493935B (zh) * | 2008-11-10 | 2012-05-09 | 西安电子科技大学 | 基于剪切波隐马尔可夫模型的合成孔径雷达图像分割方法 |
IT1394181B1 (it) * | 2009-05-07 | 2012-06-01 | Marchesini Group Spa | Metodo di segmentazione basato sulle caratteristiche per segmentare una pluralita' di articoli duplicati disposti alla rinfusa e gruppo che attua tale metodo per alimentare una macchina confezionatrice |
JP5430243B2 (ja) * | 2009-06-17 | 2014-02-26 | キヤノン株式会社 | 画像検索装置及びその制御方法並びにプログラム |
US8280196B2 (en) * | 2009-05-12 | 2012-10-02 | Canon Kabushiki Kaisha | Image retrieval apparatus, control method for the same, and storage medium |
US9019349B2 (en) * | 2009-07-31 | 2015-04-28 | Naturalpoint, Inc. | Automated collective camera calibration for motion capture |
CN102073878B (zh) * | 2010-11-15 | 2012-11-07 | 上海大学 | 非穿戴指势视觉识别方法 |
JP5922428B2 (ja) | 2011-02-16 | 2016-05-24 | 綜合警備保障株式会社 | 複写防止用紙 |
-
2013
- 2013-05-20 CN CN201380044205.3A patent/CN104584073B/zh active Active
- 2013-05-20 JP JP2014531522A patent/JP6143111B2/ja active Active
- 2013-05-20 EP EP13831171.7A patent/EP2889835A4/en not_active Ceased
- 2013-05-20 WO PCT/JP2013/063982 patent/WO2014030399A1/ja active Application Filing
- 2013-05-20 EP EP20159526.1A patent/EP3693893A1/en active Pending
- 2013-05-20 US US14/423,051 patent/US20150213328A1/en not_active Abandoned
- 2013-08-20 TW TW102129833A patent/TW201419169A/zh unknown
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6711293B1 (en) | 1999-03-08 | 2004-03-23 | The University Of British Columbia | Method and apparatus for identifying scale invariant features in an image and use of same for locating an object in an image |
JP2010079545A (ja) | 2008-09-25 | 2010-04-08 | Canon Inc | 画像処理装置、画像処理方法およびプログラム |
JP2011113197A (ja) * | 2009-11-25 | 2011-06-09 | Kddi Corp | 画像検索方法およびシステム |
JP2012033022A (ja) * | 2010-07-30 | 2012-02-16 | Panasonic Corp | 空間における変化領域検出装置及び方法 |
Non-Patent Citations (3)
Title |
---|
DAVID G. LOWE: "Distinctive image features from scale-invariant keypoints", USA, INTERNATIONAL JOURNAL OF COMPUTER VISION, vol. 60, no. 2, 2004, pages 91 - 110 |
KYOTA HIGA ET AL.: "Kyokusho Tokuchoryo no Tokuchoten Ichi ni Motozuku Clustering o Mochiita Hishatai Shikibetsu", FIT2012 DAI 11 KAI FORUM ON INFORMATION TECHNOLOGY KOEN RONBUNSHU, SEPARATE VOL.3, SADOKU TSUKI RONBUN? IPPAN RONBUN, GAZO NINSHIKI?MEDIA RIKAI, GRAPHICS?GAZO, HUMAN COMMUNICATION & INTERACTION, KYOIKU KOGAKU-FUKUSHI KOGAKU? MULTIMEDIA OYO, FORUM ON, vol. 3, 21 August 2012 (2012-08-21), pages 129 - 130, XP008176218 * |
See also references of EP2889835A4 |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2020101969A (ja) * | 2018-12-21 | 2020-07-02 | 株式会社モルフォ | 画像処理装置、画像処理方法およびプログラム |
JP2021114223A (ja) * | 2020-01-21 | 2021-08-05 | セコム株式会社 | 認識装置、学習装置、認識システム、認識方法、認識プログラム、学習方法、及び学習プログラム |
JP7441654B2 (ja) | 2020-01-21 | 2024-03-01 | セコム株式会社 | 認識装置、学習装置、認識システム、認識方法、認識プログラム、学習方法、及び学習プログラム |
JP2021117565A (ja) * | 2020-01-23 | 2021-08-10 | セコム株式会社 | 認識装置、学習装置、認識システム、認識方法、及び認識プログラム |
JP7441656B2 (ja) | 2020-01-23 | 2024-03-01 | セコム株式会社 | 認識装置、学習装置、認識システム、認識方法、及び認識プログラム |
Also Published As
Publication number | Publication date |
---|---|
JP6143111B2 (ja) | 2017-06-07 |
US20150213328A1 (en) | 2015-07-30 |
EP2889835A4 (en) | 2016-06-08 |
CN104584073B (zh) | 2017-04-05 |
EP2889835A1 (en) | 2015-07-01 |
EP3693893A1 (en) | 2020-08-12 |
TW201419169A (zh) | 2014-05-16 |
CN104584073A (zh) | 2015-04-29 |
JPWO2014030399A1 (ja) | 2016-07-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6143111B2 (ja) | 物体識別装置、物体識別方法、及びプログラム | |
JP6278276B2 (ja) | 物体識別装置、物体識別方法、及びプログラム | |
JP6354589B2 (ja) | 物体識別装置、方法及びプログラム | |
US8792715B2 (en) | System and method for forms classification by line-art alignment | |
KR101617681B1 (ko) | 히스토그램들을 갖는 다중 층 연결 요소들을 사용하는 텍스트 검출 | |
JP2020030857A (ja) | 画像処理装置及び画像処理方法 | |
US20070242882A1 (en) | Image processing apparatus for identifying the position of a process target within an image | |
JP2021057054A (ja) | 画像認識装置 | |
US20100158375A1 (en) | Signal processing apparatus, signal processing method, computer-readable medium and computer data signal | |
JP6810392B2 (ja) | 個体識別装置 | |
WO2017179728A1 (ja) | 画像認識装置、画像認識方法および画像認識プログラム | |
JP2007140729A (ja) | 物品の位置及び姿勢を検出する方法および装置 | |
JP5712415B2 (ja) | 帳票処理システム及び帳票処理方法 | |
JP5169648B2 (ja) | 原画像探索装置及び原画像探索プログラム | |
JP4221960B2 (ja) | 帳票識別装置及びその識別方法 | |
JP2019046184A (ja) | 画像処理装置、訓練画像処理装置、画像識別装置、訓練画像処理方法、画像識別方法、及びプログラム | |
JP6941331B2 (ja) | 画像認識システム | |
CN114565750A (zh) | 一种纸质试题的处理方法和系统 | |
JP2023028563A (ja) | 画像処理プログラム、画像処理装置、及び画像処理方法 | |
JP2005339398A (ja) | 特徴抽出装置、特徴抽出方法、および、プログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 13831171 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2013831171 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2014531522 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14423051 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |