US9626761B2 - Sampling method and image processing apparatus of CS-RANSAC for estimating homography - Google Patents

Sampling method and image processing apparatus of CS-RANSAC for estimating homography Download PDF

Info

Publication number
US9626761B2
US9626761B2 US14/680,969 US201514680969A US9626761B2 US 9626761 B2 US9626761 B2 US 9626761B2 US 201514680969 A US201514680969 A US 201514680969A US 9626761 B2 US9626761 B2 US 9626761B2
Authority
US
United States
Prior art keywords
features
sampling
input image
constraints
abstracted
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US14/680,969
Other languages
English (en)
Other versions
US20160189359A1 (en
Inventor
Geun Sik Jo
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inha Industry Partnership Institute
Original Assignee
Inha University Research and Business Foundation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inha University Research and Business Foundation filed Critical Inha University Research and Business Foundation
Assigned to INHA-INDUSTRY PARTNERSHIP INSTITUTE reassignment INHA-INDUSTRY PARTNERSHIP INSTITUTE ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: JO, GEUN SIK
Assigned to JO, GEUN SIK reassignment JO, GEUN SIK ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: INHA INDUSTRY PARTNERSHIP INSTITUTE
Publication of US20160189359A1 publication Critical patent/US20160189359A1/en
Application granted granted Critical
Publication of US9626761B2 publication Critical patent/US9626761B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G06T7/004—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/35—Determination of transform parameters for the alignment of images, i.e. image registration using statistical methods
    • G06K9/4604—
    • G06K9/6202—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/77—Determining position or orientation of objects or cameras using statistical methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/757—Matching configurations of points or features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/24—Character recognition characterised by the processing or recognition method
    • G06V30/248—Character recognition characterised by the processing or recognition method involving plural approaches, e.g. verification by template match; Resolving confusion among similar patterns, e.g. "O" versus "Q"
    • G06V30/2504—Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image

Definitions

  • Embodiments of the inventive concept relate to a sampling method and image processing apparatus of RANSAC and CS-RANSAC algorithms for obtaining plane homography between two images.
  • Augmented Reality (AR) systems are attracted as effective means for visualizing weather information which can be expressed in texts, still images, animations, videos, displays for mobile devices, or 3D objects of cameras.
  • AR systems are steadily studied to enable highly complex and microscopic works such as maintenance and repair of aircrafts.
  • Those virtual object poses may be estimated by calculating homography matrixes between reference images and camera images.
  • the homography matrix is calculated through a Random Sample Consensus (RANSAC) algorithm and each sample set is selected at random.
  • RANSAC Random Sample Consensus
  • a general RANSAC algorithm usually operates to select features without regarding positional correlations between the features. During this, sets of the selected features form linearity or placed so closely, finally degrading the accuracy of the homography matrix.
  • FIGS. 1 and 2 illustrate pixels where reference images are mapped with camera images.
  • pixels included in the reference images of the left are mapped to camera images of the right.
  • This pixel mapping is carried out by means of a homography matrix which is calculated from selected four pairs of features.
  • the feature pairs are indicated by yellow circles.
  • the features just can accurately estimate poses of objects which are placed in a selected specific area (e.g. the area in the yellow rectangular box of FIG. 1 ), but inaccurate in estimating poses of objects placed in other areas (e.g. a pitch trimmer indicated by the orange circuit).
  • FIG. 2 even if selected features are placed in a distance, it is difficult to accurately estimate poses of objects as like the case of linearity. In other words, the accuracy of the homography matrix is degraded to be worse.
  • a general RANSAC algorithm is formed of hypothesis and estimation steps.
  • the hypothesis step features are selected at random.
  • the estimation step the consistency between two features (e.g. the number of inliers) is inspected. Those two steps are repeated until a process of finding better features for representing datasets becomes failed.
  • an LO-RS algorithm e.g. the number of inliers
  • T-RS algorithm e.g. the number of inliers
  • MF-RS algorithm e.g. the number of inliers
  • homography matrixes calculated by the LO-RS algorithm, the T-RS algorithm, and the MF-RS algorithm also include insignificant samples (features laid on linearity or crowded in a specific area), it is difficult to accurately estimate poses of objects throughout the whole area of image. That is, the accuracy of homography matrix is still standing in a low level.
  • One aspect of embodiments of the inventive concept is therefore directed to highly accurate homography estimation, without abstracting an insignificant sample (features laid on linearity or crowded in a specific area) while sampling an RANSAC algorithm, in estimating a homography by means of the RANSAC algorithm.
  • Another aspect of embodiments of the inventive concept is directed to reducing the whole number of iterations of an RANSAC algorithm by estimating a superior homography in one sampling operation.
  • Still another aspect of embodiments of the inventive concept are directed to efficiently processing an RANSAC algorithm by estimating a homography matrix from dynamic application with constraints and sizes of grid patterns in correspondence with images.
  • a sampling method for estimating a homography matrix that represents conversion correlations between pluralities of images by means of Constraint Satisfaction-Random Sample Consensus (CS-RANSAC) may include the steps of: sampling to divide an input image into a form of grids (N by N), select features, which are used for calculating a homography matrix, from features, which are abstracted from the input image, by means of a random sampling, and inspect whether the features selected by the random sampling satisfy predefined constraints; and executing model estimation to calculate the homography matrix from the features only if the selected features satisfy the constraints.
  • CS-RANSAC Constraint Satisfaction-Random Sample Consensus
  • the sampling step may be configured to dynamically apply grid sizes in accordance with a distribution pattern of the features abstracted from the input image.
  • the sampling step as the features selected through the random sampling disagree with the constraints, may be configured to reapply the random sampling to the features abstracted from the input image, and reselect features for calculating the homography matrix.
  • the sampling step may be configured to reselect features, which are used for calculating the homography matrix, from features, which are abstracted from the input image, with reference to similarity of the abstracted features.
  • the sampling step as the features selected through the random sampling disagree with the constraints, may be configured to reselect features, which are used for calculating the homography matrix, with reference to the predetermined number of iteration times and the maximum number of sampling times.
  • the sampling step may be configured to dispersedly classify features, which are distributed over the whole area of the input image, and select the features, which are used for calculating the homography matrix, by applying Constraint Satisfaction Problems (CSP).
  • CSP Constraint Satisfaction Problems
  • the CSP may be configured to include variables that represent the features, domains that represent positional correlations of the features, and the constraints.
  • a size of a grid area, in which the features are dispersedly classified, may be dynamically determined by a distribution pattern of the features abstracted from the input image.
  • the sampling step may be configured to execute a predisposing process that divides the input image of the plural images into pluralities of grid areas, and classifies the features, which are abstracted from the input image, into the grid areas.
  • the sampling step may be configured to classify the features, which correspond to each unit grid, with reference to coordinate values of the features.
  • the sampling step may be configured to apply at least one of linear and distance constraints, the linear constraint being configured to prevent other features from being selected at a position forming linearity with the selected features, and the distance constraint being configured to prevent other features from being selected in a distance from the selected features.
  • the linear constraint may satisfy all conditions of selecting one feature from one unit grid, selecting one feature from grid areas of the same row, selecting one feature from grid areas of the same column, and selecting one feature from grid areas of the same diagonal.
  • a computer readable medium may include instructions to control a computer system to execute a Constraint Satisfaction-Random Sample Consensus (CS-RANSAC) algorithm.
  • the CS-RANSAC algorithm may be a method of a homography matrix that represents conversion correlations between pluralities of images.
  • the instructions may be configured to control the computer system through a method including the steps of: sampling to select features, which are used for calculating a homography matrix, from features, which are abstracted from an input image of the plural images, by means of a random sampling, and inspect whether the features selected by the random sampling satisfy predefined constraints; and executing model estimation to calculate the homography matrix from the features that satisfy the constraints.
  • the sampling step may be configured to execute a predisposing process that divides the input image of the plural images into pluralities of grid areas, and classifies the features, which are abstracted from the input image, into the grid areas.
  • an image processing apparatus for estimating a homography matrix that represents conversion correlations between pluralities of images by means of Constraint Satisfaction-Random Sample Consensus (CS-RANSAC) may include: a sampler configured to select features, which are used for calculating a homography matrix, from features, which are abstracted from the input image, by means of a random sampling, and inspect whether the features selected by the random sampling satisfy predefined constraints; and a model estimator configured to calculate the homography matrix from the features that satisfy the constraints.
  • a sampler configured to select features, which are used for calculating a homography matrix, from features, which are abstracted from the input image, by means of a random sampling, and inspect whether the features selected by the random sampling satisfy predefined constraints
  • a model estimator configured to calculate the homography matrix from the features that satisfy the constraints.
  • the sampler may be configured to dynamically apply one of pluralities of predefined grid sizes in accordance with a distribution pattern of the feature abstracted from the input image.
  • the sampler as the features selected through the random sampling disagree with the constraints, may be configured to reapply the random sampling to the features abstracted from the input image, and reselect features for calculating the homography matrix.
  • the sampler as the features selected through the random sampling disagree with the constraints, may be configured to reselect features, which are used for calculating the homography matrix, from features, which are abstracted from the input image, with reference to similarity of the abstracted features.
  • the sampler may be configured to execute a predisposing process that divides the input image of the plural images into pluralities of grid areas, and classifies the features, which are abstracted from the input image, into the grid areas.
  • the sampler may be configured to apply at least one of linear and distance constraints, the linear constraint being configured to prevent other features from being selected at a position forming linearity with the selected features, the distance constraint being configured to prevent other features from being selected in a distance from the selected features.
  • estimating a homography by means of the RANSAC algorithm may be carried out to exclude abstracting an insignificant sample (features laid on linearity or crowded in a specific area) while sampling an RANSAC algorithm, thereby obtaining highly accurate homography estimation.
  • FIG. 1 shows pixels mapped with features which form linearity.
  • FIG. 2 shows pixels mapped with features which are placed within a distance.
  • FIG. 3 shows a process of dividing a target image into grid areas for applying CSP to a sampling process of the RANSAC algorithm in an embodiment of the inventive concept.
  • FIG. 4 shows an operation of select features, which are distributed over the whole area of an image, without insignificant samples in an embodiment of the inventive concept.
  • FIG. 5 is a block diagram illustrating an image processing apparatus to estimate a homography matrix in the inventive concept.
  • FIG. 6 shows distributions of features in an embodiment of the inventive concept.
  • FIG. 7 shows a process of dynamically applying constraints in an embodiment of the inventive concept.
  • FIG. 8 shows a result of experiment for optimal grid sizes to calculate a homography along patterns of features in an embodiment of the inventive concept.
  • FIG. 9 shows a result of homography estimated when selected features are laid on linearity or crowded in a specific area.
  • FIG. 10 shows CSP parameters corresponding to grid areas in an embodiment of the inventive concept.
  • the present embodiments of the inventive concept will be provided to improve the accuracy of an RNASAC algorithm for obtaining a plane homography between two images, which is involved in a sampling method of the RNASAC algorithm capable of abstracting samples to estimate a highly accurate homography, without selecting samples that may estimate an incorrect homography, when abstracting samples for obtaining a homography in a sampling process of an RANSAC algorithm.
  • the present embodiments of the inventive concept will be provided to more accurately estimate a homography matrix, which is especially concerned in a sampling method of estimating a homography matrix by means of a Constraint Satisfaction-Random Sample Consensus (CS-RABSAC) algorithm.
  • CS-RABSAC Constraint Satisfaction-Random Sample Consensus
  • a Speeded Up Robust Feature (SURF) descriptor may work to abstract features which can be respectively representative of images. These abstracted features may be compared in similarity by means of Euclidean distance to match the features up with their similars. Then a homography matrix may be obtained from the matched pairs of the features.
  • SURF Speeded Up Robust Feature
  • an RANSAC algorithm may be employed to obtain a homography matrix without any mismatched pairs of features (outliers).
  • the RABSAC algorithm is a technique of repetitively estimating mathematical model parameters from a data set, which includes false information, through random sampling.
  • the RANSAC algorithm is applied to evade mismatched distinct pairs in probability from a set of mismatched distinct pairs (a data set including outliers) by way of random sampling, and then obtain a more accurate homography matrix.
  • the present embodiments of the inventive concept are provided to raise efficiency of the RANSAC algorithm and accuracy of the homography matrix by applying constraints to a sampling process of the RANSAC algorithm.
  • a process of selecting features for estimating a homography matrix may be involved in a combination by which r-numbered (r ⁇ 4) features are selected from different m-numbered features.
  • Equation 1 is used to calculate the occasional number of combinations and it is assumed that the number of features abstracted from a single image is 100.
  • a homography matrix may be calculated by means of features which are randomly selected through a random sampling not by counting all of the occasional number. A process of calculating a homography matrix by means of the RANSAC algorithm will be described later with reference to FIGS. 3 and 4 .
  • FIG. 3 shows a process of dividing a target image into grid areas for applying CSP to a sampling process of the RANSAC algorithm in an embodiment of the inventive concept
  • FIG. 4 shows an operation of select features, which are distributed over the whole area of an image, without insignificant samples in an embodiment of the inventive concept
  • FIG. 5 is a block diagram illustrating an image processing apparatus to estimate a homography matrix in the inventive concept.
  • the image processing apparatus 500 may include a sampler 501 and an estimator 502 .
  • the embodiments of the inventive concept, relevant to improving accuracy of the RANSAC algorithm for obtaining a plane homography between two images, are concerned with a sampling method (CS-RANSAC) under a CSP-based RANSAC algorithm capable of abstracting a highly accurate homography without samples involved in an incorrect homography when abstracting samples to obtain homographies during a sampling process of the RANSAC algorithm.
  • a homography matrix is calculated using features which are randomly selected by means of a random sampling.
  • the process of calculating a homography matrix by the RANSAC algorithm is as follows.
  • a step of selecting a feature and inspecting whether the selected feature satisfies a predefined constraint may be carried out by the sampler 501 , and an operation of calculating, evaluating, determining whether the RANSAC algorithm is terminated, and returning the last homography matrix may be carried out by the model estimator 502 .
  • the process stated above may be carried out to eventually obtain a homography matrix.
  • a general random sampling usually results in inconsistency of abstracted features and inaccuracy of a homography matrix which is calculated from the abstracted features, the RANSAC algorithm would be degraded overall in efficiency.
  • the present embodiments of the inventive concept are provided to raise the overall efficiency of a RANSAC algorithm for obtaining a homography matrix by applying a CSP to a sampling process of the RANSAC algorithm. For instance, in selecting features, constraints may be applied to prevent insignificant features from being abstracted, unnecessary calculation steps may be reduced to improve the overall efficiency of the RANSAC algorithm, and features abstracted from the state of linearity or a specific area may be excluded to raise accuracy of a homography matrix calculated therein. Accordingly, if features are properly distributed and then selected for calculation of a homography matrix, it may be possible to obtain a more accurate homography matrix.
  • a predisposing process may be first carried out to divide a target image into N ⁇ N grids and classify features into coordinates of grid areas.
  • Features abstracted from each image by means of the SURF may have coordinate values indicating target positions on x and y axes that originate from the left top when a 2D image forms a plane.
  • the image is divided into N ⁇ N grid area, and the grid areas are correspondingly classified with reference to the coordinate values of the features which are abstracted from the image.
  • a unit grid area may include 0 ⁇ m-numbered features.
  • the features distributed over the image area may be dispersedly classified into the N ⁇ N grid areas.
  • the Constraint Satisfaction Problem is an efficient method for searching a solution to satisfying a given constraint, which is capable of finding a solution without considering all cases in a reduced search range because any one that does not satisfy the constraint is excluded from the search range.
  • the CSP is formed of domains respective to parameters, and constraints, as a problem for finding the constraints in the domains belonging to the parameters.
  • a CSP to be applied to a sampling process of the RANSAC algorithm is as follows.
  • 1.
  • linear and distance constraints respective to the grid areas may be applied thereto to select features.
  • the linear constraints will be described later with reference to Table 1.
  • Most Artificial Intelligence may be formalized according to a CSP which is defined with a set of variables X 1 , X 2 , . . . , X n and a set of constraints C 1 , C 2 , . . . , C m .
  • a parameter X i may correspond to a nonempty domain D i among several available values v i .
  • the CSP is provided to complete an allocation that satisfies all constraints.
  • the CSP may be representative of patterns of constraints, and used for enabling an effective and general inference even without a specific knowledge about additional domains.
  • the exemplary embodiments of the inventive concept will be described about a sampling method under the CS-RANSAC algorithm where the CSP is applied to a sampling process that calculates a homography matrix.
  • the CSP is applied to a sampling process that calculates a homography matrix.
  • variables prepared to calculate a data model are defined as a set of features and represented in f k (k ⁇ 1, 2, . . . , n ⁇ ).
  • n may denote the number of samples.
  • an input image I s may be divided into N ⁇ N grid areas. Then, with a target of features which are included in the divided grid area, the CSP may be applied to inspect whether each feature pair satisfies the constraints.
  • the features may represent features which are randomly abstracted among features, which are abstracted from the input image, through a random sampling. During this, values allocable respectively to the features may be determined according to positions (marked with rows and columns) of their corresponding cells where the features are placed. Then the features placed in the same cells may be mapped with the same parameter values.
  • a model estimation step may be carried out for samples which have been processed through a sampling step.
  • the constraints may be applied to a group of samples which are selected from one of cells forming a grid area. By inspecting whether the constraints are satisfied only in a group of samples, it may be permissible to reduce a search range thereof.
  • the predetermined constraints may be applied thereto.
  • Table 1 summarizes the correlations between features which are randomly selected.
  • the linear and distance constraints may be used to exclude insignificant features, such as problems of linearity and crowd with the features, from a homography matrix.
  • the linear constraint is predefined to inhibit linear samples from being selected, while the distance constraint is predefined to restrict all samples to be sufficiently distanced each other.
  • C fifj denotes a set of the constraints between two features f i and f j .
  • row i and row j are indexes indicating rows of their corresponding cells where features f i and f j are respectively placed, while the terms col i and col j are indexes indicating columns of their corresponding cells where features f i and f j are respectively placed.
  • the distance constraint is applied thereto, by selecting one of features for calculating a homography matrix, it may be permissible to make another feature unselected within a distance from the selected feature. In other words, a feature distanced from a selected feature by a distance may be selected as another one.
  • the CS-RANSAC algorithm may be formed of a sampling step that abstracts features, for calculating a homography matrix, from features which are abstracted from an input image, through a random sampling and then applies the constraints thereto, and a model estimation step that calculates the homography matrix with reference to features which satisfy the constraints.
  • the CS-RANSAC algorithm as a modification of a sampling step of an RANSAC algorithm, is proposed to exclude insignificant samples, which would be inadvertently selected as features for calculating a homography matrix, before the model estimation step by applying the CSP thereto.
  • Table 3 summarizes the CS-RANSAC algorithm.
  • Algorithm 1 of Table 3 it may be allowable to first select n-numbered features at random among features which are abstracted from an input image I s . Then, according to the constraints defined in Table 2, the selected features may be inspected in consistency. In other words, the selected features may be inspected about whether they satisfy the linear and distance constraints. During this, only if the inspected features are agreed with all of the constraints, the next step may be carried out.
  • the CS-RANSAC algorithm may repetitively execute a sampling operation defined as sIterations in Table 3. During this, the sampling operation may be repetitively carried out with reference to the maximum sampling times ⁇ s in which the number of sampling iteration times sIterations is predetermined.
  • ⁇ s the number of sampling iteration times sIterations
  • the homography matrix may be calculated from the reselected features.
  • An operation of calculating a homography matrix from features in the CS-RANSAC algorithm is substantially the same as the RANSAC algorithm.
  • a data model (a set of features selected to calculate a homography matrix) may be repetitively calculated and updated through a process of reselecting features by applying similarity or CSP model thereto.
  • the data model may be evaluated with reference to the number of inliers.
  • the number of inliers may correspond to a critical error value ⁇ ⁇ , and indicate the number of features corresponding to a data model that is generated by iteration.
  • the number of iteration times of the CS-RANSAC algorithm may be updated at the last of iteration, as given by Equation 3.
  • nIterations ⁇ log ⁇ ( 1 - p ) log ⁇ ( 1 - ( 1 - e ) n ) ⁇ [ Equation ⁇ ⁇ 3 ]
  • Equation 3 the term p denotes probability that the selected features are all inliers, and for example may be predetermined as 0.999 for obtaining high accuracy.
  • the term e denotes probability that at least one of the selected features is false, which may be calculated by Equation 4.
  • n may denote the number of features that are used for calculating a data model (homography matrix).
  • a data model having the largest amount of inliers may be determined as an optimal data model for calculating a homography matrix.
  • the optimal data model may be a homography matrix H.
  • a CSP is applied to a sampling process of the CS-RANSAC algorithm for estimating a homography, thus insignificant samples selected during the sampling process may be excluded to reduce the number of iteration times of the RANSAC algorithm and improve accuracy of an estimated homography.
  • the CS-RANSAC algorithm may be used to reduce an error rate and a processing time, thereby capable of calculating a homography matrix with high probability of inliers. Now an operation of dynamically applying the constraints will be detailed in conjunction with FIGS. 6 and 7 .
  • FIG. 6 shows distributions of features in an embodiment of the inventive concept
  • FIG. 7 shows a process of dynamically applying constraints in an embodiment of the inventive concept.
  • operations of abstracting features by means of SURF or SHIFT and dynamically applying the constraints may be carried out in the image processing apparatus 500 of FIG. 5 .
  • the operations may be carried out by the sampler 501 and the model estimator 502 .
  • the features abstracted according to contents of an input image may be classified into five groups G 1 to G 5 .
  • the group G 1 corresponds to that few features are distributed over the whole image
  • the group G 2 corresponds to that many features are distributed over the whole image
  • the group G 3 corresponds to that many features are distributed over the center of the image
  • the group G 4 corresponds to that few features are distributed over a specific area of the image
  • the group G 5 corresponds to that many features are distributed over a specific area of the image.
  • the CS-RANSAC algorithm may operate to abstract features from an input image, and effectively select four features exemplarily to calculate a homography from the abstracted features.
  • the image processing apparatus 500 operates to show various distribution patterns of features according to a type of input image, it may be allowable to dynamically applying the constraints in accordance with a distribution pattern of the features.
  • images 701 and 702 may be exemplarily shown to represent the same image respectively into 7 ⁇ 7 grids 701 and 14 ⁇ 14 grids 702 .
  • the image processing apparatus 500 may dynamically determine a size of image in accordance with a distribution of abstracted features, i.e. a pattern of the abstracted features. Subsequently, the image processing apparatus 500 may abstract one feature from each grid at maximum, and then determine whether four features selected from the whole image satisfy the distance and linear constraints.
  • FIG. 8 shows a result of experiment for optimal grid sizes to calculate a homography along patterns of features in an embodiment of the inventive concept.
  • the image processing apparatus 500 may classify features, which are abstracted from an input image, into five groups. Then, an optimal grid size may be dynamically determined to calculate a homography in accordance with a distribution pattern and the number of features of each group. Referring to FIGS. 6 and 8 , the image processing apparatus 500 may classify abstracted features into one of the groups G 1 to G 5 in accordance with whether few features, among abstracted features, are distributed over the whole image, whether many features are distributed over the whole image, whether few features are distributed over a specific area of the image, or whether many features are distributed over a specific area of the image.
  • the image processing apparatus 500 may abstract features from an input image and then analyze which one of the predetermined five groups G 1 to G 5 has a pattern similar or identical to a distribution of the abstracted features. For instance, the image processing apparatus 500 may use a K-means classification algorithm to determine whether a distribution of the abstracted features is similar or identical to one of the groups G 1 to G 5 in a predetermined similarity error range. And the image processing apparatus 500 may dynamically apply an optimal grid size, which corresponds to a determined group, to the input image. For example, if a pattern of the abstracted features is determined as corresponding to the group G 1 , the image processing apparatus 500 may apply an optimal grid size, which corresponds to the preliminarily stored group G 1 , to the input image. In this manner, the image processing apparatus 500 may assure to improve the performance of homography, for which a homography matrix is calculated by way of applying optimal grid sizes which respectively correspond to the groups.
  • the image processing apparatus 500 may determine whether the abstracted four features satisfy the predefined distance and linear constraints. If the four features satisfy the constraints, the image processing apparatus 500 may calculate a homography matrix. Unless the four features satisfy the constraints, the image processing apparatus 500 may repeat a process of abstracting one feature from each grid and abstracting four features at maximum from the whole grid. In this manner, by dynamically applying the grid sizes and constraints to calculate a homography matrix, it may be possible to efficiently process the RANSAC algorithm, as well as calculating the homography matrix in high accuracy.
  • the aforementioned CS-RANSAC sampling method may be applicable to an AR-based image processing apparatus.
  • Each process included in the CS-RANSAC sampling method may be carried out by the image processing apparatus of FIG. 5 .
  • an image processing apparatus for executing the CS-RANSAC sampling method may be organized to include a processor and a memory.
  • the memory may be a program corresponding to the CS-RANSAC algorithm that estimates a homography matrix representing conversion correlations between pluralities of images.
  • the memory may store a program including instructions that select features for calculating a homography matrix by means of a random sampling, applying the constraints to the selected features, and calculating the homography matrix with reference to features satisfying the constraints.
  • the processor may be a unit for processing a sampling process and a model estimation process in accordance with instructions loaded in the memory, which may include a microprocessor such as a CPU.
  • Methods according to embodiments of the inventive concept may be implemented in a form of program instructions, which are executable by various computer systems, and recorded in a computer readable medium.
  • the computer readable medium may include program instructions, data files, data structures, or combinations of them.
  • Program instructions recorded in the computer readable medium may be specifically designed and composed according to embodiments of the inventive concept, but may be available as being known even by those skilled in the art of computer software. Additionally, the file systems may be recorded in a computer readable medium.
  • Any of the functions disclosed herein may be implemented using means for performing those functions. Such means include, but are not limited to, any of the components disclosed herein, such as the computer-related components described below.
  • the techniques described above may be implemented, for example, in hardware, one or more computer programs tangibly stored on one or more computer-readable media, firmware, or any combination thereof.
  • the techniques described above may be implemented in one or more computer programs executing on (or executable by) a programmable computer including any combination of any number of the following: a processor, a storage medium readable and/or writable by the processor (including, for example, volatile and non-volatile memory and/or storage elements), an input device, and an output device.
  • Program code may be applied to input entered using the input device to perform the functions described and to generate output using the output device.
  • Embodiments of the present invention include features which are only possible and/or feasible to implement with the use of one or more computers, computer processors, and/or other elements of a computer system. Such features are either impossible or impractical to implement mentally and/or manually.
  • any claims herein which affirmatively require a computer, a processor, a memory, or similar computer-related elements, are intended to require such elements, and should not be interpreted as if such elements are not present in or required by such claims. Such claims are not intended, and should not be interpreted, to cover methods and/or systems which lack the recited computer-related elements.
  • any method claim herein which recites that the claimed method is performed by a computer, a processor, a memory, and/or similar computer-related element is intended to, and should only be interpreted to, encompass methods which are performed by the recited computer-related element(s).
  • Such a method claim should not be interpreted, for example, to encompass a method that is performed mentally or by hand (e.g., using pencil and paper).
  • any product claim herein which recites that the claimed product includes a computer, a processor, a memory, and/or similar computer-related element is intended to, and should only be interpreted to, encompass products which include the recited computer-related element(s). Such a product claim should not be interpreted, for example, to encompass a product that does not include the recited computer-related element(s).
  • Each computer program within the scope of the claims below may be implemented in any programming language, such as assembly language, machine language, a high-level procedural programming language, or an object-oriented programming language.
  • the programming language may, for example, be a compiled or interpreted programming language.
  • Each such computer program may be implemented in a computer program product tangibly embodied in a machine-readable storage device for execution by a computer processor.
  • Method steps of the invention may be performed by one or more computer processors executing a program tangibly embodied on a computer-readable medium to perform functions of the invention by operating on input and generating output.
  • Suitable processors include, by way of example, both general and special purpose microprocessors.
  • the processor receives (reads) instructions and data from a memory (such as a read-only memory and/or a random access memory) and writes (stores) instructions and data to the memory.
  • Storage devices suitable for tangibly embodying computer program instructions and data include, for example, all forms of non-volatile memory, such as semiconductor memory devices, including EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROMs. Any of the foregoing may be supplemented by, or incorporated in, specially-designed ASICs (application-specific integrated circuits) or FPGAs (Field-Programmable Gate Arrays).
  • a computer can generally also receive (read) programs and data from, and write (store) programs and data to, a non-transitory computer-readable storage medium such as an internal disk (not shown) or a removable disk.
  • Any data disclosed herein may be implemented, for example, in one or more data structures tangibly stored on a non-transitory computer-readable medium. Embodiments of the invention may store such data in such data structure(s) and read such data from such data structure(s).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Computer Graphics (AREA)
  • Image Analysis (AREA)
US14/680,969 2014-12-31 2015-04-07 Sampling method and image processing apparatus of CS-RANSAC for estimating homography Active 2035-04-25 US9626761B2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020140195389A KR101618996B1 (ko) 2014-12-31 2014-12-31 호모그래피를 추정하기 위한 샘플링 방법 및 영상 처리 장치
KR10-2014-0195389 2014-12-31

Publications (2)

Publication Number Publication Date
US20160189359A1 US20160189359A1 (en) 2016-06-30
US9626761B2 true US9626761B2 (en) 2017-04-18

Family

ID=56020609

Family Applications (1)

Application Number Title Priority Date Filing Date
US14/680,969 Active 2035-04-25 US9626761B2 (en) 2014-12-31 2015-04-07 Sampling method and image processing apparatus of CS-RANSAC for estimating homography

Country Status (2)

Country Link
US (1) US9626761B2 (ko)
KR (1) KR101618996B1 (ko)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210073443A1 (en) * 2018-05-15 2021-03-11 Shanghaitech University Fast and deterministic algorithm for consensus set maximization

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106570935B (zh) * 2016-09-09 2020-07-07 北京林业大学 植被分布模拟方法及装置
CN109325510B (zh) * 2018-07-27 2021-06-08 华南理工大学 一种基于网格统计的图像特征点匹配方法
KR102167071B1 (ko) 2019-11-29 2020-10-16 한길씨앤씨 주식회사 서술자 매칭쌍 분석 기술을 적용한 이미지 인식 시스템 및 방법
JP7253573B2 (ja) * 2020-04-09 2023-04-06 センスタイム インターナショナル ピーティーイー．リミテッド マッチング方法、装置、電子機器及びコンピュータ可読記憶媒体
CN112819904B (zh) * 2021-03-15 2022-02-01 亮风台（上海）信息科技有限公司 一种用于标定ptz摄像机的方法与设备
CN112927263B (zh) * 2021-04-25 2022-08-02 南京邮电大学 一种基于互补几何模型与相似度矩阵的运动分割方法
CN113516697B (zh) * 2021-07-19 2024-02-02 北京世纪好未来教育科技有限公司 图像配准的方法、装置、电子设备及计算机可读存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110206296A1 (en) * 2010-02-09 2011-08-25 Satoshi Sakaguchi Super-resolution processor and super-resolution processing method
US20140044348A1 (en) * 2011-02-18 2014-02-13 National Ict Australia Limited Image quality assessment
US9189966B2 (en) * 2012-05-02 2015-11-17 Agency For Science, Technology And Research System for learning trail application creation
US20160142644A1 (en) * 2014-11-17 2016-05-19 Industrial Technology Research Institute Surveillance systems and image processing methods thereof

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005309782A (ja) 2004-04-21 2005-11-04 Toyota Motor Corp 画像処理装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110206296A1 (en) * 2010-02-09 2011-08-25 Satoshi Sakaguchi Super-resolution processor and super-resolution processing method
US20140044348A1 (en) * 2011-02-18 2014-02-13 National Ict Australia Limited Image quality assessment
US9189966B2 (en) * 2012-05-02 2015-11-17 Agency For Science, Technology And Research System for learning trail application creation
US20160142644A1 (en) * 2014-11-17 2016-05-19 Industrial Technology Research Institute Surveillance systems and image processing methods thereof

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Dubrofsky "Homography Estimation", The University of British Columbia, pp. 1-28, 2009. *
Thomas, et al. ((Fast Robust Perspective Transform Estimation for Automatic Registration in Disaster Response Applications), IEEE, pp. 2190-2193, 2012. *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210073443A1 (en) * 2018-05-15 2021-03-11 Shanghaitech University Fast and deterministic algorithm for consensus set maximization

Also Published As

Publication number Publication date
KR101618996B1 (ko) 2016-05-09
US20160189359A1 (en) 2016-06-30

Similar Documents

Publication Publication Date Title
US9626761B2 (en) Sampling method and image processing apparatus of CS-RANSAC for estimating homography
US11556797B2 (en) Systems and methods for polygon object annotation and a method of training an object annotation system
US10540772B2 (en) Feature trackability ranking, systems and methods
JP6832504B2 (ja) 物体追跡方法、物体追跡装置およびプログラム
US9619691B2 (en) Multi-view 3D object recognition from a point cloud and change detection
CN109740588B (zh) 基于弱监督和深度响应重分配的x光图片违禁品定位方法
CN101398934B (zh) 对图像中的对象进行定位的方法和系统
Ulrich et al. Real-time object recognition using a modified generalized Hough transform
US20210158549A1 (en) Computer Vision Systems and Methods for Unsupervised Learning for Progressively Aligning Noisy Contours
Cicconet et al. A convolutional approach to reflection symmetry
US10740901B2 (en) Encoder regularization of a segmentation model
CN110189341B (zh) 一种图像分割模型训练的方法、图像分割的方法及装置
US9886772B2 (en) System for correcting RPC camera model pointing errors using 2 sets of stereo image pairs and probabilistic 3-dimensional models
CN110930413A (zh) 一种基于弱监督多核分类优化合并的图像分割方法
EP3526728B1 (en) System and method for object recognition based estimation of planogram compliance
Li et al. Predictive RANSAC: Effective model fitting and tracking approach under heavy noise and outliers
CN110969199A (zh) 一种图像分类的方法、装置和存储介质
CN110517300A (zh) 基于局部结构算子的弹性图像配准算法
Mayer Efficient hierarchical triplet merging for camera pose estimation
US11200676B2 (en) Shift invariant loss for deep learning based image segmentation
Wang et al. Degree of nonlinearity (DoN) measure for target tracking in videos
Pujol et al. On searching for an optimal threshold for morphological image segmentation
CN111161303A (zh) 标记方法、装置、计算机设备和存储介质
Hoseinnezhad et al. Multi-bernoulli sample consensus for simultaneous robust fitting of multiple structures in machine vision
Zhang A congruent hybrid model for conflation of satellite image and road database

Legal Events

Date Code Title Description
AS Assignment

Owner name: INHA-INDUSTRY PARTNERSHIP INSTITUTE, KOREA, REPUBL

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:JO, GEUN SIK;REEL/FRAME:037816/0256

Effective date: 20150401

AS Assignment

Owner name: JO, GEUN SIK, KOREA, REPUBLIC OF

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INHA INDUSTRY PARTNERSHIP INSTITUTE;REEL/FRAME:037962/0857

Effective date: 20160211

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

Year of fee payment: 4