WO2022190224A1 - 基礎行列生成装置、制御方法、及びコンピュータ可読媒体 - Google Patents
基礎行列生成装置、制御方法、及びコンピュータ可読媒体 Download PDFInfo
- Publication number
- WO2022190224A1 WO2022190224A1 PCT/JP2021/009391 JP2021009391W WO2022190224A1 WO 2022190224 A1 WO2022190224 A1 WO 2022190224A1 JP 2021009391 W JP2021009391 W JP 2021009391W WO 2022190224 A1 WO2022190224 A1 WO 2022190224A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- feature
- point
- distance
- pairs
- Prior art date
Links
- 239000011159 matrix material Substances 0.000 title claims abstract description 178
- 238000000034 method Methods 0.000 title claims description 80
- 238000001514 detection method Methods 0.000 claims description 55
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 10
- 230000004323 axial length Effects 0.000 claims description 4
- 238000009795 derivation Methods 0.000 abstract description 13
- 230000008569 process Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 11
- 238000004364 calculation method Methods 0.000 description 5
- 230000009466 transformation Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 2
- 230000001174 ascending effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
Definitions
- This disclosure relates to the generation of basic matrices.
- Relative camera parameters are extrinsic parameters whose absolute magnitude is unknown (3D translation vector with 2 degrees of freedom and rotation with 3 degrees of freedom) and internal parameters such as focal length, which are multiplied. expressed as a fundamental matrix.
- Non-Patent Document 1 describes a method of calculating a basic matrix by using eight or more sets of corresponding points projected onto the images with the same three-dimensional coordinates between images.
- Non-Patent Document 2 describes a method of estimating the focal length at the same time as the basic matrix using six pairs of corresponding points.
- Non-Patent Document 3 describes a method of estimating the focal length and lens distortion at the same time as the basic matrix using seven or more pairs of corresponding points.
- Non-Patent Document 4 describes a method of calculating a fundamental matrix and a focal length from two sets of corresponding points by using affine-invariant feature points.
- Non-Patent Documents 1 to 4 mentioned above multiple pairs of feature points corresponding to each other are detected from two images, and a robust estimation algorithm such as RANSAC (RANdom SAmple Consensus) is used from a set of detected feature point pairs. Then, the correct base matrix is generated by removing the erroneous corresponding points.
- RANSAC Random SAmple Consensus
- An object of the present disclosure is to provide a new technique for generating fundamental matrices.
- the fundamental matrix generation device of the present disclosure includes a first detection unit that detects three or more feature point pairs that are pairs of feature points that correspond to each other from the first image and the second image, and for each of the feature point pairs, A point separated by a first distance in a first direction from a point on the first image included in the feature point pair, and a point on the second image included in the feature point pair separated by a second distance in a second direction. a point on the first image and a point on the second image using a second detection unit that detects derived point pairs that are pairs of points, and each of the detected feature point pairs and derived point pairs; and a generator for generating a fundamental matrix representing the geometric constraints of .
- Each of the first direction and the first distance is determined based on feature amounts calculated for points on the first image included in the feature point pair.
- Each of the second direction and the second distance is determined based on feature amounts calculated for points on the second image included in the feature point pair.
- the control method of the present disclosure is executed by a computer.
- the control method includes a first detection step of detecting three or more feature point pairs, which are feature point pairs corresponding to each other, from a first image and a second image; A pair of a point a first distance in a first direction from a point on the first image included and a point a second distance in a second direction from a point on the second image included in the feature point pair. and using each of the detected feature point pairs and derived point pairs to determine the geometry of the points on the first image and the points on the second image and a generation step of generating a fundamental matrix representing the constraints.
- Each of the first direction and the first distance is determined based on feature amounts calculated for points on the first image included in the feature point pair.
- Each of the second direction and the second distance is determined based on feature amounts calculated for points on the second image included in the feature point pair.
- the computer-readable medium of the present disclosure stores a program that causes a computer to execute the control method of the present disclosure.
- FIG. 4 is a diagram illustrating an overview of the operation of the fundamental matrix generation device of Embodiment 1;
- FIG. FIG. 4 is a diagram illustrating feature point pairs and derived point pairs;
- 2 is a block diagram illustrating the functional configuration of the fundamental matrix generation device of Embodiment 1;
- FIG. It is a block diagram which illustrates the hardware constitutions of the computer which implement
- 4 is a flowchart illustrating the flow of processing executed by the fundamental matrix generation device of Embodiment 1;
- Fig. 10 is a flow chart illustrating the flow of processing performed by a fundamental matrix generator using RANSAC;
- FIG. 7 is a diagram exemplifying the flowchart of FIG. 6 to which processing for determining whether or not to generate a base matrix using signed areas is added;
- predetermined values such as predetermined values and threshold values are stored in advance in a storage device or the like that can be accessed from a device that uses the values.
- FIG. 1 is a diagram illustrating an overview of the operation of the basic matrix generation device 2000 of Embodiment 1.
- FIG. 1 is a diagram for facilitating understanding of the outline of fundamental matrix generation device 2000, and the operation of fundamental matrix generation device 2000 is not limited to that shown in FIG.
- the basic matrix generation device 2000 acquires the first image 10 and the second image 20, and applies geometric constraints (called epipolar constraints) between points on the first image 10 and points on the second image 20.
- a base matrix 40 is generated which is a matrix for representing.
- the epipolar constraint that the base matrix 40 should satisfy is represented by, for example, Equation (1) below.
- point m is a point on the first image 10
- point n is a point on the second image 20
- the same three-dimensional coordinates are projected onto each image. That is, the points n and m are points that represent the same location on the real space.
- the points m and n are similarly represented by the coordinates of a 3 ⁇ 1 homogeneous coordinate system whether lens distortion is included or not.
- F is a 3x3 fundamental matrix 40, of which one of the three singular values is known to be zero.
- K is a 3x3 matrix representing camera intrinsic parameters such as focal length.
- the lens distortion parameters are included in addition to the matrix K.
- the basic matrix generation device 2000 generates six or more pairs of corresponding points (corresponding points) between the first image 10 and the second image 20 in order to calculate the basic matrix 40 .
- a pair of corresponding points is hereinafter referred to as a corresponding point pair.
- the points on the first image 10 and the points on the second image 20 included in the corresponding point pair are points that represent the same location on the real space.
- the basic matrix generation device 2000 detects corresponding point pairs by the following method.
- the fundamental matrix generation device 2000 detects pairs of feature points (feature point pairs) corresponding to each other from the feature points detected from the first image 10 and the feature points detected from the second image 20 . That is, a feature point on the first image 10 and a feature point on the second image 20 corresponding to the feature point are detected as a feature point pair.
- at least three sets of feature point pairs are detected as corresponding point pairs to be used for generating the base matrix 40 .
- the basic matrix generation device 2000 uses the feature point pairs detected by the above method to further detect corresponding point pairs. Specifically, the fundamental matrix generation device 2000 generates a derivative point that is the first distance away in the first direction from the feature point on the first image 10 included in the feature point pair, and the second image included in the feature point pair. Detect pairs of derived points that are a second distance apart in a second direction from feature points on 20 . A pair of derived points detected in this manner is hereinafter also referred to as a derived point pair.
- the first direction, the first distance, the second direction, and the second distance are determined using feature amounts calculated for feature points.
- a feature amount such as SIFT that is invariant with respect to the scale and the principal axis direction (hereinafter referred to as a scale-invariant feature amount) is used as the feature amount.
- the first direction for example, the principal axis direction determined by the feature quantity calculated for the feature points on the first image 10 is used.
- the second direction for example, the principal axis direction determined by the feature quantity calculated for the feature points on the second image 20 is used.
- the first distance for example, the size of the scale determined by the feature amount calculated for the feature points on the first image 10 is used.
- the second distance for example, the size of the scale determined by the feature quantity calculated for the feature points on the second image 20 is used.
- FIG. 2 is a diagram illustrating feature point pairs and derived point pairs.
- (m1, n1), (m2, n2), and (m3, n3) are detected as feature point pairs.
- m1, m2 and m3 are feature points on the first image 10 respectively
- n1, n2 and n3 are feature points on the second image 20 respectively.
- the scale a1 and the principal axis direction ⁇ 1 are determined by the scale-invariant feature quantity calculated for the feature point m1.
- the scale b1 and principal axis direction ⁇ 1 are determined by the scale-invariant feature quantity calculated for the feature point n1.
- the direction is represented by an angle with the horizontal direction of the image to the right as a reference of 0 degree.
- the basic matrix generation device 2000 detects a derivative point p1 that is moved a1 in the principal axis direction ⁇ 1 in the feature quantity for the feature point m1. Further, the basic matrix generation device 2000 detects a derivative point q1 obtained by moving the feature point n1 by b1 in the principal axis direction ⁇ 1 in the feature amount. As a result, the pair (p1,q1) of derived point p1 and derived point q1 is detected as a derived point pair.
- the derived point p1 can also be expressed as a point in the principal axis direction on the circumference of the radius a1 centered at the feature point m1. The same is true for the derivation point q1.
- the basic matrix generation device 2000 detects derived points p2 and p3 that are moved a2 and a3 in the principal axis directions ⁇ 2 and ⁇ 3 in the feature amount for the feature points m2 and m3 on the first image 10. Further, the basic matrix generation device 2000 detects derived points q2 and q3 obtained by moving b2 and b3 in the principal axis directions ⁇ 2 and ⁇ 3 in the feature amount for the feature points n2 and n3 on the second image 20. As a result, derived point pairs (p2,q2) and (p3,q3) are detected.
- the basic matrix generation device 2000 generates the basic matrix 40 using the detected 6 or more corresponding point pairs.
- the base matrix 40 is generated using eight or more feature point pairs for the first image 10 and the second image 20 in the present disclosure.
- the fundamental matrix generation device 2000 of the present embodiment can generate the fundamental matrix 40 if the total number of feature point pairs and derivative point pairs is six or more. Therefore, the minimum number of feature point pairs that need to be detected from an image is three. Therefore, compared with the invention of Patent Document 1, there is an advantage that the number of feature point pairs that need to be detected from the image is small.
- the basic matrix generation device 2000 of this embodiment will be described in more detail below.
- FIG. 3 is a block diagram illustrating the functional configuration of the fundamental matrix generation device 2000 of the first embodiment.
- Fundamental matrix generation apparatus 2000 has first detection section 2020 , second detection section 2040 and generation section 2060 .
- the first detection unit 2020 detects three or more feature point pairs from the first image 10 and the second image 20 .
- the second detection unit 2040 detects three or more derivative point pairs from the first image 10 and the second image 20 using each of the three or more feature point pairs.
- the generation unit 2060 generates the basic matrix 40 using the detected feature point pairs and derived point pairs.
- Each functional configuration unit of the basic matrix generation device 2000 may be implemented by hardware (eg, hardwired electronic circuit, etc.) that implements each functional configuration unit, or may be implemented by a combination of hardware and software (eg, : a combination of an electronic circuit and a program that controls it, etc.).
- hardware e.g, hardwired electronic circuit, etc.
- software e.g. : a combination of an electronic circuit and a program that controls it, etc.
- FIG. 4 is a block diagram illustrating the hardware configuration of the computer 500 that implements the basic matrix generation device 2000.
- Computer 500 is any computer.
- the computer 500 is a stationary computer such as a PC (Personal Computer) or a server machine.
- the computer 500 is a portable computer such as a smart phone or a tablet terminal.
- Computer 500 may be a dedicated computer designed to realize fundamental matrix generation device 2000, or may be a general-purpose computer.
- the functions of the fundamental matrix generation device 2000 are realized on the computer 500.
- the application is composed of a program for realizing the functional components of the fundamental matrix generation device 2000 .
- the acquisition method of the above program is arbitrary.
- the program can be acquired from a storage medium (DVD disc, USB memory, etc.) in which the program is stored.
- the program can be obtained by downloading the program from a server device that manages the storage device in which the program is stored.
- Computer 500 has bus 502 , processor 504 , memory 506 , storage device 508 , input/output interface 510 and network interface 512 .
- the bus 502 is a data transmission path through which the processor 504, memory 506, storage device 508, input/output interface 510, and network interface 512 exchange data with each other.
- the method of connecting the processors 504 and the like to each other is not limited to bus connection.
- the processor 504 is various processors such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or FPGA (Field-Programmable Gate Array).
- the memory 506 is a main memory implemented using a RAM (Random Access Memory) or the like.
- the storage device 508 is an auxiliary storage device implemented using a hard disk, SSD (Solid State Drive), memory card, ROM (Read Only Memory), or the like.
- the input/output interface 510 is an interface for connecting the computer 500 and input/output devices.
- the input/output interface 510 is connected to an input device such as a keyboard and an output device such as a display device.
- a network interface 512 is an interface for connecting the computer 500 to a network.
- This network may be a LAN (Local Area Network) or a WAN (Wide Area Network).
- the storage device 508 stores a program that implements each functional component of the basic matrix generation device 2000 (a program that implements the application described above).
- the processor 504 implements each functional component of the fundamental matrix generation device 2000 by reading this program into the memory 506 and executing it.
- the basic matrix generation device 2000 may be realized by one computer 500 or may be realized by a plurality of computers 500. In the latter case, the configuration of each computer 500 need not be the same, and can be different.
- FIG. 4 is a flowchart illustrating the flow of processing executed by the fundamental matrix generation device 2000 of the first embodiment.
- the first detection unit 2020 acquires the first image 10 and the second image 20 (S102).
- the first detection unit 2020 detects three or more feature point pairs using the first image 10 and the second image 20 (S104).
- the second detection unit 2040 detects derived point pairs for each feature point pair using the first image 10 and the second image 20 (S106).
- the generation unit 2060 generates the basic matrix 40 using the feature point pairs and the derived point pairs (S108).
- the first image 10 and the second image 20 are arbitrary captured images generated by an arbitrary camera. However, at least a part of the first image 10 and the second image 20 includes an image area in which the same location is imaged. For example, the first image 10 and the second image 20 are generated by imaging the same building or person from mutually different positions and angles.
- the first detection unit 2020 acquires the first image 10 and the second image 20 (S102).
- the method by which the first detection unit 2020 acquires the first image 10 and the second image 20 is arbitrary.
- the first detection unit 2020 acquires the first image 10 and the second image 20 from the storage device in which they are stored.
- the first image 10 and the second image 20 may be stored in the same storage device, or may be stored in different storage devices.
- the first detection unit 2020 may acquire the first image 10 and the second image 20 from the camera that generated the first image 10 and the camera that generated the second image 20, respectively.
- the first detection unit 2020 detects three or more feature point pairs from the first image 10 and the second image 20 (S104). Therefore, the first detection unit 2020 detects feature points from each of the first image 10 and the second image 20 .
- the feature points detected from the first image 10 and the second image 20 may be arbitrary types of feature points.
- an existing technique can be used as a technique for detecting feature points from an image.
- the first detection unit 2020 calculates the feature amount of the area including the feature points detected from each of the first image 10 and the second image 20 .
- the features calculated here are, for example, scale-invariant features such as SIFT, and features that are invariant to affine transformations such as Hessian-Affine and Affine-SIFT (hereafter referred to as affine-invariant features). .
- An existing technique can be used also for the calculation method of these feature amounts.
- the first detection unit 2020 performs feature point matching between the feature points on the first image 10 and the feature points on the second image 20 using the feature amount calculated for each feature point. conduct. That is, the first detection unit 2020 associates the feature points on the first image 10 and the feature points on the second image 20 with each other based on the degree of similarity of feature amounts. In this way, the feature points on the first image 10 and the feature points on the second image 20 that are associated by feature point matching can be used as a feature point pair.
- An existing technique can be used as a technique for detecting corresponding points from two images by feature point matching.
- the first detection unit 2020 detects any three or more pairs of feature points on the first image 10 and the feature points on the second image 20 that are associated in this way as feature point pairs. For example, the first detection unit 2020 arbitrarily selects one of the feature points detected from the first image 10, and specifies a feature point on the second image 20 that is associated with the feature point by feature point matching. do. That is, the first detection unit 2020 detects that the second image 20 has a feature amount sufficiently similar to the feature amount calculated for the feature points extracted from the first image 10 (the similarity of the feature amount is equal to or higher than the threshold value). is specified, and a pair of the specified feature point and the feature point extracted from the first image 10 is detected as a feature point pair. The first detection unit 2020 detects an arbitrary number of feature point pairs by repeating the processing an arbitrary number of times.
- the flow of processing for detecting feature point pairs is not limited to the flow described above.
- the first detection unit 2020 arbitrarily selects one of the feature points detected from the second image 20 and detects a feature point corresponding to the selected feature point from the first image 10 to detect the feature point. Pairs may be detected.
- the second detection unit 2040 detects derived point pairs for each feature point pair (S106).
- a derived point detected from a feature point on the first image 10 is a point at a first distance in a first direction from the feature point on the first image 10 .
- the derived point detected from the feature point on the second image 20 is a point separated from the feature point on the second image 20 by the second distance in the second direction.
- the first direction, first distance, second direction, and second distance are determined using feature amounts calculated for feature points.
- the main axis direction in the feature amount calculated for the feature points on the first image 10 is used as the first direction.
- the second direction for example, the main axis direction in the feature quantity calculated for the feature points on the second image 20 is used.
- first direction and the second direction may be directions determined based on the main axis direction, and may be directions different from the main axis direction.
- first direction and the second direction may be directions opposite to the direction of the main axis (directions different by 180 degrees) or directions rotated by a predetermined angle (for example, +90 degrees) from the direction of the main axis.
- the first direction is a feature point on the first image 10 included in a certain feature point pair and its derived point, and a feature point on the first image 10 included in another feature point pair and its derived point. are preferably defined so as not to pass through the same straight line. This is because two of the three feature points and two derivation points are linearly dependent in this case.
- the second detection unit 2040 determines whether or not the three feature points and the three derivation points on the first image 10 are located on the same straight line. Derived points may be detected again by changing the first direction. For example, the derived point is detected with the initial value of the first direction set to the direction of the main axis. Then, when two feature points and two derivation points on the first image 10 are positioned on the same straight line, the second detection unit 2040 shifts the first direction from the main axis direction by a predetermined direction, Derived points are detected again.
- An existing technique can be used as a technique for determining whether or not a plurality of points are positioned on one straight line.
- the second detection unit 2040 also uses a similar method so that feature points and derived points detected from the second image 20 are not positioned on one straight line.
- a predetermined multiple of the size of the scale in the feature quantity calculated for the feature points on the first image 10 is used.
- a predetermined multiple of the size of the scale in the feature quantity calculated for the feature points on the second image 20 is used.
- the feature amount is not limited to the scale-invariant feature amount, and may be an affine deformation feature amount.
- the first direction for example, the direction of a specific axis that is determined for feature amounts calculated for feature points on the first image 10 is used.
- the second direction for example, the direction of a specific axis determined for feature amounts calculated for feature points on the second image 20 is used.
- a specific axis is, for example, a short axis or a long axis.
- the first direction and the second direction may be directions opposite to the minor axis direction or the major axis direction (directions different by 180 degrees), or directions rotated by a predetermined angle from the minor axis direction or the major axis direction.
- first direction and the second direction are of the same type. That is, when the first direction is the minor axis direction, the second direction is also the minor axis direction, and when the first direction is the major axis direction, the second direction is also the major axis direction.
- a predetermined multiple of the length of the specific axis determined for the feature amount calculated for the feature points on the first image 10 is used.
- a predetermined multiple of the length of the specific axis determined for the feature quantity calculated for the feature points on the second image 20 is used. The predetermined multiple used for calculating the first distance and the predetermined multiple used for calculating the second distance are equal to each other.
- the second detection unit 2040 detects (p11, q11) and (p12, q12) respectively as derived point pairs.
- the second detection unit 2040 detects four sets of derived points from the feature points on the first image 10 included in the feature point pairs.
- first direction minor axis direction
- first distance k1 times the length of the minor axis
- the first distance k3 times the length of the major axis
- first distance k4 times the length of the major axis.
- k1, k2, k3 and k4 may or may not be equal.
- the second detection unit 2040 also detects four sets of derived points q11, q12, q13, and q14 from the feature points on the second image 20 included in the feature point pairs.
- second direction short axis direction
- second distance k1 times the length of the short axis
- second direction opposite direction to short axis direction
- second Distance k2 times the length of the short axis
- 2nd direction the direction opposite to the major axis direction
- 2nd distance k4 times the length of the major axis.
- the second detection unit 2040 detects (p11, q11), (p12, q12), (p13, q13), and (p14, q14) as derived point pairs.
- the generator 2060 generates the base matrix 40 using six or more corresponding point pairs (feature point pairs and derived point pairs).
- an existing technique can be used as the technique for calculating the basic matrix using six or more corresponding point pairs.
- the fundamental matrix 40 is calculated by solving the optimization problem expressed by Equation (3) below.
- vector f is a vector representation of matrix F (fundamental matrix 40)
- matrix M is a coefficient matrix composed of vector m and vector n.
- Equation (3) can be solved by reducing to the polynomial problem described in Non-Patent Document 2 or Non-Patent Document 3 in the case of 6 points or 7 points, respectively.
- 8 points or more as described in Non-Patent Document 1, it is known that ignoring constraints other than
- ⁇ 2 1 results in a linear least squares method. It is A DLT (Direct Linear Transform) method or the like can be used as a calculation method using the linear least squares method.
- DLT Direct Linear Transform
- the generation unit 2060 may use normalized coordinates instead of using the coordinates of each point included in the corresponding point pair as they are. By doing so, errors in numerical calculation can be reduced.
- normalization of coordinates there is a method of applying similarity transformation so that the mean of coordinate values is zero and the variance is ⁇ 2.
- the generation unit 2060 can generate the base matrix 40 by performing inverse transformation of the similarity transformation on the matrix obtained by a method such as the DLT method. .
- the coordinates of each point of the feature point pair may be normalized before detecting the derived point pair.
- the second detection unit 2040 performs similar conversion on the scale size of the scale-invariant feature quantity and the length of the specific axis of the affine-invariant feature quantity, and then detects derived point pairs.
- Non-Patent Document 2 describes a technique for estimating the focal length as well as the basic matrix using six corresponding point pairs.
- Non-Patent Document 3 describes a technique for estimating a focal length and lens distortion together with a fundamental matrix using seven or more corresponding point pairs. Therefore, by applying these techniques to the fundamental matrix generation device 2000, the generating section 2060 may further estimate the internal parameters in addition to the generation of the fundamental matrix 40.
- FIG. 1 is a technique for estimating the focal length as well as the basic matrix using six corresponding point pairs.
- Non-Patent Document 3 describes a technique for estimating a focal length and lens distortion together with a fundamental matrix using seven or more corresponding point pairs. Therefore, by applying these techniques to the fundamental matrix generation device 2000, the generating section 2060 may further estimate the internal parameters in addition to the generation of the fundamental matrix 40.
- the first detection unit 2020 detects four or more feature point pairs.
- the second detection unit 2040 detects three or more derived point pairs.
- Fundamental matrix generating apparatus 2000 outputs information (hereinafter referred to as output information) including generated fundamental matrix 40 .
- the output mode of the output information is arbitrary.
- the fundamental matrix generation device 2000 displays the output information on a display device accessible from the fundamental matrix generation device 2000 .
- the fundamental matrix generation device 2000 stores the output information in a storage device accessible from the fundamental matrix generation device 2000 .
- fundamental matrix generation device 2000 transmits output information to other devices communicably connected to fundamental matrix generation device 2000 .
- the output information may include only the basic matrix 40, or may further include information other than the basic matrix 40.
- the output information also include information that enables an understanding of whether the basic matrix 40 is a basic matrix that connects which image to which image. Therefore, for example, the output information includes the identifier of the first image 10 and the identifier of the second image 20 as identifiers of images (eg, file names and image data themselves) that are associated with each other by the basic matrix 40 .
- Fundamental matrix generation device 2000 may generate a more accurate fundamental matrix 40 by the following technique.
- the accuracy of the basic matrix 40 here means that the three-dimensional coordinates restored by triangulation using the point mi on the first image 10, the point ni on the second image 20, and the basic matrix are the first image 10 and the The small error between mi and the two-dimensional point reprojected onto the second image 20 and mi and the error between the two-dimensional point reprojected onto the second image 20 and ni means The smaller these reprojection errors are, the more accurate the base matrix 40 is, since the base matrix 40 ensures that the points on the first image 10 and the points on the second image 20 exactly satisfy the geometric constraints. can be said to be high. It should be noted that an algebraic error (for example, Sampson's error) with less computational complexity may be used instead of the reprojection error. These errors are hereinafter collectively referred to as epipolar errors.
- the basic matrix generation device 2000 generates a plurality of basic matrices 40 while variously changing corresponding point pairs used to generate the basic matrix 40 .
- Fundamental matrix generation apparatus 2000 selects the most accurate one from among the plurality of fundamental matrices 40 and outputs output information including the selected fundamental matrix 40 .
- the basic matrix generation device 2000 uses RANSAC to realize the generation of a highly accurate basic matrix 40.
- FIG. 6 is a flow chart illustrating the flow of processing performed by the base matrix generator 2000 using RANSAC.
- the first detection unit 2020 acquires the first image 10 and the second image 20 (S202).
- S204 to S218 are loop processing L1 that is repeatedly executed until the number of times of execution reaches the maximum number of repetitions N.
- FIG. In S204 the basic matrix generation device 2000 determines whether or not the number of executions of the loop process L1 is equal to or greater than the maximum number of iterations N. If the number of executions of the loop process L1 is equal to or greater than the maximum number of iterations N, the process of FIG. 6 proceeds to S220. On the other hand, if the number of executions of the loop process L1 is not equal to or greater than the maximum number of iterations N, the process of FIG. 6 proceeds to S206.
- the first detection unit 2020 detects a plurality of feature point pairs from the first image 10 and the second image 20 (S206).
- the second detection unit 2040 selects at least three arbitrary feature point pairs from among the feature point pairs detected in S206, and detects derived point pairs for each of the selected feature point pairs (S208).
- the generation unit 2060 generates the base matrix 40 using the selected three feature point pairs and the three derived point pairs (that is, six corresponding point pairs) detected using them (S210 ).
- estimation of internal parameters such as focal length and lens distortion may be performed.
- four sets of feature point pairs are used in S210.
- the basic matrix generation device 2000 identifies the number of feature point pairs that satisfy the epipolar constraint by the basic matrix 40 among the plurality of feature point pairs detected in S206 (S212).
- the feature point pair satisfies the epipolar constraint by the base matrix 40 means that the point mi on the first image 10 and the point ni on the second image 20 included in the feature point pair are It means that the defined epipolar error is small enough (eg below a threshold).
- a feature point pair correctly associated by the basic matrix 40 (a feature point pair whose error is less than the threshold) will be referred to as a "correct feature point pair”, and a feature point pair not correctly associated by the basic matrix 40 (the error is equal to or greater than the threshold value) is called an "incorrect feature point pair”.
- the basic matrix generation device 2000 performs, for each feature point pair, 1) the points mi on the first image 10 included in the feature point pair and the points mi included in the feature point pair Calculate the epipolar error with a point ni on the second image 20, and 2) determine whether the calculated error is less than a threshold. Fundamental matrix generation apparatus 2000 then identifies the number of feature point pairs whose error is less than the threshold (that is, correct feature point pairs).
- the basic matrix generation device 2000 determines whether or not the number of correct feature point pairs is the largest among the numbers calculated in the loop processing L1 executed so far. If the number of correct feature point pairs is not the largest number calculated so far (S214: NO), the process of FIG. 6 proceeds to S218. On the other hand, if the number of correct feature point pairs is the largest among the numbers calculated so far (S214: YES), the fundamental matrix generation device 2000 updates the maximum number of iterations of the loop process L1 (S216). .
- Equation (4) the maximum number of iterations is represented, for example, by Equation (4) below.
- N the maximum number of iterations.
- p represents the probability that there exists a feature point pair correctly transformed by the base matrix 40 once in N times.
- s represents the number of corresponding point pairs used to generate the base matrix 40 (3 in the above example).
- ⁇ is the ratio of incorrect feature point pairs to the total number of feature point pairs.
- the fundamental matrix generation device 2000 performs estimation using the maximum number of correct feature point pairs calculated in the loop processing L1 executed so far. Denoting this maximum number as Km and denoting the total number of feature point pairs as Kall, ⁇ can be estimated as (Kall-Km)/Kall.
- the process of FIG. 6 proceeds to S220.
- the fundamental matrix generating device 2000 selects the fundamental matrix 40 generated in the loop process L1 having the largest number of correct feature point pairs among the fundamental matrices 40 generated in each of the loop processes L1 executed multiple times. is included in the output information and output. By doing so, the basic matrix 40 with the highest accuracy among the multiple generated basic matrices 40 is output.
- Non-Patent Document 4 describes a method of using two sets of affine-invariant feature points as a method of generating a basic matrix with fewer than six corresponding point pairs.
- the fundamental matrix is calculated by solving the constraint conditions satisfied by the local affine transformation and the epipolar constraint.
- the fundamental matrix generation device 2000 of this embodiment has the advantage of shortening the overall execution time compared to the technique of Non-Patent Document 4.
- the amount of computation for affine-invariant feature points is generally several times to several tens of times that of scale-invariant feature points. . Therefore, when comparing the overall execution time, the fundamental matrix generation device 2000 of this embodiment is considered to be faster.
- the fundamental matrix generation device 2000 may generate the fundamental matrix 40 only when a specific condition is satisfied instead of generating the fundamental matrix 40 each time in the loop processing L1. Specifically, the fundamental matrix generation device 2000 uses the three feature point pairs selected in S206 and the three derived point pairs detected using them to calculate the signed area. Then, it is determined whether or not to generate the base matrix 40 based on the correctness of the sign of the signed area. A specific description will be given below.
- Equation (5) is equivalent to the determinant of the so-called 3x3 matrix.
- the signs will always be the same as each other. becomes.
- the selected feature point pairs are (m1,n1) and (m2,n2), and the derived point pairs detected using these are (p1,q1) and (p2,q2).
- det(m1,m2,p1) and det(n1 ,n2,q1) is calculated. If all of the six corresponding point pairs are correct corresponding point pairs, the two calculated signed areas have the same sign.
- the fundamental matrix generation device 2000 selects three sets of corresponding point pairs from six sets of corresponding point pairs, performs the above-described signed area calculation for them, and determines whether the signs of the two calculated signed areas are equal. determine whether or not Then, if the sign of the signed area is correct, the basic matrix generation device 2000 executes the processes from S210 onwards. On the other hand, if the sign of the signed area is not correct, the basic matrix generating device 2000 does not generate the basic matrix 40 and returns to the beginning of the loop processing L1.
- FIG. 7 is a diagram illustrating the flowchart of FIG. 6 with the addition of processing for determining whether or not to generate the base matrix 40 using the signed area. The processing for the determination is S302.
- Fundamental matrix generation device 2000 calculates the above-described signed area for each of at least one of these 20 selection methods, and determines whether or not the signs are the same. For example, the fundamental matrix generation device 2000 makes the determination for all 20 patterns. Then, when the signs of the two calculated signed areas are equal in all cases, the basic matrix generation device 2000 generates the basic matrix 40 (in S302, it is determined that the signs of the signed areas are correct). do).
- signed areas may be calculated for three sets of feature point pairs, and derived point pairs may be calculated only when the signs are the same. In this case, first, the determination process is performed in S302, and only if YES, the derivative point pair is calculated in S208, and the processes after S210 are performed.
- the method of improving the precision of the base matrix 40 is not limited to the method of using RANSAC.
- RANSAC has various derivations, it is possible to selectively combine them.
- PROSAC Processive Sample Consensus
- feature point pairs are selected in ascending order of matching scores of feature quantities. That is, in S208, instead of selecting feature point pairs at random, feature point pairs are selected in descending order of matching score of feature amounts (that is, in descending order of feature amount similarity).
- LO-RANSAC Longed RANSAC
- the generating unit 2060 configured to solve the equation (2) using the corresponding point pairs processes may be executed, or a weighted least-squares method such as M-estimator may be used.
- Non-transitory computer readable media include various types of tangible storage media.
- Examples of non-transitory computer-readable media include magnetic recording media (e.g., floppy disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical discs), CD-ROMs, CD-Rs, CD-Rs /W, including semiconductor memory (e.g. mask ROM, PROM (programmable ROM), EPROM (erasable PROM), flash ROM, RAM);
- the program may also be provided to the computer on various types of transitory computer readable medium. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can deliver the program to the computer via wired channels, such as wires and optical fibers, or wireless channels.
- (Appendix 1) a first detection unit that detects three or more pairs of feature points corresponding to each other from the first image and the second image; For each feature point pair, a point located a first distance in a first direction from a point on the first image included in the feature point pair and a point on the second image included in the feature point pair at a first distance a second detection unit that detects a derived point pair that is a pair of points separated by a second distance in two directions; a generation unit that generates a base matrix representing epipolar constraints between points on the first image and points on the second image using each of the detected feature point pairs and derived point pairs.
- Each of the first direction and the first distance is determined based on feature amounts calculated for points on the first image included in the feature point pair
- the basic matrix generation device wherein the second direction and the second distance are each determined based on feature amounts calculated for points on the second image included in the feature point pair.
- the first direction and the first distance are respectively determined based on the principal axis direction and scale length of the scale-invariant feature quantity calculated for the point on the first image
- the fundamental matrix generating device according to appendix 1, wherein the second direction and the second distance are respectively determined based on the principal axis direction and scale length of the scale-invariant feature amount calculated for the point on the second image.
- the first direction and the first distance are each determined based on a specific axial direction and the length of the axis of the affine invariant feature calculated for the point on the first image, 2.
- matrix generator (Appendix 4) Repeating the generation of the basic matrix while changing the feature point pairs used to detect the derived point pairs, and outputting the most accurate one of the plurality of generated basic matrices, from Supplementary Note 1 3.
- the basic matrix generation device according to any one of items 3 to 3.
- (Appendix 5) Calculating a signed area using any three pairs of the plurality of feature point pairs and the plurality of derived point pairs, and generating the basic matrix based on the sign of the calculated signed area 5.
- the fundamental matrix generation device according to any one of appendices 1 to 4, which determines whether or not to perform.
- the generating unit uses each of the detected feature point pairs and derived point pairs to estimate internal parameters of the camera that generated the first image and the camera that generated the second image.
- the basic matrix generation device according to any one of the items.
- a control method implemented by a computer comprising: a first detection step of detecting three or more pairs of feature points corresponding to each other from the first image and the second image; For each feature point pair, a point located a first distance in a first direction from a point on the first image included in the feature point pair and a point on the second image included in the feature point pair at a first distance a second detection step of detecting derived point pairs that are pairs with points separated by a second distance in two directions; using each of the detected feature point pairs and derived point pairs to generate a base matrix representing epipolar constraints between points on the first image and points on the second image.
- Each of the first direction and the first distance is determined based on feature amounts calculated for points on the first image included in the feature point pair,
- the control method wherein the second direction and the second distance are each determined based on feature amounts calculated for points on the second image included in the feature point pair.
- the first direction and the first distance are respectively determined based on the principal axis direction and scale length of the scale-invariant feature quantity calculated for the point on the first image, 8.
- the control method according to claim 7, wherein the second direction and the second distance are respectively determined based on the principal axis direction and scale length of the scale-invariant feature calculated for the point on the second image.
- the first direction and the first distance are each determined based on a specific axial direction and the length of the axis of the affine invariant feature calculated for the point on the first image, 8.
- Method. (Appendix 10) Repeating the generation of the basic matrix while changing the feature point pairs used to detect the derived point pairs, and outputting the one with the highest accuracy among the plurality of generated basic matrices, from Supplementary Note 7 9.
- a computer-readable medium storing a program, The program, in a computer, a first detection step of detecting three or more pairs of feature points corresponding to each other from the first image and the second image; For each feature point pair, a point located a first distance in a first direction from a point on the first image included in the feature point pair and a point on the second image included in the feature point pair at a first distance a second detection step of detecting derived point pairs that are pairs with points separated by a second distance in two directions; and a generating step of using each of the detected feature point pairs and derived point pairs to generate a base matrix representing epipolar constraints between points on the first image and points on the second image.
- Each of the first direction and the first distance is determined based on feature amounts calculated for points on the first image included in the feature point pair
- the first direction and the first distance are respectively determined based on the principal axis direction and scale length of the scale-invariant feature quantity calculated for the point on the first image, 14.
- the computer-readable medium of Clause 13 wherein the second direction and the second distance are determined based on principal axis directions and scale lengths of scale-invariant features calculated for points on the second image, respectively.
- the first direction and the first distance are each determined based on a specific axial direction and the length of the axis of the affine invariant feature calculated for the point on the first image, 14.
- the computer of Claim 13 wherein the second direction and the second distance are each determined based on a particular axial direction and axial length of an affine-invariant feature calculated for a point on the second image.
- readable medium The computer repeatedly generates the base matrix while changing the feature point pairs used to detect the derived point pairs, and outputs the base matrix with the highest accuracy among the plurality of generated base matrices. 16.
- the computer readable medium of any one of clauses 13-15 causing the steps to be performed.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
Description
非特許文献1の発明では、本開示における第1画像10と第2画像20について、特徴点のペアを8組以上利用して、基礎行列40の生成が行われる。これに対し、本実施形態の基礎行列生成装置2000では、特徴点ペアと派生点ペアが合計で6組以上あれば、基礎行列40を生成できる。そのため、画像から検出する必要がある特徴点ペアの最小数は3組である。よって、特許文献1の発明と比較し、画像から検出する必要がある特徴点ペアの数が少ないという利点がある。
図3は、実施形態1の基礎行列生成装置2000の機能構成を例示するブロック図である。基礎行列生成装置2000は、第1検出部2020、第2検出部2040、及び生成部2060を有する。第1検出部2020は、第1画像10及び第2画像20から、3組以上の特徴点ペアを検出する。第2検出部2040は、3組以上の特徴点ペアそれぞれを用いて、第1画像10及び第2画像20から、3組以上の派生点ペアを検出する。生成部2060は、検出した特徴点ペアと派生点ペアを用いて、基礎行列40を生成する。
基礎行列生成装置2000の各機能構成部は、各機能構成部を実現するハードウエア(例:ハードワイヤードされた電子回路など)で実現されてもよいし、ハードウエアとソフトウエアとの組み合わせ(例:電子回路とそれを制御するプログラムの組み合わせなど)で実現されてもよい。以下、基礎行列生成装置2000の各機能構成部がハードウエアとソフトウエアとの組み合わせで実現される場合について、さらに説明する。
図4は、実施形態1の基礎行列生成装置2000によって実行される処理の流れを例示するフローチャートである。第1検出部2020は第1画像10及び第2画像20を取得する(S102)。第1検出部2020は、第1画像10及び第2画像20を用いて、3組以上の特徴点ペアを検出する(S104)。第2検出部2040は、第1画像10及び第2画像20を用いて、各特徴点ペアについての派生点ペアを検出する(S106)。生成部2060は、特徴点ペアと派生点ペアを用いて、基礎行列40を生成する(S108)。
第1画像10と第2画像20は、任意のカメラによって生成された任意の撮像画像である。ただし、第1画像10と第2画像20は、少なくともその一部に、互いに同一の場所が撮像された画像領域が含まれている。例えば、同じ建物や人物を互いに異なる位置や角度から撮像することにより、第1画像10と第2画像20が生成される。
第1検出部2020は、第1画像10及び第2画像20を取得する(S102)。第1検出部2020が第1画像10と第2画像20を取得する方法は任意である。例えば第1検出部2020は、第1画像10と第2画像20を、それぞれが格納されている記憶装置から取得する。なお、第1画像10と第2画像20は、同じ記憶装置に格納されていてもよいし、互いに異なる記憶装置に格納されていてもよい。その他にも例えば、第1検出部2020は、第1画像10を生成したカメラと第2画像20を生成したカメラのそれぞれから、第1画像10と第2画像20を取得してもよい。
第1検出部2020は、第1画像10と第2画像20から特徴点ペアを3組以上検出する(S104)。そのために、第1検出部2020は、第1画像10と第2画像20のそれぞれから、特徴点の検出を行う。ここで、第1画像10と第2画像20から検出する特徴点は、任意の種類の特徴点でよい。また、画像から特徴点を検出する技術には、既存の技術を利用することができる。
第2検出部2040は、各特徴点ペアについて、派生点ペアを検出する(S106)。第1画像10上の特徴点から検出される派生点は、第1画像10上の特徴点から第1方向に第1距離離れた点である。一方、第2画像20上の特徴点から検出される派生点は、第2画像20上の特徴点から第2方向に第2距離離れた点である。
生成部2060は、6組以上の対応点ペア(特徴点ペアと派生点ペア)を用いて、基礎行列40を生成する。ここで、対応点ペアを6組以上用いて基礎行列を算出する技術には、既存の技術を利用することができる。
前述したように、非特許文献2には、6組の対応点ペアを用いて、基礎行列と共に、焦点距離も推定する技術が記載されている。また、非特許文献3には、7組以上の対応点ペアを用いて、基礎行列と共に、焦点距離とレンズ歪を推定する技術が記載されている。そこで、これらの技術を基礎行列生成装置2000へ適用することにより、生成部2060が、基礎行列40の生成に加え、内部パラメータの推定をさらに行うようにしてもよい。
基礎行列生成装置2000は、生成した基礎行列40を含む情報(以下、出力情報)を出力する。出力情報の出力態様は任意である。例えば基礎行列生成装置2000は、出力情報を、基礎行列生成装置2000からアクセス可能なディスプレイ装置に表示させる。その他にも例えば、基礎行列生成装置2000は、出力情報を、基礎行列生成装置2000からアクセス可能な記憶装置に格納する。その他にも例えば、基礎行列生成装置2000は、出力情報を、基礎行列生成装置2000と通信可能に接続されている他の装置へ送信する。
基礎行列生成装置2000は、以下の手法により、より精度の高い基礎行列40を生成してもよい。ここでいう基礎行列40の精度とは、第1画像10上の点 mi と第2画像20上の点 ni と基礎行列を用いて三角測量して復元した三次元座標を、第1画像10と第2画像20へと再投影し、第1画像10上に再投影された2次元点と mi との誤差および第2画像20上に再投影された2次元点と ni との誤差の小ささを意味する。これらの再投影誤差が小さいほど、基礎行列40によって、第1画像10上の点と第2画像20上の点とが正確に幾何的な制約条件を満たすこととなるため、基礎行列40の精度が高いと言える。なお、再投影誤差の代わりに、計算量がより少ない代数学的誤差(例えばSampson誤差)を用いてもよい。以下では、これらの誤差を総称してエピポーラ誤差と呼ぶ。
基礎行列生成装置2000は、ループ処理L1において毎回基礎行列40の生成を行うのではなく、特定の条件が満たされた場合のみ、基礎行列40の生成を行うようにしてもよい。具体的には、基礎行列生成装置2000は、S206で選択された3組の特徴点ペア、及びそれらを用いて検出された3組の派生点ペアを利用して、符号付面積を算出する。そして、符号付き面積の符号の正しさに基づいて、基礎行列40の生成を行うか否かを判定する。以下、具体的に説明する。
基礎行列40の精度を高める方法は、RANSAC を利用する方法に限定されない。例えば、RANSAC には様々な派生が存在するため、それらを選択的に組み合わせることが可能である。例えば、PROSAC(Progressive Sample Consensus)を用いる場合、特徴量のマッチングスコアが小さい順に特徴点ペアが選択される。すなわち、S208において、特徴点ペアがランダムに選択される代わりに、特徴量のマッチングスコアが小さい(すなわち、互いの特徴量の類似度合いが大きい)順に特徴点ペアが選択される。
(付記1)
第1画像と第2画像から、互いに対応する特徴点のペアである特徴点ペアを3組以上検出する第1検出部と、
各前記特徴点ペアについて、その特徴点ペアに含まれる前記第1画像上の点から第1方向に第1距離離れた点と、その特徴点ペアに含まれる前記第2画像上の点から第2方向に第2距離離れた点とのペアである派生点ペアを検出する第2検出部と、
各前記検出された特徴点ペア及び派生点ペアを利用して、前記第1画像上の点と前記第2画像上の点とのエピポーラ制約を表す基礎行列を生成する生成部と、を有し、
前記第1方向と前記第1距離はそれぞれ、前記特徴点ペアに含まれる前記第1画像上の点について算出された特徴量に基づいて定まり、
前記第2方向と前記第2距離はそれぞれ、前記特徴点ペアに含まれる前記第2画像上の点について算出された特徴量に基づいて定まる、基礎行列生成装置。
(付記2)
前記第1方向と第1距離はそれぞれ、前記第1画像上の点について算出されたスケール不変特徴量の主軸方向とスケールの長さに基づいて決定され、
前記第2方向と第2距離はそれぞれ、前記第2画像上の点について算出されたスケール不変特徴量の主軸方向とスケールの長さに基づいて決定される、付記1に記載の基礎行列生成装置。
(付記3)
前記第1方向と第1距離はそれぞれ、前記第1画像上の点について算出されたアフィン不変特徴量の特定の軸方向及びその軸の長さに基づいて決定され、
前記第2方向と第2距離はそれぞれ、前記第2画像上の点について算出されたアフィン不変特徴量の特定の軸方向及びその軸の長さに基づいて決定される、付記1に記載の基礎行列生成装置。
(付記4)
前記派生点ペアの検出に利用する前記特徴点ペアを変更しながら、前記基礎行列の生成を繰り返し行い、生成された複数の前記基礎行列のうち、最も精度が高いものを出力する、付記1から3いずれか一項に記載の基礎行列生成装置。
(付記5)
複数の前記特徴点ペア及び複数の前記派生点ペアのうちのいずれか3組を用いて符号付面積の算出を行い、当該算出された符号付面積の符号に基づいて、前記基礎行列の生成を行うか否かを判定する、付記1から4いずれか一項に記載の基礎行列生成装置。
(付記6)
前記生成部は、各前記検出された特徴点ペア及び派生点ペアを利用して、前記第1画像を生成したカメラ及び前記第2画像を生成したカメラの内部パラメータを推定する、付記1から5いずれか一項に記載の基礎行列生成装置。
(付記7)
コンピュータによって実行される制御方法であって、
第1画像と第2画像から、互いに対応する特徴点のペアである特徴点ペアを3組以上検出する第1検出ステップと、
各前記特徴点ペアについて、その特徴点ペアに含まれる前記第1画像上の点から第1方向に第1距離離れた点と、その特徴点ペアに含まれる前記第2画像上の点から第2方向に第2距離離れた点とのペアである派生点ペアを検出する第2検出ステップと、
各前記検出された特徴点ペア及び派生点ペアを利用して、前記第1画像上の点と前記第2画像上の点とのエピポーラ制約を表す基礎行列を生成する生成ステップと、を有し、
前記第1方向と前記第1距離はそれぞれ、前記特徴点ペアに含まれる前記第1画像上の点について算出された特徴量に基づいて定まり、
前記第2方向と前記第2距離はそれぞれ、前記特徴点ペアに含まれる前記第2画像上の点について算出された特徴量に基づいて定まる、制御方法。
(付記8)
前記第1方向と第1距離はそれぞれ、前記第1画像上の点について算出されたスケール不変特徴量の主軸方向とスケールの長さに基づいて決定され、
前記第2方向と第2距離はそれぞれ、前記第2画像上の点について算出されたスケール不変特徴量の主軸方向とスケールの長さに基づいて決定される、付記7に記載の制御方法。
(付記9)
前記第1方向と第1距離はそれぞれ、前記第1画像上の点について算出されたアフィン不変特徴量の特定の軸方向及びその軸の長さに基づいて決定され、
前記第2方向と第2距離はそれぞれ、前記第2画像上の点について算出されたアフィン不変特徴量の特定の軸方向及びその軸の長さに基づいて決定される、付記7に記載の制御方法。
(付記10)
前記派生点ペアの検出に利用する前記特徴点ペアを変更しながら、前記基礎行列の生成を繰り返し行い、生成された複数の前記基礎行列のうち、最も精度が高いものを出力する、付記7から9いずれか一項に記載の制御方法。
(付記11)
複数の前記特徴点ペア及び複数の前記派生点ペアのうちのいずれか3組について符号付面積の算出を行い、当該算出された符号付面積の符号に基づいて、前記基礎行列の生成を行うか否かを判定する、付記7から10いずれか一項に記載の制御方法。
(付記12)
前記生成ステップにおいて、各前記検出された特徴点ペア及び派生点ペアを利用して、前記第1画像を生成したカメラ及び前記第2画像を生成したカメラの内部パラメータを推定する、付記7から11いずれか一項に記載の制御方法。
(付記13)
プログラムが格納されているコンピュータ可読媒体であって、
前記プログラムは、コンピュータに、
第1画像と第2画像から、互いに対応する特徴点のペアである特徴点ペアを3組以上検出する第1検出ステップと、
各前記特徴点ペアについて、その特徴点ペアに含まれる前記第1画像上の点から第1方向に第1距離離れた点と、その特徴点ペアに含まれる前記第2画像上の点から第2方向に第2距離離れた点とのペアである派生点ペアを検出する第2検出ステップと、
各前記検出された特徴点ペア及び派生点ペアを利用して、前記第1画像上の点と前記第2画像上の点とのエピポーラ制約を表す基礎行列を生成する生成ステップと、を実行させ、
前記第1方向と前記第1距離はそれぞれ、前記特徴点ペアに含まれる前記第1画像上の点について算出された特徴量に基づいて定まり、
前記第2方向と前記第2距離はそれぞれ、前記特徴点ペアに含まれる前記第2画像上の点について算出された特徴量に基づいて定まる、コンピュータ可読媒体。
(付記14)
前記第1方向と第1距離はそれぞれ、前記第1画像上の点について算出されたスケール不変特徴量の主軸方向とスケールの長さに基づいて決定され、
前記第2方向と第2距離はそれぞれ、前記第2画像上の点について算出されたスケール不変特徴量の主軸方向とスケールの長さに基づいて決定される、付記13に記載のコンピュータ可読媒体。
(付記15)
前記第1方向と第1距離はそれぞれ、前記第1画像上の点について算出されたアフィン不変特徴量の特定の軸方向及びその軸の長さに基づいて決定され、
前記第2方向と第2距離はそれぞれ、前記第2画像上の点について算出されたアフィン不変特徴量の特定の軸方向及びその軸の長さに基づいて決定される、付記13に記載のコンピュータ可読媒体。
(付記16)
前記コンピュータに、前記派生点ペアの検出に利用する前記特徴点ペアを変更しながら、前記基礎行列の生成を繰り返し行い、生成された複数の前記基礎行列のうち、最も精度が高いものを出力するステップを実行させる、付記13から15いずれか一項に記載のコンピュータ可読媒体。
(付記17)
前記コンピュータに、複数の前記特徴点ペア及び複数の前記派生点ペアのうちのいずれか3組について符号付面積の算出を行い、当該算出された符号付面積の符号に基づいて、前記基礎行列の生成を行うか否かを判定するステップを実行させる、付記13から16いずれか一項に記載のコンピュータ可読媒体。
(付記18)
前記生成ステップにおいて、各前記検出された特徴点ペア及び派生点ペアを利用して、前記第1画像を生成したカメラ及び前記第2画像を生成したカメラの内部パラメータを推定する、付記13から17いずれか一項に記載のコンピュータ可読媒体。
20 第2画像
40 基礎行列
500 コンピュータ
502 バス
504 プロセッサ
506 メモリ
508 ストレージデバイス
510 入出力インタフェース
512 ネットワークインタフェース
2000 基礎行列生成装置
2020 第1検出部
2040 第2検出部
2060 生成部
Claims (18)
- 第1画像と第2画像から、互いに対応する特徴点のペアである特徴点ペアを3組以上検出する第1検出部と、
各前記特徴点ペアについて、その特徴点ペアに含まれる前記第1画像上の点から第1方向に第1距離離れた点と、その特徴点ペアに含まれる前記第2画像上の点から第2方向に第2距離離れた点とのペアである派生点ペアを検出する第2検出部と、
各前記検出された特徴点ペア及び派生点ペアを利用して、前記第1画像上の点と前記第2画像上の点とのエピポーラ制約を表す基礎行列を生成する生成部と、を有し、
前記第1方向と前記第1距離はそれぞれ、前記特徴点ペアに含まれる前記第1画像上の点について算出された特徴量に基づいて定まり、
前記第2方向と前記第2距離はそれぞれ、前記特徴点ペアに含まれる前記第2画像上の点について算出された特徴量に基づいて定まる、基礎行列生成装置。 - 前記第1方向と第1距離はそれぞれ、前記第1画像上の点について算出されたスケール不変特徴量の主軸方向とスケールの長さに基づいて決定され、
前記第2方向と第2距離はそれぞれ、前記第2画像上の点について算出されたスケール不変特徴量の主軸方向とスケールの長さに基づいて決定される、請求項1に記載の基礎行列生成装置。 - 前記第1方向と第1距離はそれぞれ、前記第1画像上の点について算出されたアフィン不変特徴量の特定の軸方向及びその軸の長さに基づいて決定され、
前記第2方向と第2距離はそれぞれ、前記第2画像上の点について算出されたアフィン不変特徴量の特定の軸方向及びその軸の長さに基づいて決定される、請求項1に記載の基礎行列生成装置。 - 前記派生点ペアの検出に利用する前記特徴点ペアを変更しながら、前記基礎行列の生成を繰り返し行い、生成された複数の前記基礎行列のうち、最も精度が高いものを出力する、請求項1から3いずれか一項に記載の基礎行列生成装置。
- 複数の前記特徴点ペア及び複数の前記派生点ペアのうちのいずれか3組を用いて符号付面積の算出を行い、当該算出された符号付面積の符号に基づいて、前記基礎行列の生成を行うか否かを判定する、請求項1から4いずれか一項に記載の基礎行列生成装置。
- 前記生成部は、各前記検出された特徴点ペア及び派生点ペアを利用して、前記第1画像を生成したカメラ及び前記第2画像を生成したカメラの内部パラメータを推定する、請求項1から5いずれか一項に記載の基礎行列生成装置。
- コンピュータによって実行される制御方法であって、
第1画像と第2画像から、互いに対応する特徴点のペアである特徴点ペアを3組以上検出する第1検出ステップと、
各前記特徴点ペアについて、その特徴点ペアに含まれる前記第1画像上の点から第1方向に第1距離離れた点と、その特徴点ペアに含まれる前記第2画像上の点から第2方向に第2距離離れた点とのペアである派生点ペアを検出する第2検出ステップと、
各前記検出された特徴点ペア及び派生点ペアを利用して、前記第1画像上の点と前記第2画像上の点とのエピポーラ制約を表す基礎行列を生成する生成ステップと、を有し、
前記第1方向と前記第1距離はそれぞれ、前記特徴点ペアに含まれる前記第1画像上の点について算出された特徴量に基づいて定まり、
前記第2方向と前記第2距離はそれぞれ、前記特徴点ペアに含まれる前記第2画像上の点について算出された特徴量に基づいて定まる、制御方法。 - 前記第1方向と第1距離はそれぞれ、前記第1画像上の点について算出されたスケール不変特徴量の主軸方向とスケールの長さに基づいて決定され、
前記第2方向と第2距離はそれぞれ、前記第2画像上の点について算出されたスケール不変特徴量の主軸方向とスケールの長さに基づいて決定される、請求項7に記載の制御方法。 - 前記第1方向と第1距離はそれぞれ、前記第1画像上の点について算出されたアフィン不変特徴量の特定の軸方向及びその軸の長さに基づいて決定され、
前記第2方向と第2距離はそれぞれ、前記第2画像上の点について算出されたアフィン不変特徴量の特定の軸方向及びその軸の長さに基づいて決定される、請求項7に記載の制御方法。 - 前記派生点ペアの検出に利用する前記特徴点ペアを変更しながら、前記基礎行列の生成を繰り返し行い、生成された複数の前記基礎行列のうち、最も精度が高いものを出力する、請求項7から9いずれか一項に記載の制御方法。
- 複数の前記特徴点ペア及び複数の前記派生点ペアのうちのいずれか3組について符号付面積の算出を行い、当該算出された符号付面積の符号に基づいて、前記基礎行列の生成を行うか否かを判定する、請求項7から10いずれか一項に記載の制御方法。
- 前記生成ステップにおいて、各前記検出された特徴点ペア及び派生点ペアを利用して、前記第1画像を生成したカメラ及び前記第2画像を生成したカメラの内部パラメータを推定する、請求項7から11いずれか一項に記載の制御方法。
- プログラムが格納されているコンピュータ可読媒体であって、
前記プログラムは、コンピュータに、
第1画像と第2画像から、互いに対応する特徴点のペアである特徴点ペアを3組以上検出する第1検出ステップと、
各前記特徴点ペアについて、その特徴点ペアに含まれる前記第1画像上の点から第1方向に第1距離離れた点と、その特徴点ペアに含まれる前記第2画像上の点から第2方向に第2距離離れた点とのペアである派生点ペアを検出する第2検出ステップと、
各前記検出された特徴点ペア及び派生点ペアを利用して、前記第1画像上の点と前記第2画像上の点とのエピポーラ制約を表す基礎行列を生成する生成ステップと、を実行させ、
前記第1方向と前記第1距離はそれぞれ、前記特徴点ペアに含まれる前記第1画像上の点について算出された特徴量に基づいて定まり、
前記第2方向と前記第2距離はそれぞれ、前記特徴点ペアに含まれる前記第2画像上の点について算出された特徴量に基づいて定まる、コンピュータ可読媒体。 - 前記第1方向と第1距離はそれぞれ、前記第1画像上の点について算出されたスケール不変特徴量の主軸方向とスケールの長さに基づいて決定され、
前記第2方向と第2距離はそれぞれ、前記第2画像上の点について算出されたスケール不変特徴量の主軸方向とスケールの長さに基づいて決定される、請求項13に記載のコンピュータ可読媒体。 - 前記第1方向と第1距離はそれぞれ、前記第1画像上の点について算出されたアフィン不変特徴量の特定の軸方向及びその軸の長さに基づいて決定され、
前記第2方向と第2距離はそれぞれ、前記第2画像上の点について算出されたアフィン不変特徴量の特定の軸方向及びその軸の長さに基づいて決定される、請求項13に記載のコンピュータ可読媒体。 - 前記コンピュータに、前記派生点ペアの検出に利用する前記特徴点ペアを変更しながら、前記基礎行列の生成を繰り返し行い、生成された複数の前記基礎行列のうち、最も精度が高いものを出力するステップを実行させる、請求項13から15いずれか一項に記載のコンピュータ可読媒体。
- 前記コンピュータに、複数の前記特徴点ペア及び複数の前記派生点ペアのうちのいずれか3組について符号付面積の算出を行い、当該算出された符号付面積の符号に基づいて、前記基礎行列の生成を行うか否かを判定するステップを実行させる、請求項13から16いずれか一項に記載のコンピュータ可読媒体。
- 前記生成ステップにおいて、各前記検出された特徴点ペア及び派生点ペアを利用して、前記第1画像を生成したカメラ及び前記第2画像を生成したカメラの内部パラメータを推定する、請求項13から17いずれか一項に記載のコンピュータ可読媒体。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2023504933A JP7477045B2 (ja) | 2021-03-09 | 2021-03-09 | 基礎行列生成装置、制御方法、及びプログラム |
PCT/JP2021/009391 WO2022190224A1 (ja) | 2021-03-09 | 2021-03-09 | 基礎行列生成装置、制御方法、及びコンピュータ可読媒体 |
US18/280,415 US20240153233A1 (en) | 2021-03-09 | 2021-03-09 | Fundamental matrix generation apparatus, control method, and computer-readable medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2021/009391 WO2022190224A1 (ja) | 2021-03-09 | 2021-03-09 | 基礎行列生成装置、制御方法、及びコンピュータ可読媒体 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022190224A1 true WO2022190224A1 (ja) | 2022-09-15 |
Family
ID=83226443
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2021/009391 WO2022190224A1 (ja) | 2021-03-09 | 2021-03-09 | 基礎行列生成装置、制御方法、及びコンピュータ可読媒体 |
Country Status (3)
Country | Link |
---|---|
US (1) | US20240153233A1 (ja) |
JP (1) | JP7477045B2 (ja) |
WO (1) | WO2022190224A1 (ja) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2016100899A (ja) * | 2014-11-20 | 2016-05-30 | 三星電子株式会社Samsung Electronics Co.,Ltd. | 映像補正方法及び装置 |
JP2017011431A (ja) * | 2015-06-19 | 2017-01-12 | カシオ計算機株式会社 | 画像処理装置、画像処理方法及びプログラム |
JP2017174105A (ja) * | 2016-03-23 | 2017-09-28 | Kddi株式会社 | 幾何検証装置及びプログラム |
-
2021
- 2021-03-09 WO PCT/JP2021/009391 patent/WO2022190224A1/ja active Application Filing
- 2021-03-09 JP JP2023504933A patent/JP7477045B2/ja active Active
- 2021-03-09 US US18/280,415 patent/US20240153233A1/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2016100899A (ja) * | 2014-11-20 | 2016-05-30 | 三星電子株式会社Samsung Electronics Co.,Ltd. | 映像補正方法及び装置 |
JP2017011431A (ja) * | 2015-06-19 | 2017-01-12 | カシオ計算機株式会社 | 画像処理装置、画像処理方法及びプログラム |
JP2017174105A (ja) * | 2016-03-23 | 2017-09-28 | Kddi株式会社 | 幾何検証装置及びプログラム |
Also Published As
Publication number | Publication date |
---|---|
US20240153233A1 (en) | 2024-05-09 |
JP7477045B2 (ja) | 2024-05-01 |
JPWO2022190224A1 (ja) | 2022-09-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Darom et al. | Scale-invariant features for 3-D mesh models | |
US9305240B2 (en) | Motion aligned distance calculations for image comparisons | |
JP2018195309A (ja) | 顔認識のための画像処理装置の訓練方法及び訓練装置 | |
CN112348863B (zh) | 图像对齐方法、图像对齐装置及终端设备 | |
US10311576B2 (en) | Image processing device and image processing method | |
CN113870215B (zh) | 中线提取方法及装置 | |
Loaiza et al. | Multi-camera calibration based on an invariant pattern | |
JP2014102746A (ja) | 被写体認識装置及び被写体認識プログラム | |
US8548225B2 (en) | Point selection in bundle adjustment | |
WO2022190224A1 (ja) | 基礎行列生成装置、制御方法、及びコンピュータ可読媒体 | |
CN117372604A (zh) | 一种3d人脸模型生成方法、装置、设备及可读存储介质 | |
US11210551B2 (en) | Iterative multi-directional image search supporting large template matching | |
WO2022185462A1 (ja) | 基本行列生成装置、制御方法、及びコンピュータ可読媒体 | |
Jiang et al. | Probabilistic Triangulation for Uncalibrated Multi-View 3D Human Pose Estimation | |
CN113643328B (zh) | 标定物的重建方法、装置、电子设备及计算机可读介质 | |
JP7448034B2 (ja) | 平面射影変換行列生成装置、制御方法、及びプログラム | |
Boukamcha et al. | A real-time auto calibration technique for stereo camera | |
JP2019105992A (ja) | 画像処理装置、画像処理プログラム及び画像処理方法 | |
CN110603535B (zh) | 支持大模板匹配的迭代多方向图像搜索 | |
JP5814714B2 (ja) | 画像類似度判定装置、画像類似度判定方法及び画像類似度判定プログラム | |
CN108121994B (zh) | 在检测目标形状中进行特征提取的方法和装置 | |
WO2023166618A1 (ja) | カメラパラメータ推定装置、カメラパラメータ推定方法、及びコンピュータ読み取り可能な記録媒体 | |
Garcke et al. | Alignment of highly resolved time-dependent experimental and simulated crash test data | |
Liu et al. | A geometry-based error estimation for cross-ratios | |
Liu et al. | Algorithm for camera parameter adjustment in multicamera systems |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21930081 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2023504933 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 18280415 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21930081 Country of ref document: EP Kind code of ref document: A1 |