WO2020217360A1 - カメラパラメータ推定装置、カメラパラメータ推定方法、及びコンピュータ読み取り可能な記録媒体 - Google Patents

カメラパラメータ推定装置、カメラパラメータ推定方法、及びコンピュータ読み取り可能な記録媒体 Download PDF

Info

Publication number
WO2020217360A1
WO2020217360A1 PCT/JP2019/017508 JP2019017508W WO2020217360A1 WO 2020217360 A1 WO2020217360 A1 WO 2020217360A1 JP 2019017508 W JP2019017508 W JP 2019017508W WO 2020217360 A1 WO2020217360 A1 WO 2020217360A1
Authority
WO
WIPO (PCT)
Prior art keywords
coordinate system
camera
dimensional coordinates
local coordinate
rotation matrix
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2019/017508
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
中野　学
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to JP2021515390A priority Critical patent/JP7099627B2/ja
Priority to PCT/JP2019/017508 priority patent/WO2020217360A1/ja
Priority to US17/604,837 priority patent/US11935266B2/en
Publication of WO2020217360A1 publication Critical patent/WO2020217360A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/20—Linear translation of whole images or parts thereof, e.g. panning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/60—Rotation of whole images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/536—Depth or shape recovery from perspective effects, e.g. by using vanishing points
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Definitions

  • the present invention relates to a camera parameter estimation device and a camera parameter estimation method for estimating external parameters of a camera whose internal parameters are known, and further relates to a computer-readable recording medium on which a program for realizing these is recorded. ..
  • the technology for estimating camera parameters from images obtained by observing known three-dimensional coordinates is an important elemental technology in robot self-position estimation and image synthesis.
  • camera parameters there are two types of camera parameters, an external parameter and an internal parameter.
  • the external parameters are parameters representing the position and orientation of the camera in the three-dimensional space, that is, a rotation matrix and a translation vector.
  • the internal parameters are parameters such as the focal length of the lens, the optical center, the aspect ratio, the shear coefficient, and the lens distortion coefficient.
  • a pair of known three-dimensional coordinates and a two-dimensional point obtained by observing the three-dimensional coordinates on an image is simply referred to as a "corresponding point".
  • the internal parameters may be the widely known Tsai method, Zhang method, etc. It can be used to measure in advance.
  • Cameras with known internal parameters (hereinafter simply referred to as "calibrated cameras") are used in applications such as automatic driving of robots and automobiles. Then, in such a camera, it is important to calculate the external parameters when the camera moves with a low calculation amount. It is known that the external parameters of a calibrated camera can be calculated given at least three sets of correspondence points, which is called the P3P (Perspective-three-point) problem.
  • P3P Perspective-three-point
  • Non-Patent Document 1 and Non-Patent Document 2 disclose a method for solving a P3P problem.
  • the rotation matrix and the translation are obtained from the projective conversion equation formed by the depth to each three-dimensional coordinate.
  • the vector is erased and a quadratic equation for one depth is obtained.
  • three depths are calculated by solving the quartic equation.
  • the depth is substituted into the projective transformation formula, and the rotation matrix and translation vector, which are external parameters, are calculated by the singular value decomposition.
  • Non-Patent Document 1 has a problem that the stability of numerical calculation is low because the amount of calculation is high because singular value decomposition is used and the calculation takes time.
  • Non-Patent Document 2 can calculate at a higher speed than the method described in Non-Patent Document 1 because it does not use singular value decomposition, but it requires two steps for conversion to the local coordinate system. It takes. Therefore, even with the method disclosed in Non-Patent Document 2, the problem of low stability of numerical calculation still remains.
  • An example of an object of the present invention is a camera parameter estimation device, a camera parameter estimation method, and a computer-readable object that can solve the above problems and estimate external parameters at high speed in one step without using singular value decomposition.
  • the purpose is to provide a recording medium.
  • the camera parameter estimation device in one aspect of the present invention is By inputting three sets of three-dimensional coordinates related to the object photographed by the camera and two-dimensional coordinates corresponding to each of the three-dimensional coordinates on the captured image, A three-dimensional coordinate conversion unit that converts the coordinate system of the three-dimensional coordinates from the world coordinate system to a local coordinate system having one of the three-dimensional coordinates as the origin. A linear transformation matrix is calculated based on the projective transformation formula from the 3D coordinates converted to the local coordinate system to the 2D coordinates, and out of the depths from the center of the camera to each of the 3D coordinates.
  • a depth calculation unit that calculates each of the depths by calculating the coefficient of the quaternary equation for any one of the above and solving the quaternary equation.
  • a rotation matrix calculation unit that calculates the rotation matrix of the camera in the local coordinate system using each of the depths and the linear transformation matrix.
  • a translation vector calculation unit that calculates the translation vector of the camera in the local coordinate system from each of the depths based on the projection conversion formula. It is characterized by including an inverse transforming unit that inversely transforms the rotation matrix and the translation vector of the camera in the local coordinate system to calculate the rotation matrix and the translation vector of the camera in the world coordinate system. And.
  • the camera parameter estimation method in one aspect of the present invention is: (A) Three sets of three-dimensional coordinates related to the object photographed by the camera and two-dimensional coordinates corresponding to each of the three-dimensional coordinates on the photographed image are input. A step of converting the coordinate system of the three-dimensional coordinates from the world coordinate system to the local coordinate system having one of the three-dimensional coordinates as the origin. (B) A linear conversion matrix is calculated based on the projection conversion formula from the three-dimensional coordinates converted to the local coordinate system to the two-dimensional coordinates, and from the center of the camera to each of the three-dimensional coordinates.
  • Steps and steps that calculate each of the depths by calculating the coefficients of the quadratic equation for any one of the depths and solving the quadratic equation are characterized by having a step of inversely transforming the rotation matrix and translation vector of the camera in the local coordinate system to calculate the rotation matrix and translation vector of the camera in the world coordinate system. To do.
  • the computer-readable recording medium in one aspect of the present invention is used.
  • On the computer (A) Three sets of three-dimensional coordinates related to the object photographed by the camera and two-dimensional coordinates corresponding to each of the three-dimensional coordinates on the photographed image are input. A step of converting the coordinate system of the three-dimensional coordinates from the world coordinate system to the local coordinate system having one of the three-dimensional coordinates as the origin. (B) A linear conversion matrix is calculated based on the projection conversion formula from the three-dimensional coordinates converted to the local coordinate system to the two-dimensional coordinates, and from the center of the camera to each of the three-dimensional coordinates.
  • Steps and steps that calculate each of the depths by calculating the coefficients of the quadratic equation for any one of the depths and solving the quadratic equation are characterized by recording a program.
  • external parameters can be estimated at high speed in one step without using singular value decomposition.
  • FIG. 1 is a diagram showing an example of a coordinate system used in the embodiment of the present invention.
  • FIG. 2 is a block diagram showing a configuration of a camera parameter estimation device according to an embodiment of the present invention.
  • FIG. 3 is a flow chart showing the operation of the camera parameter estimation device 10 according to the embodiment of the present invention.
  • FIG. 4 is a block diagram showing an example of a computer that realizes the camera parameter estimation device according to the embodiment of the present invention.
  • FIG. 1 is a diagram showing an example of a coordinate system used in the embodiment of the present invention.
  • three three-dimensional coordinates X 1 , X 2 , and X 3 are image coordinates, respectively, by a camera located at a position represented by a rotation matrix R and a translation vector t with respect to the origin of the world coordinate system.
  • the state of observation as m 1 , m 2 , and m 3 is shown.
  • image coordinate a certain three-dimensional coordinate and the corresponding two-dimensional coordinate on the image (hereinafter referred to as "image coordinate") are collectively referred to as a corresponding point.
  • image coordinate a certain three-dimensional coordinate and the corresponding two-dimensional coordinate on the image
  • FIG. 2 is a block diagram showing a configuration of a camera parameter estimation device according to an embodiment of the present invention.
  • the camera parameter estimation device 10 is a device that estimates the external parameters of the camera, that is, the rotation matrix and translation vector of the camera when the object is photographed by the camera.
  • the camera parameter estimation device 10 includes a three-dimensional coordinate conversion unit 11, a depth calculation unit 12, a rotation matrix calculation unit 13, a translation vector calculation unit 14, and an inverse conversion unit 15. There is.
  • the three-dimensional coordinate conversion unit 11 includes three sets of three-dimensional coordinates relating to the object photographed by the camera, and two-dimensional coordinates (image coordinates) corresponding to each of the three-dimensional coordinates on the image captured by the camera. Is obtained as input.
  • the three-dimensional coordinate conversion unit 11 converts the coordinate system of the three-dimensional coordinates from the world coordinates to the local coordinate system having one of the three-dimensional coordinates as the origin.
  • the depth calculation unit 12 calculates a linear transformation matrix based on a projective transformation formula from each three-dimensional coordinate converted to the local coordinate system to the corresponding image coordinate.
  • the linear transformation matrix is a matrix that represents the relationship between the rotation matrix described later and the depth from the camera center to each of the three-dimensional coordinates.
  • the depth calculation unit 12 calculates the coefficient of the quartic equation for any one of these depths by utilizing the orthogonality of the rotation matrix described later, and solves the quartic equation to solve each depth. To calculate.
  • the rotation matrix calculation unit 13 calculates the rotation matrix of the camera in the local coordinate system using each depth and the linear transformation matrix.
  • the translation vector calculation unit 14 calculates the translation vector of the camera in the local coordinate system from each depth based on the projective transformation formula. Specifically, the translation vector calculation unit 14 calculates the translation vector by substituting the rotation matrix and the depth into the projective transformation formula.
  • the inverse conversion unit 15 reverse-converts the camera rotation matrix and translation vector calculated in the local coordinate system to the original world coordinate system, and calculates the camera rotation matrix and translation vector in the world coordinate system. Then, the inverse conversion unit 15 outputs the calculated rotation matrix and translation vector as external parameters of the camera.
  • the camera parameter estimation device in the present embodiment calculates the depth from the camera center to the three-dimensional coordinates and the linear transformation matrix, and calculates the rotation matrix and translation vector of the camera from these calculation results. .. That is, in the present embodiment, it is not necessary to execute the singular value decomposition, and the local coordinate system is not defined by the two-step coordinate transformation. According to the present embodiment, it is possible to estimate the rotation matrix and the translation vector, which are the external parameters of the camera, at high speed in one step without using the singular value decomposition.
  • FIG. 3 is a flow chart showing the operation of the camera parameter estimation device 10 according to the embodiment of the present invention.
  • FIG. 2 will be referred to as appropriate.
  • the camera parameter estimation method is implemented by operating the camera parameter estimation device 10. Therefore, the description of the camera parameter estimation method in the present embodiment will be replaced with the following operation description of the camera parameter estimation device 10.
  • the three-dimensional coordinate conversion unit 11 acquires these corresponding points. Then, the three-dimensional coordinate conversion unit 11 converts the coordinate system of the three sets of three-dimensional coordinates constituting the corresponding points from the world coordinates to the local coordinate system centered on one of the three-dimensional coordinates. (Step S11).
  • the input corresponding point is not the so-called degenerate arrangement (also called Degenerate Configuration or Critical Configuration) in the camera parameter to be estimated. This is because the camera parameters cannot be theoretically estimated for such a corresponding point.
  • the coordinate values it is assumed that both the three-dimensional coordinates and the two-dimensional points are different. This is because, for example, it is practically impossible for one three-dimensional coordinate to correspond to a plurality of different two-dimensional points.
  • a device different from the camera parameter estimation device 10 may perform an error determination of the corresponding point instead of the camera parameter estimation device 10.
  • the depth calculation unit 12 linearly transforms the rotation matrix and the depth based on the projective transformation formula for projecting each three-dimensional coordinate converted to the local coordinate system into the corresponding image coordinate in step S11. Calculate the matrix (step S12).
  • the depth calculation unit 12 calculates the coefficient of the quartic equation with respect to one depth by using the orthogonality of the rotation matrix calculated in the later step, and finds the solution of the quartic equation. Let the solution be the depth (step S13).
  • the solution of the quartic equation can be a complex number, but since all camera parameters are always real numbers, only the real number solution is the target in this embodiment. Further, when the calculation result of each step is only a complex number, the camera parameter estimation device 10 may issue a flag of no solution and end the subsequent processing.
  • the solution of the quadratic equation is It can be complex.
  • the rotation matrix calculation unit 13 calculates the rotation matrix of the camera in the local coordinate system using the depth calculated in step S13 and the linear transformation matrix calculated in step S12 (step S14).
  • the rotation matrix calculation unit 13 calculates a rotation matrix for each of the plurality of depths. That is, since the quartic equation has a maximum of four real number solutions, the number of calculations in step S14 is at most four.
  • the translation vector calculation unit 14 calculates the translation vector of the camera in the local coordinate system from the depth calculated in step S13 based on the projection conversion formula used in step S12 (step S15). Specifically, the translation vector calculation unit 14 calculates the translation vector by substituting the rotation matrix calculated in step S14 and the depth calculated in step S13 into the projection conversion formula. Further, also in step S15, the translation vector calculation unit 14 calculates the translation vector for all the real number solutions calculated in step S13, similarly to the rotation matrix calculation unit 13.
  • the inverse conversion unit 15 inversely transforms the rotation matrix calculated in the local coordinate system in step S14 and the translation vector calculated in step S15 into the original world coordinate system, and obtains the rotation.
  • the matrix and the translation vector are output as external parameters of the camera (step S16). Further, in step S16, the inverse conversion unit 15 executes output for each combination obtained by all the rotation matrices calculated in step S14 and all the translation vectors calculated in step S15.
  • FIGS. 1 to 3 will be referred to as appropriate.
  • the superscript "T” represents the transpose of the matrix n and the vector
  • "0” represents the zero matrix and the zero vector
  • "I” represents the unit matrix
  • " represents the L2 norm of the vector
  • "x" represents the cross product of the three-dimensional vectors.
  • the projective conversion formula between the three-dimensional coordinates and the image coordinates, which are the corresponding points, is defined by the following equation 1.
  • d i is the depth from the center of the camera to X i .
  • a, b, and c are the coordinate values after conversion.
  • Equation 4 [d 1 , d 2 , d 3 ] T.
  • the above equation 6 has two constraint equations for the two variables x and y, so the solution can be obtained.
  • One of the methods is to eliminate either x or y from the above equation 6 and reduce it to a one-variable equation based on the theory of the resultant. For example, eliminating y yields a quartic equation for x. It has been clarified in Non-Patent Documents 1 and 2 that the solution of the P3P problem results in a quartic equation of one variable, and it is understood that the clarified contents and the above-mentioned number 6 are in agreement. It supports the theoretical correctness of the present invention.
  • various known solutions can be used to obtain solutions to quartic equations.
  • an algebraic direct method such as Ferrari's solution or Euler's solution may be used, or a companion matrix-based solution method applying eigenvalue eigenvector calculation may be used.
  • the first and second columns of the rotation matrix R are calculated from the above equations 4 and 5 based on the matrix vector product which is a simple linear transformation.
  • or k k /
  • the translation vector t can be calculated from the first equation of the above equation 3. Since the rotation matrix R and the translation vector t calculated so far are values in the local coordinate system, the values in the original world coordinates can be obtained by performing the inverse transformation of Equation 2. That is, the final rotation matrix R and the translation vector t are expressed by the following equation 8.
  • step S11 the three-dimensional coordinate conversion unit 11 executes the coordinate conversion represented by the above equation 2 to convert the three-dimensional coordinates into the local coordinate system.
  • step S12 the depth calculation unit 12 calculates the matrix A and the matrix B, which represent the linear conversion between the rotation matrix and the depth, as represented by the above equations 4 and 5.
  • step S13 the depth calculation unit 12 sets the first component or the second component of the depth to 1, calculates the coefficient of the quartic equation which is the conclusion equation of the above equation 6, and solves the quartic equation. , Calculate the depth.
  • step S14 the rotation matrix calculation unit 13 calculates the first and second columns of the rotation matrix based on the above number 4 and the above number 5, and further rotates based on the above number 7. Calculate all the components of the matrix.
  • step S15 the translation vector calculation unit 14 calculates the translation vector based on the above equation 3.
  • step S16 the inverse conversion unit 15 converts the rotation matrix and the translation vector into the values in the original world coordinate system based on the above equation 8, and outputs the obtained rotation matrix and the translation vector. To do.
  • the components of the rotation matrix can be represented by the linear transformation of the depth as shown in the above equations 4 and 5. .. That is, if the depth can be calculated in the local coordinate system, the rotation matrix can be estimated directly. Therefore, in the present embodiment, it is not necessary to perform the two-step coordinate transformation as in the method disclosed in Non-Patent Document 2. Further, since the third row of the matrix A shown in the above equation 4 is zero, the calculation amount of the coefficient of the quartic equation is minimized by setting the first component or the second component of the depth vector to 1. It is possible to increase the speed.
  • T the three-dimensional coordinate conversion unit 11 and the inverse conversion unit 16 do not have to execute the process.
  • such a case may be a case where the three-dimensional coordinates are the corner points of the square marker.
  • the local coordinate system and the world coordinate system match, conversion to the local coordinate system becomes unnecessary, and as a result, the amount of calculation in the camera parameter estimation device 10 can be reduced.
  • the local coordinate system is a mathematically equivalent problem because it matches by rigid transformation.
  • the depth calculation unit 12 may output all the solutions to the rotation matrix calculation unit 13, or may output only positive real solutions when the nature of the depth is taken into consideration. good. Further, the depth calculation unit 12 extracts only the real part of the imaginary solution and outputs only the real part of the extracted imaginary solution in consideration of the possibility that a slight imaginary component is added to the real solution due to the error of the numerical calculation. You may.
  • Modification 4 In the present embodiment, when all the solutions obtained by the depth calculation unit 12 are imaginary numbers, the camera parameter estimation device 10 terminates the subsequent processing, returns a flag without a solution, and ends the processing. Is also good.
  • the program in the present embodiment may be any program that causes a computer to execute steps S11 to S16 shown in FIG.
  • the computer processor functions as a three-dimensional coordinate conversion unit 11, a depth calculation unit 12, a rotation matrix calculation unit 13, a translation vector calculation unit 14, and an inverse conversion unit 15 to perform processing.
  • each computer may function as one of the three-dimensional coordinate conversion unit 11, the depth calculation unit 12, the rotation matrix calculation unit 13, the translation vector calculation unit 14, and the inverse conversion unit 15, respectively. good.
  • FIG. 4 is a block diagram showing an example of a computer that realizes the camera parameter estimation device according to the embodiment of the present invention.
  • the computer in the present embodiment is not limited to the computer shown in FIG. 4, and may be, for example, a computer mounted on a device such as a robot or a smartphone.
  • the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. And. Each of these parts is connected to each other via a bus 121 so as to be capable of data communication. Further, the computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to the CPU 111 or in place of the CPU 111.
  • a GPU Graphics Processing Unit
  • FPGA Field-Programmable Gate Array
  • the CPU 111 expands the programs (codes) of the present embodiment stored in the storage device 113 into the main memory 112 and executes them in a predetermined order to perform various operations.
  • the main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).
  • the program according to the present embodiment is provided in a state of being stored in a computer-readable recording medium 120.
  • the program in the present embodiment may be distributed on the Internet connected via the communication interface 117.
  • the storage device 113 include a semiconductor storage device such as a flash memory in addition to a hard disk drive.
  • the input interface 114 mediates data transmission between the CPU 111 and an input device 118 such as a keyboard and mouse.
  • the display controller 115 is connected to the display device 119 and controls the display on the display device 119.
  • the data reader / writer 116 mediates the data transmission between the CPU 111 and the recording medium 120, reads the program from the recording medium 120, and writes the processing result in the computer 110 to the recording medium 120.
  • the communication interface 117 mediates data transmission between the CPU 111 and another computer.
  • the recording medium 120 include a general-purpose semiconductor storage device such as CF (CompactFlash (registered trademark)) and SD (SecureDigital), a magnetic recording medium such as a flexible disk, or a CD-.
  • CF CompactFlash (registered trademark)
  • SD Secure Digital
  • magnetic recording medium such as a flexible disk
  • CD- CompactDiskReadOnlyMemory
  • optical recording media such as ROM (CompactDiskReadOnlyMemory).
  • the camera parameter estimation device 10 in the present embodiment can also be realized by using hardware corresponding to each part instead of the computer on which the program is installed. Further, the camera parameter estimation device 10 may be partially realized by a program and the rest may be realized by hardware.
  • a three-dimensional coordinate conversion unit that converts the coordinate system of the three-dimensional coordinates from the world coordinate system to a local coordinate system having one of the three-dimensional coordinates as the origin.
  • a linear transformation matrix is calculated based on the projective transformation formula from the 3D coordinates converted to the local coordinate system to the 2D coordinates, and out of the depths from the center of the camera to each of the 3D coordinates.
  • a depth calculation unit that calculates each of the depths by calculating the coefficient of the quaternary equation for any one of the above and solving the quaternary equation.
  • a rotation matrix calculation unit that calculates the rotation matrix of the camera in the local coordinate system using each of the depths and the linear transformation matrix.
  • a translation vector calculation unit that calculates the translation vector of the camera in the local coordinate system from each of the depths based on the projection conversion formula. It is provided with an inverse conversion unit that inversely transforms the rotation matrix and translation vector of the camera in the local coordinate system to calculate the rotation matrix and translation vector of the camera in the world coordinate system.
  • the camera parameter estimation device according to Appendix 1.
  • the local coordinate system is a coordinate system represented by linearly transforming one of the depths by two columns of vectors constituting the rotation matrix.
  • the camera parameter estimation device according to Appendix 1 or 2.
  • the depth calculation unit selects the depth that minimizes the amount of calculation when calculating the coefficients of the quartic equation. A camera parameter estimation device characterized by this.
  • (Appendix 4) (A) Three sets of three-dimensional coordinates related to the object photographed by the camera and two-dimensional coordinates corresponding to each of the three-dimensional coordinates on the photographed image are input. A step of converting the coordinate system of the three-dimensional coordinates from the world coordinate system to the local coordinate system having one of the three-dimensional coordinates as the origin. (B) A linear conversion matrix is calculated based on the projection conversion formula from the three-dimensional coordinates converted to the local coordinate system to the two-dimensional coordinates, and from the center of the camera to each of the three-dimensional coordinates. Steps and steps that calculate each of the depths by calculating the coefficients of the quadratic equation for any one of the depths and solving the quadratic equation.
  • (C) A step of calculating the rotation matrix of the camera in the local coordinate system using each of the depths and the linear transformation matrix.
  • (D) A step of calculating the translation vector of the camera in the local coordinate system from each of the depths based on the projective transformation formula.
  • (E) It has a step of inversely transforming the rotation matrix and translation vector of the camera in the local coordinate system to calculate the rotation matrix and translation vector of the camera in the world coordinate system.
  • a camera parameter estimation method characterized by this.
  • the camera parameter estimation method described in Appendix 4 The camera parameter estimation method described in Appendix 4,
  • the local coordinate system is a coordinate system represented by linearly transforming one of the depths by two columns of vectors constituting the rotation matrix.
  • step (Appendix 6) The camera parameter estimation method according to Appendix 4 or 5.
  • step (b) when calculating the coefficients of the quartic equation, the depth that minimizes the amount of calculation is selected. A camera parameter estimation method characterized by this.
  • (C) A step of calculating the rotation matrix of the camera in the local coordinate system using each of the depths and the linear transformation matrix.
  • (D) A step of calculating the translation vector of the camera in the local coordinate system from each of the depths based on the projective transformation formula.
  • (E) Including an instruction to execute a step of inversely transforming the rotation matrix and translation vector of the camera in the local coordinate system to calculate the rotation matrix and translation vector of the camera in the world coordinate system.
  • a computer-readable recording medium that records the program.
  • Appendix 8 The computer-readable recording medium according to Appendix 7.
  • the local coordinate system is a coordinate system represented by linearly transforming one of the depths by two columns of vectors constituting the rotation matrix.
  • a computer-readable recording medium characterized by that.
  • step (Appendix 9) A computer-readable recording medium according to Appendix 7 or 8.
  • step (b) when calculating the coefficients of the quartic equation, the depth that minimizes the amount of calculation is selected.
  • the present invention it is possible to solve the P3P problem at high speed in one step and estimate the external parameters without using the singular value decomposition.
  • the present invention is effective in fields where estimation of external parameters of a calibrated camera is required, for example, robot control, image processing, and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
PCT/JP2019/017508 2019-04-24 2019-04-24 カメラパラメータ推定装置、カメラパラメータ推定方法、及びコンピュータ読み取り可能な記録媒体 Ceased WO2020217360A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2021515390A JP7099627B2 (ja) 2019-04-24 2019-04-24 カメラパラメータ推定装置、カメラパラメータ推定方法、及びプログラム
PCT/JP2019/017508 WO2020217360A1 (ja) 2019-04-24 2019-04-24 カメラパラメータ推定装置、カメラパラメータ推定方法、及びコンピュータ読み取り可能な記録媒体
US17/604,837 US11935266B2 (en) 2019-04-24 2019-04-24 Camera parameter estimation apparatus, camera parameter estimation method, and computer-readable recording medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/017508 WO2020217360A1 (ja) 2019-04-24 2019-04-24 カメラパラメータ推定装置、カメラパラメータ推定方法、及びコンピュータ読み取り可能な記録媒体

Publications (1)

Publication Number Publication Date
WO2020217360A1 true WO2020217360A1 (ja) 2020-10-29

Family

ID=72941166

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/017508 Ceased WO2020217360A1 (ja) 2019-04-24 2019-04-24 カメラパラメータ推定装置、カメラパラメータ推定方法、及びコンピュータ読み取り可能な記録媒体

Country Status (3)

Country Link
US (1) US11935266B2 (https=)
JP (1) JP7099627B2 (https=)
WO (1) WO2020217360A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115082815A (zh) * 2022-07-22 2022-09-20 山东大学 基于机器视觉的茶芽采摘点定位方法、装置及采摘系统
CN116385560A (zh) * 2023-03-28 2023-07-04 杭州海康机器人股份有限公司 一种相机参数的标定方法、装置及设备

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11830223B2 (en) * 2019-04-08 2023-11-28 Nec Corporation Camera calibration apparatus, camera calibration method, and nontransitory computer readable medium storing program
CN114758016B (zh) * 2022-06-15 2022-09-13 超节点创新科技(深圳)有限公司 摄像设备标定方法、电子设备和存储介质
US11887338B2 (en) * 2022-06-16 2024-01-30 Motional Ad Llc Maintaining calibration of an IBIS camera
CN115578469B (zh) * 2022-09-23 2026-01-02 武汉理工大学 一种面向大型复杂工件的多视角联合标定方法
CN116309012A (zh) * 2023-02-08 2023-06-23 苏州理工雷科传感技术有限公司 一种雷达二维图像到三维图像的映射方法
CN119444586B (zh) * 2024-10-22 2025-09-23 清华大学深圳国际研究生院 一种基于三维几何性质的实时双目rgb-d拼接方法
CN121458796B (zh) * 2025-12-31 2026-04-21 清华大学 用于视线估计的深度图像标准化方法、装置、设备及视线估计系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010250452A (ja) * 2009-04-14 2010-11-04 Tokyo Univ Of Science 任意視点画像合成装置
WO2013111229A1 (ja) * 2012-01-23 2013-08-01 日本電気株式会社 カメラ校正装置、カメラ校正方法およびカメラ校正用プログラム
JP2015106287A (ja) * 2013-11-29 2015-06-08 キヤノン株式会社 キャリブレーション装置及び方法
WO2019065536A1 (ja) * 2017-09-26 2019-04-04 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 再構成方法および再構成装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016106287A (ja) 2013-03-27 2016-06-16 太陽誘電株式会社 機械の稼動情報を収集するシステム及び方法
TWI720447B (zh) * 2019-03-28 2021-03-01 財團法人工業技術研究院 影像定位方法及其系統

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010250452A (ja) * 2009-04-14 2010-11-04 Tokyo Univ Of Science 任意視点画像合成装置
WO2013111229A1 (ja) * 2012-01-23 2013-08-01 日本電気株式会社 カメラ校正装置、カメラ校正方法およびカメラ校正用プログラム
JP2015106287A (ja) * 2013-11-29 2015-06-08 キヤノン株式会社 キャリブレーション装置及び方法
WO2019065536A1 (ja) * 2017-09-26 2019-04-04 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 再構成方法および再構成装置

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115082815A (zh) * 2022-07-22 2022-09-20 山东大学 基于机器视觉的茶芽采摘点定位方法、装置及采摘系统
CN115082815B (zh) * 2022-07-22 2023-04-07 山东大学 基于机器视觉的茶芽采摘点定位方法、装置及采摘系统
CN116385560A (zh) * 2023-03-28 2023-07-04 杭州海康机器人股份有限公司 一种相机参数的标定方法、装置及设备

Also Published As

Publication number Publication date
US20220262037A1 (en) 2022-08-18
US11935266B2 (en) 2024-03-19
JP7099627B2 (ja) 2022-07-12
JPWO2020217360A1 (https=) 2020-10-29

Similar Documents

Publication Publication Date Title
JP7099627B2 (ja) カメラパラメータ推定装置、カメラパラメータ推定方法、及びプログラム
JP7052788B2 (ja) カメラパラメータ推定装置、カメラパラメータ推定方法、及びプログラム
US20210110599A1 (en) Depth camera-based three-dimensional reconstruction method and apparatus, device, and storage medium
Wang et al. On error assessment in stereo-based deformation measurements: part I: theoretical developments for quantitative estimates
JP6547754B2 (ja) 三角測量装置、三角測量方法およびそのプログラム
CN110148179A (zh) 一种训练用于估计图像视差图的神经网络模型方法、装置及介质
JP7111183B2 (ja) カメラパラメータ推定装置、カメラパラメータ推定方法、及びプログラム
JPWO2020188799A1 (ja) カメラ校正装置、カメラ校正方法、及びプログラム
JP7546748B2 (ja) カメラ校正方法及びカメラ校正システム
CN111932628B (zh) 一种位姿确定方法及装置、电子设备、存储介质
Barath et al. Relative pose from sift features
CN110738730A (zh) 点云匹配方法、装置、计算机设备和存储介质
CN110930444A (zh) 一种基于双边优化的点云匹配方法、介质、终端和装置
Tan et al. Improved shape-from-template method with perspective space constraints for disappearing features
Ferrer et al. Eigen-factors a bilevel optimization for plane SLAM of 3D point clouds
CN106651950B (zh) 一种基于二次曲线透视投影不变性的单相机位姿估计方法
JP2020041950A (ja) 測量装置、測量方法、及びプログラム
WO2023166618A1 (ja) カメラパラメータ推定装置、カメラパラメータ推定方法、及びコンピュータ読み取り可能な記録媒体
CN117036498A (zh) 一种双目相机标定方法、装置及电子设备
JP2017163386A (ja) カメラパラメータ推定装置、カメラパラメータ推定方法、及びプログラム
Li et al. Robust pose estimation which guarantees positive depths
Yu et al. An efficient and reasonably simple solution to the perspective-three-point problem
Weilong et al. On camera calibration and distortion correction based on bundle adjustment
Meng et al. Camera motion estimation and optimization approach
CN111242995A (zh) 一种快速鲁棒的摄像机绝对姿态估计方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19926150

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021515390

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19926150

Country of ref document: EP

Kind code of ref document: A1