CN109685852A - The scaling method of camera and inertial sensor, system, equipment and storage medium - Google Patents
The scaling method of camera and inertial sensor, system, equipment and storage medium Download PDFInfo
- Publication number
- CN109685852A CN109685852A CN201811398121.0A CN201811398121A CN109685852A CN 109685852 A CN109685852 A CN 109685852A CN 201811398121 A CN201811398121 A CN 201811398121A CN 109685852 A CN109685852 A CN 109685852A
- Authority
- CN
- China
- Prior art keywords
- camera
- rotary shaft
- inertial sensor
- rotaxis
- group
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C25/00—Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass
- G01C25/005—Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass initial alignment, calibration or starting-up of inertial devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Manufacturing & Machinery (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Gyroscopes (AREA)
Abstract
The present invention provides the scaling method of camera and inertial sensor, system, equipment and storage mediums, wherein, method includes: by different shooting postures, obtain i group rotary shaft pair, every group of rotary shaft to include based on same rotary shaft rotate and obtain camera coordinates system under camera rotary shaft and the inertial sensor rotary shaft under inertial sensor coordinate system, i be natural number；Acquire the normal direction n of the equidistant point of every group of rotary shaft pairi；Pass through the normal direction n of the equidistant point of i group rotary shaft pairiA reference planes are constructed, using the normal direction of reference planes as the conversion rotary shaft of camera coordinates system to inertial sensor coordinate system；When obtaining the small value of error formula, the value of rotation angle；Spin matrix of the acquisition inertial sensor coordinate system to camera coordinates system.Recombinant after the present invention can decouple the measurement of camera and inertial sensor, enhancing make robot precise positioning in the environment to improve the fusion accuracy of camera data and inertial sensor data to the resistance of error.
Description
Technical field
The present invention relates to hand and eye calibrating fields, specifically, be related to the scaling method of camera and inertial sensor, system,
Equipment and storage medium.
Background technique
With the development of computer technology, an important research field of the computer vision as artificial intelligence, extensively
It is applied to all trades and professions generally.Computer vision technique is combined with robot technology, but also field in intelligent robotics obtains
To greatly developing.The technology of simultaneous localization and mapping (SLAM) is the important problem in one, robot navigation field, SLAM
Problem can be described as: robot can establish a global map to the environment explored, at the same can at any time,
Go to deduce the position of itself using this map.Influence two consistency and environment because being known as observation data of SLAM system
Noise, the data consistency that the correctness of ambient enviroment observation has depended on, and then influence the building of environmental map.It moves at present
The most common sensor has camera and IMU sensor etc. in dynamic equipment, and how algorithm for design goes to utilize these common sensors
Equipment realizes high-precision while positioning and drawing is the hot issue of a current research.
The scaling method of camera coordinates system of the coordinate system of existing inertial sensor to mobile robot is easy to be made an uproar
Sound shadow is rung, and there are cameras and the nonsynchronous problem of inertial sensor data.
Therefore, the present invention provides the scaling method of a kind of camera and inertial sensor, system, equipment and storage mediums.
Summary of the invention
For the problems of the prior art, the purpose of the present invention is to provide the scaling method of camera and inertial sensor,
System, equipment and storage medium can decouple the measurement of camera and inertial sensor, enhance the resistance to error, due to
Rotary shaft under respective coordinate system need to be only measured respectively, and it is synchronous with inertial sensor data not depend on camera, passes through the calibration
The fusion accuracy of camera data and inertial sensor data can be improved in method, makes robot precise positioning in the environment.
The embodiment of the present invention provides the scaling method of a kind of camera and inertial sensor, is applied to have camera and inertia
The electronic equipment of sensor, comprising the following steps:
By different shooting postures, i group rotary shaft pair is obtained, every group of rotary shaft is to including based on the rotation of same rotary shaft
And the camera rotary shaft rotAxis under the camera coordinates system obtainedCiAnd the inertial sensor rotation under inertial sensor coordinate system
Shaft rotAxisIi, i is natural number；
Acquire the normal direction n of the equidistant point of every group of rotary shaft pairi:
Pass through the normal direction n of the equidistant point of the i group rotary shaft pairiA reference planes are constructed, by the reference planes
Conversion rotary shaft rotAxis of the normal direction as the camera coordinates system to inertial sensor coordinate systemI2C；
When obtaining the small value of error formula, rotation angle rotAngI2CValue, the error formula are as follows:
Obtain the inertial sensor coordinate system to the camera coordinates system spin matrix RI2C, RI2C=rodrigues
(rotAngI2C*rotAxisI2C)。
Preferably, the inertial sensor rotary shaft rotAxisIi* spin matrix RI2C=camera rotary shaft rotAxisCi,
The spin matrix RI2CIt is the matrix of a 3*3.
Preferably, the method for one group of rotary shaft pair of acquisition includes:
Camera coordinates system is established, electronic equipment is based on an axial direction and makees dead axle movement, a gridiron pattern is shot by camera,
According to the change in location of angle point in the gridiron pattern, the camera rotary shaft rotAxis under camera coordinates system is obtainedCi；
Inertial sensor coordinate system is established, the electronic equipment is continued to make dead axle movement based on same axial direction, record rotation
The numerical value change of journey gyroscope is turned over, inertial sensor spin matrix and inertial sensor rotary shaft rotAxis are obtainedIi。
Preferably, the camera rotary shaft rotAxisCiFor a normalized vector (ai,bi,ci), the inertial sensor
Rotary shaft rotAxisIiFor a normalized vector (di,ei,fi)。
Preferably, to the normal direction n of the equidistant point of every group of rotary shaft pairiIt is normalized.
Preferably, the rotary shaft of the rotary shaft pair of different groups is respectively different.
The embodiment of the present invention also provides the calibration system of a kind of camera and inertial sensor, for realizing above-mentioned camera
With the scaling method of inertial sensor, comprising:
Rotary shaft is to acquisition module, by different shooting postures, obtains i group rotary shaft pair, and every group of rotary shaft is to including
Camera rotary shaft rotAxis under the camera coordinates system obtained based on the rotation of same rotary shaftCiAnd inertial sensor coordinate
Inertial sensor rotary shaft rotAxis under systemIi, i is natural number；
Normal direction obtains module, acquires the normal direction n of the equidistant point of every group of rotary shaft pairi:
Reference planes establish module, pass through the normal direction n of the equidistant point of the i group rotary shaft pairiOne is constructed with reference to flat
Face, using the normal direction of the reference planes as the conversion rotary shaft of the camera coordinates system to inertial sensor coordinate system
rotAxisI2C；
Rotation angle obtains module, when obtaining the small value of error formula, rotation angle rotAngI2CValue, the error formula are as follows:
Spin matrix obtain module, obtain the inertial sensor coordinate system to the camera coordinates system spin matrix
RI2C, RI2C=rodrigues (rotAngI2C*rotAxisI2C)。
Preferably, the inertial sensor rotary shaft rotAxisIi* spin matrix RI2C=camera rotary shaft rotAxisCi,
The spin matrix RI2CIt is the matrix of a 3*3.
The embodiment of the present invention also provides the calibration facility of a kind of camera and inertial sensor, comprising:
Processor；
Memory, wherein being stored with the executable instruction of the processor；
Wherein, the processor is configured to execute above-mentioned camera and inertial sensor via the executable instruction is executed
Scaling method the step of.
The embodiment of the present invention also provides a kind of computer readable storage medium, and for storing program, described program is held
The step of scaling method of above-mentioned camera and inertial sensor is realized when row.
Scaling method, system, equipment and the storage medium of camera and inertial sensor of the invention by camera and can be used to
Property sensor measurement decoupling, enhance to the resistance of error, due to need to only measure the rotary shaft under respective coordinate system respectively, no
It is synchronous with inertial sensor data to rely on camera, by the scaling method, camera data and inertial sensor number can be improved
According to fusion accuracy, make robot precise positioning in the environment.
Detailed description of the invention
Upon reading the detailed description of non-limiting embodiments with reference to the following drawings, other feature of the invention,
Objects and advantages will become more apparent upon.
Fig. 1 is the flow chart using the scaling method of camera and inertial sensor of the invention.
Fig. 2 is to acquire first group of rotary shaft centering camera in scaling method using camera and inertial sensor of the invention
The schematic diagram of rotary shaft.
Fig. 3 is to acquire first group of rotation shaft assignment inertia in scaling method using camera and inertial sensor of the invention
The schematic diagram of sensor rotation axis.
Fig. 4 is to acquire second group of rotary shaft centering camera in scaling method using camera and inertial sensor of the invention
The schematic diagram of rotary shaft.
Fig. 5 is to acquire second group of rotation shaft assignment inertia in scaling method using camera and inertial sensor of the invention
The schematic diagram of sensor rotation axis.
Fig. 6 is the module diagram of the calibration system of camera and inertial sensor of the invention；
Fig. 7 is the structural schematic diagram of the calibration facility of camera and inertial sensor of the invention；And
Fig. 8 is the structural schematic diagram of the computer readable storage medium of one embodiment of the invention.
Specific embodiment
Example embodiment is described more fully with reference to the drawings.However, example embodiment can be with a variety of shapes
Formula is implemented, and is not understood as limited to embodiment set forth herein.On the contrary, thesing embodiments are provided so that the present invention will
Fully and completely, and by the design of example embodiment comprehensively it is communicated to those skilled in the art.It is identical attached in figure
Icon note indicates same or similar structure, thus will omit repetition thereof.
Fig. 1 is the flow chart using the scaling method of camera and inertial sensor of the invention.As shown in Figure 1, of the invention
The scaling method of a kind of camera and inertial sensor is provided, applied to the electronic equipment for having camera and inertial sensor, including
Following steps:
S101, by different shooting postures, obtain i group rotary shaft pair, every group of rotary shaft is to including being based on same rotation
Axis rotates and the camera rotary shaft rotAxis under the camera coordinates system of acquisitionCiAnd the inertia under inertial sensor coordinate system passes
Sensor rotary shaft rotAxisIi, i is natural number.The rotary shaft of the rotary shaft pair of difference group is respectively different.Preferably at one
Embodiment in, obtain camera rotary shaft rotAxisC1With inertial sensor rotary shaft rotAxisIiSequencing it is not solid
It is fixed, it can be with exchange sequence.
S102, acquire every group of rotary shaft pair equidistant point normal direction ni:
S103, the normal direction n by the equidistant point of the i group rotary shaft pairiA reference planes are constructed, by the reference
Conversion rotary shaft rotAxis of the normal direction of plane as the camera coordinates system to inertial sensor coordinate systemI2C.It is excellent at one
It selects in scheme, to the normal direction n of the equidistant point of every group of rotary shaft pairiIt is normalized, but not limited to this.
S104, when obtaining the small value of error formula, rotation angle rotAngI2CValue, the error formula are as follows:
The spin matrix R of S105, the acquisition inertial sensor coordinate system to the camera coordinates systemI2C, RI2C=
rodrigues(rotAngI2C*rotAxisI2C).Pass through spin matrix RI2CThe mode converted are as follows: the inertial sensor
Rotary shaft rotAxisIi* spin matrix RI2C=camera rotary shaft rotAxisCi, the spin matrix RI2CIt is the square of a 3*3
Battle array.
Inertial Measurement Unit (abbreviation IMU) in the present invention is measurement object triaxial attitude angle (or angular speed) and accelerates
The device of degree.In general, an Inertial Measurement Unit contains three uniaxial accelerometers and three uniaxial gyros, accelerate
Degree meter detection object founds the acceleration signal of three axis in carrier coordinate system unification and independence, and gyro detects carrier relative to navigational coordinate system
Angular velocity signal, measurement object angular speed and acceleration in three dimensions, and calculate with this posture of object.
In step s101, Fig. 2 is to acquire first group in the scaling method using camera and inertial sensor of the invention
The schematic diagram of rotary shaft centering camera rotary shaft.As shown in Fig. 2, black and white gridiron pattern pendulum before camera and is remained stationary, pass through
Electronic equipment 1 is rotated around axis S, rotary camera 2 makes it do dead axle movement, and records the angle point of gridiron pattern 4 in motion process
Position can then be asked according to the change in location of angle point in the gridiron pattern by the prior art since camera is demarcated in advance
Rotation of the gridiron pattern relative to camera in camera rotary course out, then the rotation between any two positions is necessarily around the same rotation
Shaft, obtain the camera rotary shaft rotAxis under camera coordinates systemC1.Camera rotary shaft is calculated about by the prior art
rotAxisC1Process be not emphasis of the invention, details are not described herein again.In a preferred embodiment, the camera rotary shaft
rotAxisC1For a normalized vector (a1,b1,c1), but not limited to this.
Fig. 3 is to acquire first group of rotation shaft assignment inertia in scaling method using camera and inertial sensor of the invention
The schematic diagram of sensor rotation axis.As shown in figure 3, keeping the posture (Fig. 2) when electronic equipment 1 rotates in previous step, equally do
Along the rotation of axis S, the degree variation of the gyroscope 3 in rotary course is recorded, the side of the prior art (such as: integrating) is passed through
Formula calculates spin matrix, and obtains inertial sensor rotary shaft rotAxisI1.Body rotary shaft is calculated about by the prior art
rotAxisI1Process be not emphasis of the invention, details are not described herein again.In a preferred embodiment, the inertial sensor
Rotary shaft rotAxisI1For a normalized vector (d1,e1,f1), but not limited to this.With reference to Fig. 2 and 3, first group of rotation is obtained
Shaft is to (containing camera rotary shaft rotAxisC1With inertial sensor rotary shaft rotAxisI1)。
Fig. 4 is to acquire second group of rotary shaft centering camera in scaling method using camera and inertial sensor of the invention
The schematic diagram of rotary shaft.As shown in figure 4, black and white gridiron pattern pendulum before camera and is remained stationary, by by electronic equipment 1 around
Axis T rotation (direction of axis T is different from axis S), rotary camera 2 makes it do dead axle movement, and records gridiron pattern 4 in motion process
The position of angle point then can pass through the prior art according to the change in location of angle point in the gridiron pattern since camera is demarcated in advance
Rotation of the gridiron pattern relative to camera in camera rotary course is found out, then the rotation between any two positions is necessarily around same
A rotary shaft, obtain the camera rotary shaft rotAxis under camera coordinates systemC2.In a preferred embodiment, the camera rotation
Shaft rotAxisC2For a normalized vector (a2,b2,c2), but not limited to this.
Fig. 5 is to acquire second group of rotation shaft assignment inertia in scaling method using camera and inertial sensor of the invention
The schematic diagram of sensor rotation axis.As shown in figure 5, keeping the posture (Fig. 4) when electronic equipment 1 rotates in previous step, equally do
Along the rotation of axis T, the degree variation of the gyroscope 3 in rotary course is recorded, the side of the prior art (such as: integrating) is passed through
Formula calculates spin matrix, and obtains rotary shaft rotAxisI2.In a preferred embodiment, the inertial sensor rotary shaft
rotAxisI2For a normalized vector (d2,e2,f2), but not limited to this.With reference to Figure 4 and 5, second group of rotary shaft pair is obtained
(contain camera rotary shaft rotAxisC2With inertial sensor rotary shaft rotAxisI2)
So can continue to select new rotary shaft, further obtain third group rotary shaft to, the 4th group of rotary shaft to,
5th group of rotary shaft is right ..., and details are not described herein again.The quantity of rotary shaft pair is more, is more conducive to the levels of precision for improving calibration.
For every group of rotary shaft to the normal direction n for acquiring corresponding equidistant point Wi, revolved by the camera of every group of rotary shaft pair
Shaft rotAxisC2With inertial sensor rotary shaft rotAxisI2To obtain the normal direction n of equidistant pointi, formula are as follows:
The 5th group of rotary shaft before is to can respectively obtain normal direction n1, normal direction n2, normal direction n3, normal direction n4, normal direction n5。
Also, the normal direction n of the equidistant point by five groups of rotary shafts pair1, normal direction n2, normal direction n3, normal direction n4, normal direction n5Building
One reference planes U, using the normal direction of the reference planes as the camera coordinates system to the conversion of inertial sensor coordinate system
Rotary shaft rotAxisI2C.In a preferred embodiment, to the normal direction n of the equidistant point of every group of rotary shaft pairiCarry out normalizing
Change, but not limited to this.
The detailed process of the present embodiment, which may is that, does camera and inertial sensor mould group under different postures around dead axle
Rotation, and calculate separately out the rotary shaft rotAxis under camera coordinates systemC1It (is passed by inertia with inertial sensor coordinate system
Sensor mould group building coordinate system) under rotary shaft rotAxisIi.Black and white gridiron pattern pendulum before camera and is remained stationary, is rotated
Camera makes it do dead axle movement, and records the position of X-comers in motion process, since camera is demarcated in advance, then can ask
Camera rotary shaft rotAxis of the gridiron pattern relative to camera in camera rotary course outC1, then the rotation between any two positions must
It is so around the same rotary shaft.Also, the posture when rotation of previous step camera is kept, System of Rotating about Fixed Axis is equally done, records backspin
The degree variation of gyroscope during turning, calculates spin matrix, and obtain inertial sensor rotary shaft by way of integral
rotAxisIi.The operation that camera pendulum is repeated to above-mentioned two step under different postures, obtains one group of rotary shaft pair.Preferably at one
Embodiment in, obtain camera rotary shaft rotAxisC1With inertial sensor rotary shaft rotAxisIiSequencing it is not solid
It is fixed, it can be with exchange sequence.
The equidistant point between respective camera, inertial sensor rotary shaft pair is sought, if camera rotary shaft is rotAxisC1, it is used to
Property sensor rotation axis be rotAxisIi, then the normal direction of its equidistant point are as follows:
For all rotary shafts pair, the normal direction of their equidistant point is found out.
Pass through the normal direction n of the equidistant point of the i group rotary shaft pairiA reference planes are constructed, by the reference planes
Conversion rotary shaft rotAxis of the normal direction as the camera coordinates system to inertial sensor coordinate systemI2C。
Under known rotary shaft, rotation angle is acquired by the method for minimizing following formula:
Then the spin matrix of inertial sensor coordinate system to camera coordinates system is rodrigues (rotAngI2C*
rotAxisI2C)
It can by the transformation matrix measured under the transformation matrix that is measured under camera coordinates system, inertial sensor coordinate system
With obtain inertial sensor to camera coordinates system transformation matrix, since the present invention pertains only to inertial sensor to camera coordinates system
Rotation relationship therefore can be by measuring under the spin matrix that is measured under camera coordinates system, inertial sensor coordinate system
Spin matrix is it can be concluded that spin matrix of the inertial sensor to camera coordinates system, spin matrix general for one, to it
Eigenvalues Decomposition is done, characteristic value and feature vector can be obtained.
Therefore, feature decomposition is carried out to the spin matrix that measures under inertial sensor coordinate system, obtain its characteristic value and
Feature vector carries out feature decomposition to the spin matrix measured under camera coordinates system, obtains its characteristic value and feature vector, with
It is the corresponding feature vector of 1 characteristic value is inertia sensing that the spin matrix measured under inertial sensor coordinate system, which decomposes,
Rotary shaft rotAxis under device coordinate systemI, decomposing with the spin matrix measured under camera coordinates system is 1 characteristic value
Corresponding feature vector is the rotary shaft rotAxis under camera coordinates systemC, and by solving camera and inertial sensor coordinate system
Under rotary shaft come solve Imu to camera spin matrix RI2C。
Solve RI2CIt is divided into following two step
Seek rotAxisI2CAnd rotAngI2C, respectively indicate RI2CRotary shaft and rotation angle:
By rotAxisCAnd rotAxisIIt is imagined as two, space vector, then rotAxisI2CNecessarily fall in rotAxisCWith
rotAxisIOn the equidistant point at the angle of composition, it is assumed that have n group rotAxisIAnd rotAxisC, then this n equidistant point is necessarily total
Line, this line are exactly rotAxis to be askedI2C。
Then, equidistant point be collinearly equivalent to equidistant point normal direction it is coplanar, the plane that the normal direction of equidistant point is constituted
Normal direction, that is, rotAxisI2C
So solving rotAxisI2CThe problem of the problem of being converted to solution equidistant point normal direction.
As known rotAxisIAnd rotAxisC, then angle plane normal direction
Then vector n1、n2……nkThe normal direction of the plane of composition is exactly rotAxisI2C
Further solve rotAngI2C
In known rotAxisI2CWhen, rotAng is acquired by way of minimizing following formulaI2CValue
Then the spin matrix of inertial sensor coordinate system to camera coordinates system is rodrigues (rotAngI2C*
rotAxisI2C)。
Fig. 6 is the module diagram of the calibration system of camera and inertial sensor of the invention.As shown in fig. 6, of the invention
The calibration system 5 of a kind of camera and inertial sensor is provided, also for realizing the calibration side of above-mentioned camera and inertial sensor
Method, comprising:
Rotary shaft is to acquisition module 501, by different shooting postures, obtains i group rotary shaft pair, and every group of rotary shaft is to including
Based on the camera rotary shaft rotAxis under the camera coordinates system that same rotary shaft rotates and obtainsIiAnd inertial sensor is sat
Inertial sensor rotary shaft rotAxis under mark systemCi, i is natural number；
Normal direction obtains module 502, acquires the normal direction n of the equidistant point of every group of rotary shaft pairi:
Reference planes establish module 503, pass through the normal direction n of the equidistant point of the i group rotary shaft pairiConstruct a reference
Plane, using the normal direction of the reference planes as the conversion rotary shaft of the camera coordinates system to inertial sensor coordinate system
rotAxisI2C；
Rotation angle obtains module 504, when obtaining the small value of error formula, rotation angle rotAngI2CValue, the error formula
Are as follows:
Spin matrix obtain module 505, obtain the inertial sensor coordinate system to the camera coordinates system spin moment
Battle array RI2C, RI2C=rodrigues (rotAngI2C*rotAxisI2C)。
In a preferred embodiment, the inertial sensor rotary shaft rotAxisCi* spin matrix RI2C=camera rotary shaft
rotAxisIi, the spin matrix RI2CIt is the matrix of a 3*3.
In a preferred embodiment, the camera rotary shaft rotAxisCiFor a normalized vector (ai,bi,ci), it is described used
Property sensor rotation axis rotAxisIiFor a normalized vector (di,ei,fi)。
In a preferred embodiment, to the normal direction n of the equidistant point of every group of rotary shaft pairiIt is normalized.
In a preferred embodiment, the rotary shaft of the rotary shaft pair of different groups is respectively different.
The calibration system of camera and inertial sensor of the invention can decouple the measurement of camera and inertial sensor,
Enhancing does not depend on camera and inertia sensing due to need to only measure the rotary shaft under respective coordinate system respectively to the resistance of error
The synchronization of device data can be improved the fusion accuracy of camera data and inertial sensor data, make machine by the scaling method
People's precise positioning in the environment.
The embodiment of the present invention also provides the calibration facility of a kind of camera and inertial sensor, including processor.Memory,
In be stored with the executable instruction of processor.Wherein, processor be configured to via execute executable instruction be performed camera with
The step of scaling method of inertial sensor.
As it appears from the above, the embodiment can decouple the measurement of camera and inertial sensor, enhance the resistance to error,
Due to need to only measure the rotary shaft under respective coordinate system respectively, it is synchronous with inertial sensor data not depend on camera, by this
The fusion accuracy of camera data and inertial sensor data can be improved in scaling method, makes robot precise positioning in the environment.
Person of ordinary skill in the field it is understood that various aspects of the invention can be implemented as system, method or
Program product.Therefore, various aspects of the invention can be embodied in the following forms, it may be assumed that complete hardware embodiment, complete
The embodiment combined in terms of full Software Implementation (including firmware, microcode etc.) or hardware and software, can unite here
Referred to as " circuit ", " module " or " platform ".
Fig. 7 is the structural schematic diagram of the calibration facility of camera and inertial sensor of the invention.It is described referring to Fig. 7
The electronic equipment 600 of this embodiment according to the present invention.The electronic equipment 600 that Fig. 7 is shown is only an example, is not answered
Any restrictions are brought to the function and use scope of the embodiment of the present invention.
As shown in fig. 7, electronic equipment 600 is showed in the form of universal computing device.The component of electronic equipment 600 can wrap
Include but be not limited to: at least one processing unit 610, at least one storage unit 620, connection different platform component (including storage
Unit 620 and processing unit 610) bus 630 etc..
Wherein, storage unit is stored with program code, and program code can be executed with unit 610 processed, so that processing is single
Member 610 executes various exemplary implementations according to the present invention described in this specification above-mentioned electronic prescription circulation processing method part
The step of mode.For example, processing unit 610 can execute step as shown in fig. 1.
Storage unit 620 may include the readable medium of volatile memory cell form, such as Random Access Storage Unit
(RAM) 6201 and/or cache memory unit 6202, it can further include read-only memory unit (ROM) 6203.
Storage unit 620 can also include program/utility with one group of (at least one) program module 6205
6204, such program module 6205 includes but is not limited to: operating system, one or more application program, other program moulds
It may include the realization of network environment in block and program data, each of these examples or certain combination.
Bus 630 can be to indicate one of a few class bus structures or a variety of, including storage unit bus or storage
Cell controller, peripheral bus, graphics acceleration port, processing unit use any bus structures in a variety of bus structures
Local bus.
Electronic equipment 600 can also be with one or more external equipments 700 (such as keyboard, sensing equipment, bluetooth equipment
Deng) communication, can also be enabled a user to one or more equipment interact with the electronic equipment 600 communicate, and/or with make
Any equipment (such as the router, modulation /demodulation that the electronic equipment 600 can be communicated with one or more of the other calculating equipment
Device etc.) communication.This communication can be carried out by input/output (I/O) interface 650.Also, electronic equipment 600 can be with
By network adapter 660 and one or more network (such as local area network (LAN), wide area network (WAN) and/or public network,
Such as internet) communication.Network adapter 660 can be communicated by bus 630 with other modules of electronic equipment 600.It should
Understand, although not shown in the drawings, other hardware and/or software module can be used in conjunction with electronic equipment 600, including but unlimited
In: microcode, device driver, redundant processing unit, external disk drive array, RAID system, tape drive and number
According to backup storage platform etc..
The embodiment of the present invention also provides a kind of computer readable storage medium, and for storing program, program is performed reality
The step of scaling method of existing camera and inertial sensor.In some possible embodiments, various aspects of the invention
It is also implemented as a kind of form of program product comprising program code, when program product is run on the terminal device, journey
Sequence code is for executing terminal device described in this specification above-mentioned electronic prescription circulation processing method part according to this hair
The step of bright various illustrative embodiments.
As it appears from the above, the embodiment can decouple the measurement of camera and inertial sensor, enhance the resistance to error,
Due to need to only measure the rotary shaft under respective coordinate system respectively, it is synchronous with inertial sensor data not depend on camera, by this
The fusion accuracy of camera data and inertial sensor data can be improved in scaling method, makes robot precise positioning in the environment.
Fig. 8 is the structural schematic diagram of computer readable storage medium of the invention.Refering to what is shown in Fig. 8, describing according to this
The program product 800 for realizing the above method of the embodiment of invention can use the read-only storage of portable compact disc
Device (CD-ROM) and including program code, and can be run on terminal device, such as PC.However, journey of the invention
Sequence product is without being limited thereto, and in this document, readable storage medium storing program for executing can be any tangible medium for including or store program, the journey
Sequence can be commanded execution system, device or device use or in connection.
Program product can be using any combination of one or more readable mediums.Readable medium can be readable signal Jie
Matter or readable storage medium storing program for executing.Readable storage medium storing program for executing for example can be but be not limited to electricity, magnetic, optical, electromagnetic, infrared ray or partly lead
System, device or the device of body, or any above combination.More specific example (the non exhaustive column of readable storage medium storing program for executing
Table) it include: the electrical connection with one or more conducting wires, portable disc, hard disk, random access memory (RAM), read-only storage
Device (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read only memory (CD-
ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.
Computer readable storage medium may include in a base band or as carrier wave a part propagate data-signal,
In carry readable program code.The data-signal of this propagation can take various forms, including but not limited to electromagnetic signal,
Optical signal or above-mentioned any appropriate combination.Readable storage medium storing program for executing can also be any readable Jie other than readable storage medium storing program for executing
Matter, the readable medium can send, propagate or transmit for by instruction execution system, device or device use or and its
The program of combined use.The program code for including on readable storage medium storing program for executing can transmit with any suitable medium, including but not
It is limited to wireless, wired, optical cable, RF etc. or above-mentioned any appropriate combination.
The program for executing operation of the present invention can be write with any combination of one or more programming languages
Code, programming language include object oriented program language-Java, C++ etc., further include conventional process
Formula programming language-such as " C " language or similar programming language.Program code can be calculated fully in user
It executes in equipment, partly execute on a user device, executing, as an independent software package partially in user calculating equipment
Upper part executes on a remote computing or executes in remote computing device or server completely.It is being related to remotely counting
In the situation for calculating equipment, remote computing device can pass through the network of any kind, including local area network (LAN) or wide area network
(WAN), it is connected to user calculating equipment, or, it may be connected to external computing device (such as utilize ISP
To be connected by internet).
To sum up, the scaling method of camera of the invention and inertial sensor, system, equipment and storage medium, can be by phase
The measurement of machine and inertial sensor decouples, and enhances the resistance to error, due to need to only measure the rotation under respective coordinate system respectively
Shaft, it is synchronous with inertial sensor data not depend on camera, by the scaling method, camera data can be improved and inertia passes
The fusion accuracy of sensor data makes robot precise positioning in the environment.
The above content is a further detailed description of the present invention in conjunction with specific preferred embodiments, and it cannot be said that
Specific implementation of the invention is only limited to these instructions.For those of ordinary skill in the art to which the present invention belongs, exist
Under the premise of not departing from present inventive concept, a number of simple deductions or replacements can also be made, all shall be regarded as belonging to of the invention
Protection scope.
Claims (10)
1. the scaling method of a kind of camera and inertial sensor, applied to the electronic equipment for having camera and inertial sensor,
It is characterized in that, comprising the following steps:
By different shooting postures, i group rotary shaft pair is obtained, every group of rotary shaft is to including obtaining based on the rotation of same rotary shaft
Camera rotary shaft rotAxis under the camera coordinates system obtainedCiAnd the inertial sensor rotary shaft under inertial sensor coordinate system
rotAxisIi, i is natural number；
Acquire the normal direction n of the equidistant point of every group of rotary shaft pairi:
Pass through the normal direction n of the equidistant point of the i group rotary shaft pairiA reference planes are constructed, by the normal direction of the reference planes
Conversion rotary shaft rotAxis as the camera coordinates system to inertial sensor coordinate systemI2C；
When obtaining the small value of error formula, rotation angle rotAngI2CValue, the error formula are as follows:
Obtain the inertial sensor coordinate system to the camera coordinates system spin matrix RI2C, RI2C=rodrigues
(rotAngI2C*rotAxisI2C)。
2. the scaling method of camera as described in claim 1 and inertial sensor, it is characterised in that: the inertial sensor rotation
Shaft rotAxisIi* spin matrix RI2C=camera rotary shaft rotAxisCi, the spin matrix RI2CIt is the matrix of a 3*3.
3. the scaling method of camera as described in claim 1 and inertial sensor, it is characterised in that: obtain one group of rotary shaft pair
Method include:
Camera coordinates system is established, electronic equipment is based on an axial direction and makees dead axle movement, a gridiron pattern is shot by camera, according to
The change in location of angle point in the gridiron pattern obtains the camera rotary shaft rotAxis under camera coordinates systemCi；
Inertial sensor coordinate system is established, the electronic equipment is continued to make dead axle movement based on same axial direction, record rotates through
The numerical value change of journey gyroscope obtains inertial sensor spin matrix and inertial sensor rotary shaft rotAxisIi。
4. the scaling method of camera as described in claim 1 and inertial sensor, it is characterised in that: the camera rotary shaft
rotAxisCiFor a normalized vector (ai,bi,ci), the inertial sensor rotary shaft rotAxisIiFor one normalize to
Measure (di,ei,fi)。
5. the scaling method of camera as described in claim 1 and inertial sensor, it is characterised in that: to every group of rotary shaft
Pair equidistant point normal direction niIt is normalized.
6. the scaling method of camera as described in claim 1 and inertial sensor, it is characterised in that: the rotation of difference group
The rotary shaft of axis pair is respectively different.
7. the calibration system of a kind of camera and inertial sensor, for realizing camera described in any one of claims 1 to 6 with
The scaling method of inertial sensor characterized by comprising
Rotary shaft is to acquisition module, by different shooting postures, obtains i group rotary shaft pair, and every group of rotary shaft is to including being based on
Same rotary shaft rotates and the camera rotary shaft rotAxis under the camera coordinates system of acquisitionCiAnd under inertial sensor coordinate system
Inertial sensor rotary shaft rotAxisIi, i is natural number；
Normal direction obtains module, acquires the normal direction n of the equidistant point of every group of rotary shaft pairi:
Reference planes establish module, pass through the normal direction n of the equidistant point of the i group rotary shaft pairiA reference planes are constructed, it will
Conversion rotary shaft rotAxis of the normal direction of the reference planes as the camera coordinates system to inertial sensor coordinate systemI2C；
Rotation angle obtains module, when obtaining the small value of error formula, rotation angle rotAngI2CValue, the error formula are as follows:
Spin matrix obtain module, obtain the inertial sensor coordinate system to the camera coordinates system spin matrix RI2C,
RI2C=rodrigues (rotAngI2C*rotAxisI2C)。
8. the calibration system of camera as claimed in claim 7 and inertial sensor, it is characterised in that: the inertial sensor rotation
Shaft rotAxisIi* spin matrix RI2C=camera rotary shaft rotAxisCi, the spin matrix RI2CIt is the matrix of a 3*3.
9. the calibration facility of a kind of camera and inertial sensor characterized by comprising
Processor；
Memory, wherein being stored with the executable instruction of the processor；
Wherein, the processor is configured to come any one of perform claim requirement 1 to 6 institute via the execution executable instruction
The step of stating the scaling method of camera and inertial sensor.
10. a kind of computer readable storage medium, for storing program, which is characterized in that described program is performed realization power
Benefit require any one of 1 to 6 described in camera and inertial sensor scaling method the step of.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811398121.0A CN109685852B (en) | 2018-11-22 | 2018-11-22 | Calibration method, system, equipment and storage medium for camera and inertial sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811398121.0A CN109685852B (en) | 2018-11-22 | 2018-11-22 | Calibration method, system, equipment and storage medium for camera and inertial sensor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109685852A true CN109685852A (en) | 2019-04-26 |
| CN109685852B CN109685852B (en) | 2020-08-21 |
Family
ID=66185426
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811398121.0A Active CN109685852B (en) | 2018-11-22 | 2018-11-22 | Calibration method, system, equipment and storage medium for camera and inertial sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109685852B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110443854A (en) * | 2019-08-05 | 2019-11-12 | 兰州交通大学 | Based on fixed target without relative pose scaling method between public view field camera |
| CN111307176A (en) * | 2020-03-02 | 2020-06-19 | 北京航空航天大学青岛研究院 | Online calibration method for visual inertial odometer in VR head-mounted display equipment |
| CN112229424A (en) * | 2020-11-16 | 2021-01-15 | 浙江商汤科技开发有限公司 | Parameter calibration method and device for visual inertial system, electronic equipment and medium |
| WO2021043213A1 (en) * | 2019-09-06 | 2021-03-11 | 深圳市道通智能航空技术有限公司 | Calibration method, device, aerial photography device, and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102162738A (en) * | 2010-12-08 | 2011-08-24 | 中国科学院自动化研究所 | Calibration method of camera and inertial sensor integrated positioning and attitude determining system |
| CN104021554B (en) * | 2014-04-23 | 2017-03-01 | 北京大学深圳研究生院 | Camera inertial sensor scaling method based on operative sensor information |
| EP3247525A1 (en) * | 2015-01-22 | 2017-11-29 | Illinois Tool Works Inc. | Manual tool tracking and guidance with inertial measurement unit |
| CN207923150U (en) * | 2017-08-04 | 2018-09-28 | 广东工业大学 | A kind of calibration system of depth camera and Inertial Measurement Unit relative attitude |
-
2018
- 2018-11-22 CN CN201811398121.0A patent/CN109685852B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102162738A (en) * | 2010-12-08 | 2011-08-24 | 中国科学院自动化研究所 | Calibration method of camera and inertial sensor integrated positioning and attitude determining system |
| CN104021554B (en) * | 2014-04-23 | 2017-03-01 | 北京大学深圳研究生院 | Camera inertial sensor scaling method based on operative sensor information |
| EP3247525A1 (en) * | 2015-01-22 | 2017-11-29 | Illinois Tool Works Inc. | Manual tool tracking and guidance with inertial measurement unit |
| CN207923150U (en) * | 2017-08-04 | 2018-09-28 | 广东工业大学 | A kind of calibration system of depth camera and Inertial Measurement Unit relative attitude |
Non-Patent Citations (2)
| Title |
|---|
| JOERN REHDER 等: "Camera/IMU Calibration Revisited", 《IEEE SENSORS JOURNAL》 * |
| 仇仲江: "基于视觉和惯性传感器的球形机器人运动控制方法研究", 《中国优秀硕士学位论文数据库 信息科技辑》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110443854A (en) * | 2019-08-05 | 2019-11-12 | 兰州交通大学 | Based on fixed target without relative pose scaling method between public view field camera |
| WO2021043213A1 (en) * | 2019-09-06 | 2021-03-11 | 深圳市道通智能航空技术有限公司 | Calibration method, device, aerial photography device, and storage medium |
| CN111307176A (en) * | 2020-03-02 | 2020-06-19 | 北京航空航天大学青岛研究院 | Online calibration method for visual inertial odometer in VR head-mounted display equipment |
| CN111307176B (en) * | 2020-03-02 | 2023-06-16 | 北京航空航天大学青岛研究院 | Online calibration method for visual inertial odometer in VR head-mounted display equipment |
| CN112229424A (en) * | 2020-11-16 | 2021-01-15 | 浙江商汤科技开发有限公司 | Parameter calibration method and device for visual inertial system, electronic equipment and medium |
| CN112229424B (en) * | 2020-11-16 | 2022-04-22 | 浙江商汤科技开发有限公司 | Parameter calibration method and device for visual inertial system, electronic equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109685852B (en) | 2020-08-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109685852A (en) | The scaling method of camera and inertial sensor, system, equipment and storage medium | |
| CN104718561B (en) | The pick up calibration determined based on end point and location estimation | |
| WO2016198009A1 (en) | Heading checking method and apparatus | |
| JP2022510418A (en) | Time synchronization processing method, electronic devices and storage media | |
| CN108389264A (en) | Coordinate system determines method, apparatus, storage medium and electronic equipment | |
| CN109813336A (en) | Inertial Measurement Unit scaling method | |
| CN110567491B (en) | Initial alignment method and device of inertial navigation system and electronic equipment | |
| CN106370178B (en) | Attitude measurement method and device of mobile terminal equipment | |
| CN110352331A (en) | The method and clouds terrace system of the attitude algorithm of hand-held holder | |
| CN109676602A (en) | Self-adapting calibration method, system, equipment and the storage medium of walking robot | |
| CN108801250B (en) | Real-time attitude acquisition method and device based on underwater robot | |
| US11554316B2 (en) | Thermopile array fusion tracking | |
| CN106291567A (en) | A kind of laser radar, laser radar system and data processing method | |
| CN109506617B (en) | Sensor data processing method, storage medium, and electronic device | |
| Peng et al. | Design of an embedded icosahedron mechatronics for robust iterative IMU calibration | |
| CN105157691B (en) | A kind of determination method and device in compass orientation | |
| CN109387219A (en) | Error calibration system | |
| Abbate et al. | Development of a MEMS based wearable motion capture system | |
| CN109099888A (en) | A kind of pose measuring method, equipment and storage medium | |
| CN115919250A (en) | Human dynamic joint angle measuring system | |
| CN108592902A (en) | A kind of positioning device and localization method based on multisensor, system and mechanical arm | |
| Blachuta et al. | Attitude and heading reference system based on 3D complementary filter | |
| CN106289205A (en) | A kind of rhumb self-operated measuring unit | |
| CN108801260B (en) | Data processing method and device based on underwater robot | |
| CN107289935B (en) | Indoor navigation algorithm suitable for wearable equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |