CN109360228A - Pose method for registering between monocular cam and millimetre-wave radar - Google Patents
Pose method for registering between monocular cam and millimetre-wave radar Download PDFInfo
- Publication number
- CN109360228A CN109360228A CN201811212519.0A CN201811212519A CN109360228A CN 109360228 A CN109360228 A CN 109360228A CN 201811212519 A CN201811212519 A CN 201811212519A CN 109360228 A CN109360228 A CN 109360228A
- Authority
- CN
- China
- Prior art keywords
- millimetre
- wave radar
- pose
- coordinate
- monocular cam
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
Abstract
The invention discloses the pose method for registering between a kind of monocular cam and millimetre-wave radar, comprising: and it is additional to place a monocular cam, binocular vision system is formed with monocular cam to be registered, image of the acquisition containing imitative body obtains the space coordinate of imitative body；Affine transformation is carried out by the coordinate by object in millimetre-wave radar coordinate system, is transformed into coordinate of the object in camera coordinate system；Loss function is established, the loss function is abstracted into the function of pose parameter, searches for so that the maximum pose parameter of loss function.What be can be convenient obtains the position orientation relation between millimetre-wave radar and monocular cam, and so as to easily and fast merge the information of visual information and millimetre-wave radar, and registration accuracy is high.
Description
Technical field
The invention belongs to the integration technology fields between millimetre-wave radar and camera, image more particularly to a kind of monocular
Pose method for registering between head and millimetre-wave radar.
Background technique
Automatic Pilot context aware systems are an information redundancy systems, and environment sensing sensor includes such as laser thunder
It reaches, binocular camera, monocular cam, ultrasonic wave, millimetre-wave radar etc..And in this redundancy sensory perceptual system, difference sensing
Information registration and fusion between device are the committed steps in the environmental perception module of automatic Pilot.
Fusion between millimetre-wave radar and camera is a current research hotspot, and cardinal principle is as shown in Figure 1.
Millimetre-wave radar data are exactly a rotational translation matrix [R, t] to the coordinate transform of camera data in fact.For
Statement is convenient, as shown in Figure 1, camera coordinate system is defined as O by uscXcYcZc, millimetre-wave radar coordinate system is defined as
OlXlYlZl, the image coordinate system of video camera is defined as OcUcVc.For space any point M, under camera coordinate system
Coordinate (Xc,Yc,Zc), the coordinate under radar fix system is (Xl,Yl,Zl), the seat under the image coordinate system of video camera
It is designated as (u, v), as follows for the coordinate transformation relation between space in this way:
Dx represents the width of one pixel of x-axis direction, and dy represents the width of a pixel on y-axis direction.Dx, dy are camera shooting
The intrinsic parameter of machine.(u0,v0) it is known as the principal point of the plane of delineation and the intrinsic parameter of video camera.The intrinsic parameter of video camera can pass through
The scaling method of Zhang Zhengyou carries out quickly accurate calibration.In order to express easily internal reference matrix is defined as P by us, and spatial point exists
Camera coordinate system and the vector of radar fix system are defined as Mc,Ml。
Meet between camera coordinate system and millimetre-wave radar coordinate system:
R represents rotation, and T represents translation.R, T is unrelated with video camera, so the two parameters are referred to as the outer parameter of video camera.
With the presence of relation above it can be concluded that the following relationship of camera review coordinate system and radar fix system:
In formula, two matrixes to be asked of only R, T indicate the coordinate system position between millimetre-wave radar and monocular cam
Appearance transformation relation.But can there are problems that 2 in actual use:
1, millimetre-wave radar can only provide two-dimensional signal, i.e., we are using the direction of vehicle advance as Z axis, vehicular transverse direction
For X-axis, vertically downward direction is that Y-axis establishes cartesian coordinate system, and millimetre-wave radar can only provide the information of XZ plane, flat in XZ
The position of barrier in face.
2, monocular cam can not provide ZcInformation, therefore also can not just pass through the sky of (u, v) coordinate value acquired disturbance object
Between coordinate.
Due to the presence of problem above, the correcting scheme of mainstream is mentioned merely by level meter, the tools such as gravimeter at present
Millimetre-wave radar and monocular cam installation accuracy are risen, position orientation relation between the two is then obtained by manual measurement.But
It is such registration accuracy is not very high.
Summary of the invention
In order to solve above-mentioned technical problem, the present invention provides between a kind of monocular cam and millimetre-wave radar
Pose method for registering, it may be convenient to the position orientation relation between millimetre-wave radar and monocular cam is obtained, so as to side
Just quickly the information of visual information and millimetre-wave radar is merged, and registration accuracy is high.
The technical scheme is that
A kind of pose method for registering between monocular cam and millimetre-wave radar, comprising the following steps:
S01: it is additional to place a monocular cam, binocular vision system is formed with monocular cam to be registered, acquisition contains
There is the image of imitative body, obtains the space coordinate of imitative body；
S02: affine transformation is carried out by the coordinate by object in millimetre-wave radar coordinate system, is transformed into object in video camera
Coordinate in coordinate system；
S03: establishing loss function, and the loss function is abstracted into the function of pose parameter, search so that loss function most
Big pose parameter.
In preferred technical solution, the loss function are as follows:
Wherein, N indicates the number of the imitative body of calibration, Xic、ZicIndicate that i-th of imitative body is calculated by binocular vision system
X, Z coordinate, X'ic、Z'icIndicate X, Z coordinate that i-th of imitative body is gone out by millimetre-wave radar pose transformation calculations.
In preferred technical solution, by installing level meter, approach the Z axis relative rotation of millimetre-wave radar and camera
In 0, imitative body will be demarcated and be remotely disposed.
In preferred technical solution, the pose each time in the step S03 updates, and needs to recalculate millimetre-wave radar
The coordinate value to be formed is converted by pose, the superiority and inferiority of this pose parameter is then evaluated according to loss function, by specifically searching
Rope algorithm is searched for so that the maximum pose parameter of loss function.
Compared with prior art, the invention has the advantages that
1, what be can be convenient obtains the position orientation relation between millimetre-wave radar and monocular cam, so as to convenient fast
The information by visual information and millimetre-wave radar of speed merges, and avoids in installation process carrying out by device repeatedly
The complex process individually debugged.Very easily pose can be registrated, substantially increase registration accuracy.
2, by being based on classification design loss function, there is extremely strong robustness and accuracy.
Detailed description of the invention
The invention will be further described with reference to the accompanying drawings and embodiments:
Pose of the Fig. 1 between millimetre-wave radar and camera converts schematic diagram；
The flow chart of pose method for registering of the Fig. 2 between monocular cam of the present invention and millimetre-wave radar；
Fig. 3 is that pose parameter updates flow chart.
Specific embodiment
In order to make the objectives, technical solutions and advantages of the present invention clearer, With reference to embodiment and join
According to attached drawing, the present invention is described in more detail.It should be understood that these descriptions are merely illustrative, and it is not intended to limit this hair
Bright range.In addition, in the following description, descriptions of well-known structures and technologies are omitted, to avoid this is unnecessarily obscured
The concept of invention.
Embodiment:
With reference to the accompanying drawing, presently preferred embodiments of the present invention is described further.
As shown in Fig. 2, the pose method for registering between a kind of monocular cam and millimetre-wave radar, comprising the following steps:
A kind of pose method for registering between monocular cam and millimetre-wave radar, comprising the following steps:
S01: it is additional to place a monocular cam, binocular vision system is formed with monocular cam to be registered, acquisition contains
There is the image of imitative body, obtains the space coordinate of imitative body；
S02: affine transformation is carried out by the coordinate by object in millimetre-wave radar coordinate system, is transformed into object in video camera
Coordinate in coordinate system；
S03: establishing loss function, and the loss function is abstracted into the function of pose parameter, search so that loss function most
Big pose parameter.
Camera is mainly used for the visual perception of ambient enviroment.In the present solution, needing to imitate using imitative body (Phantom)
Body is a vertical bar, in this way, image very easily can be subjected to disparity map acquisition, so that its space coordinate is calculated,
Bar is also easy to be detected by millimetre-wave radar simultaneously.Camera and PC processing terminal are transmitted by usb protocol or network interface agreement.
Primarily with respect to the problem 1 in background technique, millimetre-wave radar is merely capable of providing (Xl,Zl) information (formula
2), then the calculated coordinate under camera coordinate system is all unknown after carrying out pose transformation by formula 2.This is asked
Topic can be by solving when installing millimetre-wave radar and camera.Spin matrix in formula 2 is transformed into rotation angle
Degree indicates are as follows:
Wherein, α, beta, gamma indicate millimetre-wave radar relative to camera coordinate system respectively along X, Y, the angle of Z axis rotation
Value.From formula 4 as can be seen that by millimetre-wave radar coordinate value (Xl,Zl) be converted to camera coordinate system (Xc,Zc) when, it is sat with X
It is designated as example:
Xc=Xl*cosβcosγ+cosβsinγ*Yl-sinβ*Zl (5)
Unknown coordinate YlCalculating is taken part in, X is causedcCalculating goes wrong.As long as guaranteeing that γ angle levels off to as far as possible as far as possible
0, then unknown YlIt would not influence to calculate.Therefore, during actually correction, two kinds of means is taken to solve the problems, such as 1.
1, it is installed by level meter, keeps the Z axis relative rotation of millimetre-wave radar and camera as small as possible.
2, imitative body will be demarcated as far as possible to place at a distance, because of YlIndicate that the imitative coordinate of body in the vertical direction, coordinate value exist
[- 1,1] section (can extremely readily satisfy), increases X, and the coordinate of Z can make XcWhen calculating, relative error becomes smaller.
By both means, (X can be calculated with approximate exactc,Zc)。
About the problems in background technique 2, use increase an interim camera with construct the method for biocular systems into
Row solves.Biocular systems can obtain the space coordinate of imitative body, i.e. (X by formula 3 with high accuracyc,Yc,Zc)。
(the X obtained by biocular systemsc,Zc) (the X that obtains of coordinate and millimetre-wave radarc,Zc) coordinate foundation loss letter
Number, loss function can be a variety of, simplest such as root-mean-square error:
Wherein N indicates the number of the imitative body of calibration, Xic、ZicIndicate that i-th of imitative body is calculated by binocular vision system
X, Z coordinate, X'ic、Z’icIndicate X, Z coordinate that i-th of imitative body is gone out by millimetre-wave radar pose transformation calculations.
Loss function is abstracted into the function of pose parameter, and pose updates each time, needs to recalculate millimetre-wave radar warp
It crosses pose and converts the coordinate value to be formed, then bring loss function calculation formula into, then evaluate the superiority and inferiority of this pose parameter,
And continue to update.
It establishes after loss function, so that it may R, the optimal value of T are found by some searching algorithms.
Specific pose parameter more new technological process is as shown in Figure 3:
A monocular cam is additionally placed, binocular vision system is formed with monocular cam to be registered, obtains camera shooting
Machine image and millimetre-wave radar coordinate points.Initialize R, T (pose parameter).
By current pose parameter, loss function value is calculated according to the loss function calculation method of design.
New pose parameter is obtained by gradient descent method, and calculates new loss function value.
Older loss function value and new loss function value decide whether to jump out circulation.
By gradient descent method, it is extremely easily trapped into local minimum, and uses some global search strategies, as simulation is moved back
Fire can solve this problem.
It should be noted that the more new algorithm of this flow chart is gradient descent method, other search plans also can be used
It omits, such as Stochastic gradient method, simulated annealing, many searching algorithms such as genetic algorithm.
It should be understood that above-mentioned specific embodiment of the invention is used only for exemplary illustration or explains of the invention
Principle, but not to limit the present invention.Therefore, that is done without departing from the spirit and scope of the present invention is any
Modification, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.In addition, appended claims purport of the present invention
Covering the whole variations fallen into attached claim scope and boundary or this range and the equivalent form on boundary and is repairing
Change example.
Claims (4)
1. the pose method for registering between a kind of monocular cam and millimetre-wave radar, which comprises the following steps:
S01: it is additional to place a monocular cam, binocular vision system is formed with monocular cam to be registered, acquisition is containing imitative
The image of body obtains the space coordinate of imitative body；
S02: affine transformation is carried out by the coordinate by object in millimetre-wave radar coordinate system, is transformed into object in camera coordinates
Coordinate in system；
S03: establishing loss function, and the loss function is abstracted into the function of pose parameter, searches for so that loss function is maximum
Pose parameter.
2. the pose method for registering between monocular cam according to claim 1 and millimetre-wave radar, which is characterized in that institute
State loss function are as follows:
Wherein, N indicates the number of the imitative body of calibration, Xic、ZicIndicate that i-th of imitative body passes through binocular vision system calculated X, Z
Coordinate, X'ic、Z’icIndicate X, Z coordinate that i-th of imitative body is gone out by millimetre-wave radar pose transformation calculations.
3. the pose method for registering between monocular cam according to claim 1 and millimetre-wave radar, which is characterized in that logical
Installation level meter is crossed, the Z axis relative rotation of millimetre-wave radar and camera is made to level off to 0, imitative body will be demarcated and be remotely disposed.
4. the pose method for registering between monocular cam according to claim 1 and millimetre-wave radar, which is characterized in that institute
The pose each time stated in step S03 updates, and needs to recalculate millimetre-wave radar by pose and converts the coordinate value to be formed, so
The superiority and inferiority for evaluating this pose parameter according to loss function afterwards is searched for by specific searching algorithm so that loss function is maximum
Pose parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811212519.0A CN109360228A (en) | 2018-10-18 | 2018-10-18 | Pose method for registering between monocular cam and millimetre-wave radar |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811212519.0A CN109360228A (en) | 2018-10-18 | 2018-10-18 | Pose method for registering between monocular cam and millimetre-wave radar |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109360228A true CN109360228A (en) | 2019-02-19 |
Family
ID=65345673
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811212519.0A Pending CN109360228A (en) | 2018-10-18 | 2018-10-18 | Pose method for registering between monocular cam and millimetre-wave radar |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109360228A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111060881A (en) * | 2020-01-10 | 2020-04-24 | 湖南大学 | Millimeter wave radar external parameter online calibration method |
| CN111098815A (en) * | 2019-11-11 | 2020-05-05 | 武汉市众向科技有限公司 | ADAS front vehicle collision early warning method based on monocular vision fusion millimeter waves |
| CN111538029A (en) * | 2020-04-24 | 2020-08-14 | 江苏盛海智能科技有限公司 | Vision and radar fusion measuring method and terminal |
| CN111898582A (en) * | 2020-08-13 | 2020-11-06 | 清华大学苏州汽车研究院（吴江） | Obstacle information fusion method and system for binocular camera and millimeter wave radar |
| CN114782556A (en) * | 2022-06-20 | 2022-07-22 | 季华实验室 | Camera and laser radar registration method, system and storage medium |
| CN116106895A (en) * | 2023-02-21 | 2023-05-12 | 郑州大学 | Multi-point micro-motion intelligent monitoring system and method based on vision and wireless sensing fusion |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106228537A (en) * | 2016-07-12 | 2016-12-14 | 北京理工大学 | A kind of three-dimensional laser radar and the combined calibrating method of monocular-camera |
| CN107564069A (en) * | 2017-09-04 | 2018-01-09 | 北京京东尚科信息技术有限公司 | The determination method, apparatus and computer-readable recording medium of calibrating parameters |
| CN107886477A (en) * | 2017-09-20 | 2018-04-06 | 武汉环宇智行科技有限公司 | Unmanned neutral body vision merges antidote with low line beam laser radar |
| CN108489496A (en) * | 2018-04-28 | 2018-09-04 | 北京空间飞行器总体设计部 | Noncooperative target Relative Navigation method for estimating based on Multi-source Information Fusion and system |
-
2018
- 2018-10-18 CN CN201811212519.0A patent/CN109360228A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106228537A (en) * | 2016-07-12 | 2016-12-14 | 北京理工大学 | A kind of three-dimensional laser radar and the combined calibrating method of monocular-camera |
| CN107564069A (en) * | 2017-09-04 | 2018-01-09 | 北京京东尚科信息技术有限公司 | The determination method, apparatus and computer-readable recording medium of calibrating parameters |
| CN107886477A (en) * | 2017-09-20 | 2018-04-06 | 武汉环宇智行科技有限公司 | Unmanned neutral body vision merges antidote with low line beam laser radar |
| CN108489496A (en) * | 2018-04-28 | 2018-09-04 | 北京空间飞行器总体设计部 | Noncooperative target Relative Navigation method for estimating based on Multi-source Information Fusion and system |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111098815A (en) * | 2019-11-11 | 2020-05-05 | 武汉市众向科技有限公司 | ADAS front vehicle collision early warning method based on monocular vision fusion millimeter waves |
| CN111060881A (en) * | 2020-01-10 | 2020-04-24 | 湖南大学 | Millimeter wave radar external parameter online calibration method |
| CN111060881B (en) * | 2020-01-10 | 2022-05-13 | 湖南大学 | Millimeter wave radar external parameter online calibration method |
| CN111538029A (en) * | 2020-04-24 | 2020-08-14 | 江苏盛海智能科技有限公司 | Vision and radar fusion measuring method and terminal |
| CN111898582A (en) * | 2020-08-13 | 2020-11-06 | 清华大学苏州汽车研究院（吴江） | Obstacle information fusion method and system for binocular camera and millimeter wave radar |
| CN111898582B (en) * | 2020-08-13 | 2023-09-12 | 清华大学苏州汽车研究院（吴江） | Obstacle information fusion method and system for binocular camera and millimeter wave radar |
| CN114782556A (en) * | 2022-06-20 | 2022-07-22 | 季华实验室 | Camera and laser radar registration method, system and storage medium |
| CN114782556B (en) * | 2022-06-20 | 2022-09-09 | 季华实验室 | Camera and laser radar registration method and system and storage medium |
| CN116106895A (en) * | 2023-02-21 | 2023-05-12 | 郑州大学 | Multi-point micro-motion intelligent monitoring system and method based on vision and wireless sensing fusion |
| CN116106895B (en) * | 2023-02-21 | 2024-01-26 | 郑州大学 | Multi-point micro-motion intelligent monitoring system and method based on vision and wireless sensing fusion |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109360228A (en) | Pose method for registering between monocular cam and millimetre-wave radar | |
| CN104200086B (en) | Wide-baseline visible light camera pose estimation method | |
| CN104677340B (en) | Point character based monocular vision pose measurement method | |
| US9355453B2 (en) | Three-dimensional measurement apparatus, model generation apparatus, processing method thereof, and non-transitory computer-readable storage medium | |
| US11830216B2 (en) | Information processing apparatus, information processing method, and storage medium | |
| CN106774431A (en) | One kind mapping unmanned plane route planning method and device | |
| CN108885791A (en) | ground detection method, related device and computer readable storage medium | |
| CN109035309A (en) | Pose method for registering between binocular camera and laser radar based on stereoscopic vision | |
| CN107567412A (en) | Measured using vehicle camera using the object's position of vehicle motion data | |
| CN106408601B (en) | A kind of binocular fusion localization method and device based on GPS | |
| JP6516558B2 (en) | Position information processing method | |
| CN109471096B (en) | Multi-sensor target matching method and device and automobile | |
| CN104704384A (en) | Image processing method, particularly used in a vision-based localization of a device | |
| CN110123257A (en) | A kind of vision testing method, device, sight tester and computer storage medium | |
| CN106651990A (en) | Indoor map construction method and indoor map-based indoor locating method | |
| CN103487033B (en) | River surface photographic surveying method based on height-change homography | |
| US20210082132A1 (en) | Laser sensor-based map generation | |
| CN110849331B (en) | Monocular vision measurement and ground test method based on three-dimensional point cloud database model | |
| CN106595578A (en) | Multi-sensor information fusion-based unmanned aerial vehicle height measurement method and system | |
| CN108470356A (en) | A kind of target object fast ranging method based on binocular vision | |
| CN109345513A (en) | A kind of cigarette package defect inspection method with cigarette packet Attitude Calculation | |
| CN108648242A (en) | Two camera scaling methods and device without public view field are assisted based on laser range finder | |
| CN112017236A (en) | Method and device for calculating position of target object based on monocular camera | |
| CN106157322B (en) | A kind of camera installation site scaling method based on plane mirror | |
| CN103411587A (en) | Positioning and attitude-determining method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190219 |