CN109859270A - A kind of human eye three-dimensional coordinate localization method and separate type binocular camera shooting device - Google Patents
A kind of human eye three-dimensional coordinate localization method and separate type binocular camera shooting device Download PDFInfo
- Publication number
- CN109859270A CN109859270A CN201811432460.6A CN201811432460A CN109859270A CN 109859270 A CN109859270 A CN 109859270A CN 201811432460 A CN201811432460 A CN 201811432460A CN 109859270 A CN109859270 A CN 109859270A
- Authority
- CN
- China
- Prior art keywords
- camera
- coordinate system
- coordinate
- binocular
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The present invention relates to field of image processings, and in particular to a kind of human eye three-dimensional coordinate localization method and separate type binocular camera shooting device.The present invention calculates the inside and outside parameter for obtaining binocular camera shooting device the following steps are included: S1；S2, the image using the shooting of binocular camera shooting device containing interior driver；S3 carries out eye recognition to image, calculates the coordinate value of driver's binocular central point projection；S4, it establishes first camera and coordinate points (Xw of driver's binocular central point in world coordinate system is calculated according to three-dimensional coordinate positioning projection relationship in the second coordinate system O2xyz of focal point in the first coordinate system O1xyz of focal point, the second camera, Yw, Zw).The present invention is based on binocular camera shooting devices, then carry out image procossing by eye recognition algorithm and three-dimensional coordinate positioning projection relationship, so that the world coordinate system coordinate points of driver's binocular central point be calculated.
Description
Technical field
The present invention relates to field of image processings, and in particular to a kind of human eye three-dimensional coordinate localization method and separate type binocular are taken the photograph
As device.
Background technique
Currently, the target three-dimensional coordinate localization method based on binocular camera is more mature, but the method is usually to be directed to
Left and right camera is closer, the relatively simple scene in relative position and realize.And car cab space is limited, camera
Arrangement must comply with the particular/special requirement of in-vehicle device position so that short distance binocular camera is difficult to arrange success in the car.
Summary of the invention
The present invention is in order to solve the above technical problems, provide a kind of human eye three-dimensional coordinate localization method, system and binocular camera shooting
Device
A kind of human eye three-dimensional coordinate localization method, comprising the following steps:
S1 calculates the second internal reference for obtaining the first internal reference matrix A 1 of the first camera in binocular camera shooting device, second camera
Outer ginseng matrix M3 between matrix A 2 and first camera, the second camera；
S2, the image using the shooting of binocular camera shooting device containing interior driver, first camera shoot to obtain the first figure
Picture, the second camera shoot to obtain the second image；
S3 carries out eye recognition to the first image and second image, calculates driver's binocular central point projection
Coordinate value, obtain first object point (U1, V1), second image of binocular central point projection in the first image
The second target point (U2, V2) of middle binocular central point projection；
S4 establishes first camera in the first coordinate system O1xyz of focal point, the second camera the of focal point
Driver's binocular central point is calculated in world coordinate system according to three-dimensional coordinate positioning projection relationship in two coordinate system O2xyz
Coordinate points (Xw, Yw, Zw).
The technical program has the beneficial effect that
Image procossing is carried out based on binocular camera shooting device, then by eye recognition algorithm and three-dimensional coordinate positioning projection relationship, from
And the world coordinate system coordinate points of driver's binocular central point are calculated.
As the preferred of this programme, the formula of internal reference matrix is calculated in the step S1 are as follows:
Wherein, f is focal length, and a is the Pixel Dimensions of the first image, and b is the Pixel Dimensions of second image, and u0 is described
The imaging center point coordinate of first camera, v0 are the imaging center point coordinate of the second camera.
The formula of outer ginseng matrix is calculated as the preferred of this programme, in the step S1 are as follows:
Wherein, R is during from world coordinate system to image projection coordinate system around x-axis, y-axis, z-axis rotation angle information, T be from
World coordinate system to image projection coordinate system around during from world coordinate system to image projection coordinate system around x-axis, y-axis, z-axis side
To translation information.
As the preferred of this programme, the step S4 includes:
S4.1, first camera establish the first coordinate system in focal point, and the second camera establishes second in focal point
Coordinate system, wherein z-axis direction is overlapped with camera direction；
S4.2 obtains binocular central point in the first coordinate points (X1, Y1, Z1) of first coordinate system, binocular central point in institute
State the second coordinate points (X2, Y2, Z2) of the second coordinate system；
S4.3 calculates acquisition driver's binocular central point according to coordinate setting formula and the similar triangles relation formula of imaging and exists
Coordinate points (Xw, Yw, Zw) in world coordinate system.
As the preferred of this programme, in the step S4.3, coordinate setting formula are as follows:
Wherein, X, Y, Z are coordinate points (Xw, Yw, Zw) of driver's binocular central point in world coordinate system in x-axis, y-axis, z-axis
On value, the target point (U, V) of binocular central point projection.
A kind of separate type binocular camera shooting device, including device operator seat front upper place and central controller distance in automobile are close
The first camera and device in automobile operator seat front upper place and with central controller apart from remote second camera, described
The shortest distance of one camera to driver's seat is not equal to the second camera to the shortest distance of driver's seat.
As the preferred of this programme, the height of the second camera is lower than the height of first camera.
As the preferred of this programme, the distance between first camera and the second camera are not less than 400mm.
Detailed description of the invention
Fig. 1 is a kind of flow chart of human eye three-dimensional coordinate localization method of the present invention；
Fig. 2 is the flow chart of step S4 of the present invention；
Fig. 3 is the schematic diagram of three-dimensional coordinate positioning projection relationship.
Specific embodiment
This specific embodiment is only explanation of the invention, is not the limitation to invention, those skilled in the art
It can according to need the modification that not creative contribution is made to the present embodiment after reading this specification, but as long as in this hair
The protection of Patent Law is all received in bright scope of the claims.
Embodiment one
In current vehicle intellectualized progress, many traditional meter key disks are shown by liquid crystal display even line holographic projections platform to be replaced,
But the liquid crystal display in vehicle or line holographic projections platform are aobvious positioned at fixed position mostly.But the driver of a vehicle is not necessarily eternal
An only people, it may be possible to which household is also possible to the user for driving personnel or automobile leasing in generation, everyone physical qualification is not
Together, cause everyone optimal viewing angle also not identical, so to automatically adjust the optimal viewing angle for meeting driver in instrument disk
Before, we first have to that the optimal viewing angle that everyone is in vehicle can be obtained.
So being based on binocular camera shooting device, then know by human eye we provide a kind of human eye three-dimensional coordinate localization method
Other algorithm and three-dimensional coordinate positioning projection relationship carry out image procossing, so that the world that driver's binocular central point is calculated is sat
Mark system coordinate points.Possibility just is provided for the optimization of subsequent automobile in this way.
As shown in Figure 1-3, for a kind of flow chart of human eye three-dimensional coordinate localization method of the present invention.
A kind of human eye three-dimensional coordinate localization method, comprising the following steps:
S1 calculates the second internal reference for obtaining the first internal reference matrix A 1 of the first camera in binocular camera shooting device, second camera
Outer ginseng matrix M3 between matrix A 2 and first camera, the second camera.
Further, the interior focal length for participating in camera, image resolution ratio are related.Internal reference matrix is calculated in the step S1
Formula are as follows:
Wherein, f is focal length, and a is the Pixel Dimensions of the first image, and b is the Pixel Dimensions of second image, and u0 is described
The imaging center point coordinate of first camera, v0 are the imaging center point coordinate of the second camera.
Further, outer ginseng is then imaging object from world coordinate system to the transform of image projection coordinate system, the step
The formula of outer ginseng matrix is calculated in rapid S1 are as follows:
Wherein, R is during from world coordinate system to image projection coordinate system around x-axis, y-axis, z-axis rotation angle information, T be from
World coordinate system to image projection coordinate system around during from world coordinate system to image projection coordinate system around x-axis, y-axis, z-axis side
To translation information.
S2, using image of the binocular camera shooting device shooting containing interior driver, first camera shoots to obtain the
One image, the second camera shoot to obtain the second image.
S3 carries out eye recognition to the first image and second image, calculates the imaging of driver's binocular central point
The coordinate value of projection, obtain binocular central point projection in the first image first object point (U1, V1), described second
The second target point (U2, V2) of binocular central point projection in image.
By deep learning algorithm, rectangle position of the human eye in two camera images is obtained, using eyes center as target
Point, position of the eyes center in two camera images available (U1, V1) and (U2, V2).
S4 establishes first camera in the first coordinate system O1xyz of focal point, the second camera in focal point
The second coordinate system O2xyz driver's binocular central point is calculated in world coordinates according to three-dimensional coordinate positioning projection relationship
Coordinate points (Xw, Yw, Zw) in system.
Shown in three-dimensional coordinate positioning projection relationship, human eye is denoted as (Xw, Yw, Zw) in the coordinate of world coordinate system.Two are taken the photograph
As head establishes coordinate system O1xyz and O2xyz in focal point respectively, wherein the direction z is overlapped with camera direction.Human eye is in camera
1, the coordinate on 2 is respectively (X1, Y1, Z1) and (X2, Y2, Z2).Projection coordinate of the human eye on camera 1 be denoted as (U1,
V1), the projection coordinate on camera 2 is denoted as (U2, V2).The focal length of two cameras is denoted as f.
Further, by demarcating available following calculating formula 1 and calculating formula 2:
Formula 1
Formula 2
The relative pose of first camera, the second camera in the coordinate system O1xyz and O2xyz of focal point is fixed
(can be measured by mechanical structure), then its spin matrix is denoted as M3, then such as calculating formula 3:
Formula 3
It can be obtained by the similar triangles relationship of two camera imagings, such as calculating formula 4, calculating formula 5:
Formula 4
Formula 5
Further, calculating formula 6 can be obtained by calculating formula 4, calculating formula 5 being substituted into calculating formula 3 respectively:
Formula 6
Further, after Z1, Z2 being calculated by calculating formula 6, calculating formula 1, calculating formula 2 is substituted into, human eye can be calculated
Coordinate (Xw, Yw, Zw) in world coordinate system.
The technical program is beneficial in that, is based on binocular camera shooting device, is intersected camera by moderate distance first
Scaling method calculates the inside and outside parameter matrix data for obtaining dual camera.It is positioned again by eye recognition algorithm and three-dimensional coordinate
Projection relation carries out image procossing, so that the world coordinate system coordinate points of driver's binocular central point be calculated.
Embodiment 2
A kind of separate type binocular camera shooting device, including device in automobile operator seat front upper place and central controller distance it is close the
One camera and device operator seat front upper place and are taken the photograph with central controller apart from remote second camera, described first in automobile
As head to driver's seat the shortest distance not equal to the second camera to driver's seat the shortest distance.
Further, the second camera is preferably arranged on the A column of automobile, and first camera is preferably set
It sets on the rearview mirror column spinner of automobile and (camera is arranged in A column, rearview mirror, instrument board, console etc., three-dimensional can be played
The place of positioning, is not limited solely on preferred A column and rearview mirror column spinner).
Further, not being connected with automobile B-column between first camera and the second camera, it is flat to be formed
Face is parallel.
Further, there are certain angle, the angle tools between first camera and the second camera
Body is adjusted according to the eye locations of driver.
Currently, the target three-dimensional coordinate localization method based on binocular camera is more mature, but the method is usually to be directed to
Left and right camera is closer, the relatively simple scene in relative position and realize.And car cab space is limited, camera
Arrangement must comply with the particular/special requirement of in-vehicle device position so that short distance binocular camera is difficult to arrange success in the car.
Camera will carry out separation arrangement in a flexible way, and becoming larger with two camera relative distance, and the change of angle
Change and needs to carry out specially treated in software algorithm.
The height of the second camera is lower than the height of first camera.
The distance between first camera and the second camera are not less than 400mm.
Claims (9)
1. a kind of human eye three-dimensional coordinate localization method, which comprises the following steps:
S1 calculates the second internal reference for obtaining the first internal reference matrix A 1 of the first camera in binocular camera shooting device, second camera
Outer ginseng matrix M3 between matrix A 2 and first camera, the second camera；
S2, the image using the shooting of binocular camera shooting device containing interior driver, first camera shoot to obtain the first figure
Picture, the second camera shoot to obtain the second image；
S3 carries out eye recognition to the first image and second image, calculates driver's binocular central point projection
Coordinate value, obtain first object point (U1, V1), second image of binocular central point projection in the first image
The second target point (U2, V2) of middle binocular central point projection；
S4 establishes first camera in the first coordinate system O1xyz of focal point, the second camera the of focal point
Driver's binocular central point is calculated in world coordinate system according to three-dimensional coordinate positioning projection relationship in two coordinate system O2xyz
Coordinate points (Xw, Yw, Zw).
2. a kind of human eye three-dimensional coordinate localization method according to claim 1, which is characterized in that calculated in the step S1
The formula of internal reference matrix are as follows:
Wherein, f is focal length, and a is the Pixel Dimensions of the first image, and b is the Pixel Dimensions of second image, and u0 is described
The imaging center point coordinate of first camera, v0 are the imaging center point coordinate of the second camera.
3. a kind of human eye three-dimensional coordinate localization method according to claim 2, which is characterized in that calculated in the step S1
The formula of outer ginseng matrix are as follows:
Wherein, R is during from world coordinate system to image projection coordinate system around x-axis, y-axis, z-axis rotation angle information, T be from
World coordinate system to image projection coordinate system around during from world coordinate system to image projection coordinate system around x-axis, y-axis, z-axis side
To translation information.
4. a kind of human eye three-dimensional coordinate localization method according to claim 3, which is characterized in that the step S4 includes:
S4.1, first camera establish the first coordinate system in focal point, and the second camera establishes second in focal point
Coordinate system, wherein z-axis direction is overlapped with camera direction；
S4.2 obtains binocular central point in the first coordinate points (X1, Y1, Z1) of first coordinate system, binocular central point in institute
State the second coordinate points (X2, Y2, Z2) of the second coordinate system；
S4.3 calculates acquisition driver's binocular central point according to coordinate setting formula and the similar triangles relation formula of imaging and exists
Coordinate points (Xw, Yw, Zw) in world coordinate system.
5. a kind of human eye three-dimensional coordinate localization method according to claim 4, which is characterized in that in the step S4.3,
Coordinate setting formula are as follows:
Wherein, X, Y, Z are coordinate points (Xw, Yw, Zw) of driver's binocular central point in world coordinate system in x-axis, y-axis, z-axis
On value, the target point (U, V) of binocular central point projection.
6. a kind of human eye three-dimensional coordinate localization method according to claim 4, which is characterized in that in the step S4.3,
The similar triangles relation formula of imaging are as follows:
。
7. a kind of separate type binocular camera shooting device, which is characterized in that including device, operator seat front upper place and center are controlled in automobile
Device processed distance close the first camera and device operator seat front upper place and remote with central controller distance second are taken the photograph in automobile
As head, the shortest distance of first camera to driver's seat is not equal to the second camera to the most short distance of driver's seat
From.
8. a kind of separate type binocular camera shooting device according to claim 7, which is characterized in that the height of the second camera
Degree is lower than the height of first camera.
9. a kind of separate type binocular camera shooting device according to claim 7, which is characterized in that first camera and institute
The distance between second camera is stated not less than 400mm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811432460.6A CN109859270A (en) | 2018-11-28 | 2018-11-28 | A kind of human eye three-dimensional coordinate localization method and separate type binocular camera shooting device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811432460.6A CN109859270A (en) | 2018-11-28 | 2018-11-28 | A kind of human eye three-dimensional coordinate localization method and separate type binocular camera shooting device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109859270A true CN109859270A (en) | 2019-06-07 |
Family
ID=66890299
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811432460.6A Pending CN109859270A (en) | 2018-11-28 | 2018-11-28 | A kind of human eye three-dimensional coordinate localization method and separate type binocular camera shooting device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109859270A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111723716A (en) * | 2020-06-11 | 2020-09-29 | 深圳地平线机器人科技有限公司 | Method, device, system, medium and electronic equipment for determining orientation of target object |
| CN112298039A (en) * | 2020-09-27 | 2021-02-02 | 浙江合众新能源汽车有限公司 | A-column imaging method |
| CN114596822A (en) * | 2020-12-04 | 2022-06-07 | 福州京东方光电科技有限公司 | Display method, display optimization device, electronic device, and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040227993A1 (en) * | 2003-05-13 | 2004-11-18 | Takahiro Amanai | Binocular image display |
| EP2805671A2 (en) * | 2013-05-23 | 2014-11-26 | Stiftung caesar - center of advanced european studies and research assoziiert mit der Max-Planck-Gesellschaft | Ocular videography system |
| CN105989326A (en) * | 2015-01-29 | 2016-10-05 | 北京三星通信技术研究有限公司 | Method and device for determining three-dimensional position information of human eyes |
| CN107124607A (en) * | 2017-05-08 | 2017-09-01 | 上海大学 | The naked-eye stereoscopic display device and method of a kind of combination visual fatigue detection |
| CN107341438A (en) * | 2017-01-24 | 2017-11-10 | 问众智能信息科技（北京）有限公司 | The method and apparatus of in-car safety monitoring based on computer vision |
| CN107481288A (en) * | 2017-03-31 | 2017-12-15 | 触景无限科技（北京）有限公司 | The inside and outside ginseng of binocular camera determines method and apparatus |
-
2018
- 2018-11-28 CN CN201811432460.6A patent/CN109859270A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040227993A1 (en) * | 2003-05-13 | 2004-11-18 | Takahiro Amanai | Binocular image display |
| EP2805671A2 (en) * | 2013-05-23 | 2014-11-26 | Stiftung caesar - center of advanced european studies and research assoziiert mit der Max-Planck-Gesellschaft | Ocular videography system |
| CN105989326A (en) * | 2015-01-29 | 2016-10-05 | 北京三星通信技术研究有限公司 | Method and device for determining three-dimensional position information of human eyes |
| CN107341438A (en) * | 2017-01-24 | 2017-11-10 | 问众智能信息科技（北京）有限公司 | The method and apparatus of in-car safety monitoring based on computer vision |
| CN107481288A (en) * | 2017-03-31 | 2017-12-15 | 触景无限科技（北京）有限公司 | The inside and outside ginseng of binocular camera determines method and apparatus |
| CN107124607A (en) * | 2017-05-08 | 2017-09-01 | 上海大学 | The naked-eye stereoscopic display device and method of a kind of combination visual fatigue detection |
Non-Patent Citations (2)
| Title |
|---|
| 蒋昌明: "基于机器视觉与面部朝向的驾驶员注意力分散检测研究", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》 * |
| 邱会贵 等: "双目立体视觉定位的三维坐标提取与分析", 《闽江学院学报》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111723716A (en) * | 2020-06-11 | 2020-09-29 | 深圳地平线机器人科技有限公司 | Method, device, system, medium and electronic equipment for determining orientation of target object |
| CN111723716B (en) * | 2020-06-11 | 2024-03-08 | 深圳地平线机器人科技有限公司 | Method, device, system, medium and electronic equipment for determining target object orientation |
| CN112298039A (en) * | 2020-09-27 | 2021-02-02 | 浙江合众新能源汽车有限公司 | A-column imaging method |
| CN114596822A (en) * | 2020-12-04 | 2022-06-07 | 福州京东方光电科技有限公司 | Display method, display optimization device, electronic device, and storage medium |
| CN114596822B (en) * | 2020-12-04 | 2024-01-19 | 福州京东方光电科技有限公司 | Display method, display optimization device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111223038B (en) | Automatic splicing method of vehicle-mounted looking-around images and display device | |
| US9866752B2 (en) | Systems and methods for producing a combined view from fisheye cameras | |
| US9858639B2 (en) | Imaging surface modeling for camera modeling and virtual view synthesis | |
| CN105678693B (en) | Panoramic video browses playback method | |
| CN104835117B (en) | Spherical panorama generation method based on overlapping mode | |
| JP3871614B2 (en) | Driving assistance device | |
| CN106101689B (en) | The method that using mobile phone monocular cam virtual reality glasses are carried out with augmented reality | |
| JP5739584B2 (en) | 3D image synthesizing apparatus and method for visualizing vehicle periphery | |
| TWI320756B (en) | ||
| JP5455124B2 (en) | Camera posture parameter estimation device | |
| JP6522630B2 (en) | Method and apparatus for displaying the periphery of a vehicle, and driver assistant system | |
| WO2023071834A1 (en) | Alignment method and alignment apparatus for display device, and vehicle-mounted display system | |
| JP2018531530A (en) | Method and apparatus for displaying surrounding scene of vehicle / towed vehicle combination | |
| JP2018531530A6 (en) | Method and apparatus for displaying surrounding scene of vehicle / towed vehicle combination | |
| CN109859270A (en) | A kind of human eye three-dimensional coordinate localization method and separate type binocular camera shooting device | |
| CN111739101B (en) | Device and method for eliminating dead zone of vehicle A column | |
| CN112655024A (en) | Image calibration method and device | |
| CN110298924A (en) | For showing the coordinate transformation method of detection information in a kind of AR system | |
| JP2002159019A (en) | Display control device, imaging position estimating device, display system, image pickup system, image positioning method, imaging position estimating method, and recording medium recorded with process program | |
| CN108174089B (en) | Backing image splicing method and device based on binocular camera | |
| US20160037154A1 (en) | Image processing system and method | |
| JP5405809B2 (en) | Vehicle periphery monitoring device and camera position / posture determination method | |
| CN206287929U (en) | A kind of 3D panoramas parking apparatus and system | |
| US20170026572A1 (en) | Rear cross traffic - quick looks | |
| WO2022133683A1 (en) | Mixed reality display method, mixed reality device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190607 |