CN113920197A - Method for assisting camera to automatically focus and focus by laser radar - Google Patents
Method for assisting camera to automatically focus and focus by laser radar Download PDFInfo
- Publication number
- CN113920197A CN113920197A CN202111207244.3A CN202111207244A CN113920197A CN 113920197 A CN113920197 A CN 113920197A CN 202111207244 A CN202111207244 A CN 202111207244A CN 113920197 A CN113920197 A CN 113920197A
- Authority
- CN
- China
- Prior art keywords
- camera
- image
- target
- laser radar
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 238000003384 imaging method Methods 0.000 claims abstract description 18
- 230000003287 optical effect Effects 0.000 claims abstract description 12
- 238000004364 calculation method Methods 0.000 claims abstract description 6
- 238000007689 inspection Methods 0.000 claims description 8
- GNFTZDOKVXKIBK-UHFFFAOYSA-N 3-(2-methoxyethoxy)benzohydrazide Chemical compound COCCOC1=CC=CC(C(=O)NN)=C1 GNFTZDOKVXKIBK-UHFFFAOYSA-N 0.000 claims description 6
- FGUUSXIOTUKUDN-IBGZPJMESA-N C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 Chemical compound C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 FGUUSXIOTUKUDN-IBGZPJMESA-N 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 3
- 238000012634 optical imaging Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 abstract description 3
- 230000007547 defect Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/40—Means for monitoring or calibrating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10141—Special mode during image acquisition
- G06T2207/10148—Varying focus
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Studio Devices (AREA)
- Optical Radar Systems And Details Thereof (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
A method for assisting automatic focusing of a camera by a laser radar comprises the following steps of firstly calibrating internal parameters of the camera, external parameters of the laser radar and the camera; then, acquiring an initial image, finding an initial area of a target in the image by using an image recognition method, and calculating a rotational angle of a holder so that an optical axis of a camera is aligned with the target; in the rotation process of the holder, according to the parameters jointly calibrated by the laser radar and the camera, finding a target area in the laser radar point cloud, determining the three-dimensional coordinates of the target, and calculating a focal length value and a focusing value according to an optical imaging model of the camera; and when the rotation of the holder is finished, the camera executes the calculated focal length value and the calculated focusing value, and an image is acquired. In the image obtained at the moment, the target is centered in the image, the imaging size is proper, and the imaging is clear. The calculation result is accurate, the manual participation is reduced, the intelligent and convenient effects are realized, and the practical value of image acquisition is very high.
Description
Technical Field
The invention relates to the field of image focusing, in particular to a method for assisting a camera to automatically focus by using a laser radar.
Background
When the intelligent inspection robot executes an inspection task, a camera carried by a rotatable holder is often used for collecting field images, and image recognition algorithms such as deep learning are used for monitoring tasks such as face recognition, vehicle recognition and fire recognition. The premise of accurate image recognition is that a clear image of the target is acquired, and the target has a proper position and size in the image. According to an imaging model of the camera, the focal length influences the imaging size; when the object distance, the image distance (focusing value) and the focal length satisfy the lens imaging formula, the focusing is clear. Therefore, the pan-tilt is required to rotate by a proper angle to aim at the target, and a proper focusing value and an image distance value of the camera are adjusted, so that a proper image can be acquired. The existing monitoring image acquisition scheme has the defects that the rotation angle, the focal length and the focusing value of a holder are continuously adjusted manually according to the image acquisition condition at the background, but the labor is too consumed, and the precision is poor. Some focal length adjustment schemes are that the inspection robot reaches a specified position every time, a fixed holder rotation angle, a fixed focal length and a fixed focusing value are adopted, but the inspection robot moves with a position error, and the mode can only collect images at the specified position and is not flexible enough. Some focusing methods use auto-focusing, estimate the sharpness of the image using an image sharpness evaluation function, and then adjust the focus value to maximize the sharpness of the image, but the end result of this approach is that the entire image appears sharpest and the target area may be blurred.
Disclosure of Invention
In order to overcome the defects of the technology, the invention provides the method for automatically focusing and focusing the laser radar auxiliary camera, which has the advantages that the target is centered in the image, the imaging size is proper and the imaging is clear.
The technical scheme adopted by the invention for overcoming the technical problems is as follows:
a method for automatic focusing of a laser radar-assisted camera is characterized by comprising the following steps:
a) installing a laser radar and a cloud deck with a camera on the outdoor inspection robot;
b) calibrating the camera to obtain an internal parameter matrix
Wherein
Which represents the focal length of the camera in pixels in the x-direction in a planar coordinate system,
denotes the focal length of the camera in pixels in the y-direction in a planar coordinate system, f is the focal length of the camera, dxFor the length of each imaging element in the CCD sensor of the camera, dyFor the width, u, of each imaging element in a camera CCD sensor0Is the pixel abscissa, v, of the optical center on the image0Is the pixel ordinate of the optical center on the image;
c) carrying out combined calibration on the laser radar and the camera to obtain external parameters between the laser radar and the camera;
d) acquiring an initial image, and finding an area where a target is located in the image by using an image recognition method of deep learning;
e) calculating the rotation angle of the holder, and aligning the optical axis of the camera with the target to center the area where the target is located in the image;
f) measuring the target distance by using a laser radar, and adjusting the focal length to enable the area where the target is located to reach a set size in the image;
g) adjusting the focus value to make the target clear in the image;
h) collecting images according to the focal length value obtained by calculation in the step f) and the focus value in the step g).
Further, the step b) finds out the camera to be calibrated by a Zhang Yongyou calibration method.
Further, step e) is based on the formula
Calculating to obtain the horizontal rotation angle of the holder
By the formula
Calculating to obtain the rotation angle of the holder in the vertical direction
Wherein u is the abscissa of the center point of the area where the target is located in the image, and v is the ordinate of the center point of the area where the target is located in the image.
Further, the step f) calculates the magnification of the target area by dividing the size of the area where the target is located by the size of the current target, namely the magnification of the focus change, wherein the size of the area where the target is located is 4/5 of the image size.
Further, step g) is performed by the formula
And calculating to obtain a focus value m, wherein n is the distance from the laser radar to the camera after the target is obtained.
The invention has the beneficial effects that: firstly, calibrating internal parameters of a camera, external parameters of a laser radar and the camera; then, acquiring an initial image, finding an initial area of a target in the image by using an image recognition method, and calculating a rotational angle of a holder so that an optical axis of a camera is aligned with the target; in the rotation process of the holder, according to the parameters jointly calibrated by the laser radar and the camera, finding a target area in the laser radar point cloud, determining the three-dimensional coordinates of the target, and calculating a focal length value and a focusing value according to an optical imaging model of the camera; and when the rotation of the holder is finished, the camera executes the calculated focal length value and the calculated focusing value, and an image is acquired. In the image obtained at the moment, the target is centered in the image, the imaging size is proper, and the imaging is clear. The calculation result is accurate, the manual participation is reduced, the intelligent and convenient effects are realized, and the practical value of image acquisition is very high.
Drawings
FIG. 1 is a flow chart of the method of the present invention
Fig. 2 is a structural diagram of the intelligent inspection robot of the present invention.
Detailed Description
The invention will be further explained with reference to fig. 1 and 2.
A method for automatic focusing of a laser radar-assisted camera is characterized by comprising the following steps:
a) and a laser radar and a cloud platform with a camera are arranged on the outdoor inspection robot.
b) Calibrating the camera to obtain an internal parameter matrix
Wherein
Which represents the focal length of the camera in pixels in the x-direction in a planar coordinate system,
denotes the focal length of the camera in pixels in the y-direction in a planar coordinate system, f is the focal length of the camera, dxFor the length of each imaging element in the CCD sensor of the camera, dyFor the width, u, of each imaging element in a camera CCD sensor0Is the pixel abscissa, v, of the optical center on the image0Is the pixel ordinate of the optical center on the image.
c) And carrying out combined calibration on the laser radar and the camera to obtain external parameters between the laser radar and the camera.
d) Acquiring an initial image, and finding a region where a target is located in the image by using an image recognition method of deep learning. The target area may not be centered in the image, too small in size, or may not be in focus.
e) And calculating the rotation angle of the holder, so that the optical axis of the camera is aligned to the target, and the area where the target is located is centered in the image.
f) And measuring the target distance by the laser radar, and adjusting the focal length to enable the area where the target is located to reach a set size in the image.
g) And adjusting the focus value to make the target clear in the image.
h) Collecting images according to the focal length value obtained by calculation in the step f) and the focus value in the step g). The obtained image target is centered, the size is moderate and the imaging is clear.
Firstly, calibrating internal parameters of a camera, external parameters of a laser radar and the camera; then, acquiring an initial image, finding an initial area of a target in the image by using an image recognition method, and calculating a rotational angle of a holder so that an optical axis of a camera is aligned with the target; in the rotation process of the holder, according to the parameters jointly calibrated by the laser radar and the camera, finding a target area in the laser radar point cloud, determining the three-dimensional coordinates of the target, and calculating a focal length value and a focusing value according to an optical imaging model of the camera; and when the rotation of the holder is finished, the camera executes the calculated focal length value and the calculated focusing value, and an image is acquired. In the image obtained at the moment, the target is centered in the image, the imaging size is proper, and the imaging is clear. The calculation result is accurate, the manual participation is reduced, the intelligent and convenient effects are realized, and the practical value of image acquisition is very high.
Further, the step b) finds out the camera to be calibrated by a Zhang Yongyou calibration method.
Further, step e) is based on the formula
Calculating to obtain the horizontal rotation angle of the holder
By the formula
Calculating to obtain the rotation angle of the holder in the vertical direction
Wherein u is the abscissa of the central point of the region where the target is located in the image, and v is the eyeAnd marking the ordinate of the central point of the area in the image.
Further, the step f) calculates the magnification of the target area by dividing the size of the area where the target is located by the size of the current target, namely the magnification of the focus change, wherein the size of the area where the target is located is 4/5 of the image size. And reading the current focal length value, and multiplying the current focal length value by the multiple of the focal length change to obtain the focal length value f to be set.
Further, step g) is performed by the formula
And calculating to obtain a focus value m, wherein n is the distance from the laser radar to the camera after the target is obtained.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (5)
1. A method for automatic focusing of a laser radar-assisted camera is characterized by comprising the following steps:
a) installing a laser radar and a cloud deck with a camera on the outdoor inspection robot;
b) calibrating the camera to obtain an internal parameter matrix
Wherein
Which represents the focal length of the camera in pixels in the x-direction in a planar coordinate system,
denotes the focal length of the camera in pixels in the y-direction in a planar coordinate system, f is the focal length of the camera, dxFor the length of each imaging element in the CCD sensor of the camera, dyFor the width, u, of each imaging element in a camera CCD sensor0Is the pixel abscissa, v, of the optical center on the image0Is the pixel ordinate of the optical center on the image;
c) carrying out combined calibration on the laser radar and the camera to obtain external parameters between the laser radar and the camera;
d) acquiring an initial image, and finding an area where a target is located in the image by using an image recognition method of deep learning;
e) calculating the rotation angle of the holder, and aligning the optical axis of the camera with the target to center the area where the target is located in the image;
f) measuring the target distance by using a laser radar, and adjusting the focal length to enable the area where the target is located to reach a set size in the image;
g) adjusting the focus value to make the target clear in the image;
h) collecting images according to the focal length value obtained by calculation in the step f) and the focus value in the step g).
2. The method of lidar assisted camera autofocus focusing of claim 1, wherein: and b) finding out the camera to calibrate the camera by a Zhang Zhengyou calibration method.
3. The method of lidar assisted camera autofocus focusing of claim 1, wherein: in step e) by formula
Calculating to obtain the horizontal rotation angle of the holder
By the formula
Calculating to obtain the rotation angle of the holder in the vertical direction
Wherein u is the abscissa of the center point of the area where the target is located in the image, and v is the ordinate of the center point of the area where the target is located in the image.
4. The method of lidar assisted camera autofocus focusing of claim 1, wherein: and f) calculating to obtain the expansion multiple of the target area by dividing the size of the area where the target is located by the size of the area where the current target is located, namely the multiple of the focusing change, wherein the size of the area where the target is located is 4/5 of the size of the image.
5. The method of lidar assisted camera autofocus focusing of claim 1, wherein: in step g) by the formula
And calculating to obtain a focus value m, wherein n is the distance from the laser radar to the camera after the target is obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111207244.3A CN113920197A (en) | 2021-10-18 | 2021-10-18 | Method for assisting camera to automatically focus and focus by laser radar |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111207244.3A CN113920197A (en) | 2021-10-18 | 2021-10-18 | Method for assisting camera to automatically focus and focus by laser radar |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113920197A true CN113920197A (en) | 2022-01-11 |
Family
ID=79240861
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111207244.3A Pending CN113920197A (en) | 2021-10-18 | 2021-10-18 | Method for assisting camera to automatically focus and focus by laser radar |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113920197A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118011421A (en) * | 2024-04-10 | 2024-05-10 | 中国科学院西安光学精密机械研究所 | Theodolite image automatic focusing method and system based on laser radar depth estimation |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109479088A (en) * | 2017-06-02 | 2019-03-15 | 深圳市大疆创新科技有限公司 | The system and method for carrying out multiple target tracking based on depth machine learning and laser radar and focusing automatically |
| CN112396664A (en) * | 2020-11-24 | 2021-02-23 | 华南理工大学 | Monocular camera and three-dimensional laser radar combined calibration and online optimization method |
| CN113205604A (en) * | 2021-05-17 | 2021-08-03 | 南昌智能新能源汽车研究院 | Feasible region detection method based on camera and laser radar |
| CN113406604A (en) * | 2021-06-30 | 2021-09-17 | 山东新一代信息产业技术研究院有限公司 | Device and method for calibrating positions of laser radar and camera |
-
2021
- 2021-10-18 CN CN202111207244.3A patent/CN113920197A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109479088A (en) * | 2017-06-02 | 2019-03-15 | 深圳市大疆创新科技有限公司 | The system and method for carrying out multiple target tracking based on depth machine learning and laser radar and focusing automatically |
| CN112396664A (en) * | 2020-11-24 | 2021-02-23 | 华南理工大学 | Monocular camera and three-dimensional laser radar combined calibration and online optimization method |
| CN113205604A (en) * | 2021-05-17 | 2021-08-03 | 南昌智能新能源汽车研究院 | Feasible region detection method based on camera and laser radar |
| CN113406604A (en) * | 2021-06-30 | 2021-09-17 | 山东新一代信息产业技术研究院有限公司 | Device and method for calibrating positions of laser radar and camera |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118011421A (en) * | 2024-04-10 | 2024-05-10 | 中国科学院西安光学精密机械研究所 | Theodolite image automatic focusing method and system based on laser radar depth estimation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3389268B1 (en) | Depth information acquisition method and apparatus, and image collection device | |
| CN109859272B (en) | Automatic focusing binocular camera calibration method and device | |
| US8098984B2 (en) | Focus detecting apparatus and an imaging apparatus | |
| CN105118055A (en) | Camera positioning correction calibration method and system | |
| CN112949478A (en) | Target detection method based on holder camera | |
| WO2022037633A1 (en) | Calibration method and apparatus for binocular camera, image correction method and apparatus for binocular camera, storage medium, terminal and intelligent device | |
| CN106482640A (en) | A kind of apparatus and method of integrated machine core optical axis correction | |
| CN112541953B (en) | Vehicle detection method based on radar signal and video synchronous coordinate mapping | |
| CN111288967A (en) | Remote high-precision displacement detection method based on machine vision | |
| CN110779491A (en) | Method, device and equipment for measuring distance of target on horizontal plane and storage medium | |
| CN110044262B (en) | Non-contact precision measuring instrument based on image super-resolution reconstruction and measuring method | |
| CN109191527B (en) | Alignment method and device based on minimum distance deviation | |
| CN105953741B (en) | System and method for measuring local geometric deformation of steel structure | |
| CN110736447B (en) | Vertical-direction horizontal position calibration method for integrated image acquisition equipment | |
| CN108020200A (en) | A kind of depth measurement method and system | |
| CN108805940B (en) | Method for tracking and positioning zoom camera in zooming process | |
| CN111595302A (en) | Double-sided array CCD auxiliary three-linear array CCD pose optical measurement and calibration method | |
| CN110595374B (en) | Large structural part real-time deformation monitoring method based on image transmission machine | |
| CN107784633B (en) | Unmanned aerial vehicle aerial image calibration method suitable for plane measurement | |
| CN113920197A (en) | Method for assisting camera to automatically focus and focus by laser radar | |
| US7999851B2 (en) | Optical alignment of cameras with extended depth of field | |
| CN116957943B (en) | Microscope stitching calibration method based on image fusion | |
| CN113421300B (en) | Method and device for determining actual position of object in fisheye camera image | |
| CN114111578B (en) | Automatic pose determining method for large-caliber element | |
| CN113805304B (en) | Automatic focusing system and method for linear array camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |