CN110599550A - Calibration system of RGB-D module and equipment and method thereof - Google Patents

Calibration system of RGB-D module and equipment and method thereof Download PDF

Info

Publication number
CN110599550A
CN110599550A CN201910848342.1A CN201910848342A CN110599550A CN 110599550 A CN110599550 A CN 110599550A CN 201910848342 A CN201910848342 A CN 201910848342A CN 110599550 A CN110599550 A CN 110599550A
Authority
CN
China
Prior art keywords
rgb
module
calibration
depth
parameters
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910848342.1A
Other languages
Chinese (zh)
Inventor
谢永明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hongkong Guang Yun Technology Co Ltd
Original Assignee
Hongkong Guang Yun Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hongkong Guang Yun Technology Co Ltd filed Critical Hongkong Guang Yun Technology Co Ltd
Priority to CN201910848342.1A priority Critical patent/CN110599550A/en
Publication of CN110599550A publication Critical patent/CN110599550A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

The invention provides a calibration method of an RGB-D module, which comprises the following steps: s1, shooting and collecting an RGB image and a Depth image by an RGB-D module to calibrate internal parameters and external parameters of the RGB-D module; s2, shooting and collecting an RGB-D image by the RGB-D module, and correcting parameters of the RGB-D module according to N different collection results; and S3, fusing the pixel level parameters of the RGB module and the Depth module to realize the spatial calibration of the RGB-D module. The calibration method of the RGB-D module provided by the invention utilizes the RGB-D module to shoot and collect RGB images and Depth images respectively so as to calibrate the internal parameters and the external parameters of the RGB-D module, and utilizes the RGB-D module to shoot and collect the RGB-D images so as to correct the parameters, so that the Depth module and the RGB module are aligned in a three-dimensional space, and the shooting effect of the RGB-D module is optimized. The invention also provides a calibration system of the RGB-D module and calibration equipment for realizing the calibration method of the RGB-D module.

Description

Calibration system of RGB-D module and equipment and method thereof
Technical Field
The present invention relates to the field of photography technologies, and in particular, to a multi-point focusing method and apparatus.
Background
With the development of hardware computing, the shooting function of mobile terminals (such as mobile phones, tablet computers, and the like) is also more and more abundant and powerful, and at present, most mobile terminals are equipped with cameras with higher performance, and especially, the appearance of multiple RGB-D modules such as double-shot, triple-shot, structured light + RGB and ToF + RGB on the mobile terminals provides possibility for users to improve shooting effect and experience and other extended applications.
The traditional camera calibration method is generally based on 2D planar target camera calibration (zhangzhengyou calibration method). The camera is used for shooting the same plane calibration plate at more than two different machine positions, the camera and the 2D plane calibration plate can move freely, and the internal parameters of the camera are guaranteed to be unchanged all the time, so that the optimal solution of the parameters of the camera is calculated through linear analysis, the calculation process is carried out, the lens distortion parameters are considered, and the internal and external parameters of the measured camera can be solved. However, for the calibration of the depth image, the traditional camera calibration method of the 2D planar target is difficult to achieve effective depth calibration, which also limits the application development of mobile terminals with RGB-D modules on the market, and further limits the rapid development of RGB-D cameras in the consumer-level market.
Disclosure of Invention
The invention aims to provide a space calibration of an RGB-D camera, which relates to the external parameter alignment, space alignment, alignment of three-dimensional space positions of a pixel level and the like of the RGB camera and a Depth camera so as to optimize the shooting effect of the RGB-D camera.
In order to achieve the above object, the present invention provides a calibration method for an RGB-D module, comprising the steps of: s1, shooting and collecting an RGB image and a Depth image by an RGB-D module to calibrate internal parameters and external parameters of the RGB-D module; s2, shooting and collecting an RGB-D image by the RGB-D module, and correcting parameters of the RGB-D module according to N different collection results; and S3, fusing the pixel level parameters of the RGB module and the Depth module to realize the spatial calibration of the RGB-D module.
Compared with the prior art, the calibration method of the RGB-D module provided by the invention has the advantages that the RGB-D module is used for shooting and collecting RGB images and Depth images respectively to calibrate the internal parameters and the external parameters of the RGB-D module, and the RGB-D module is used for shooting and collecting the RGB-D images to correct the parameters; the pixel level parameter fusion based on the RGB module and the Depth module realizes the alignment of the Depth module and the RGB module in a three-dimensional space, and ensures that Depth information acquired by the RGB-D module is accurately aligned with the RGB information, so that effective pixels of the Depth module are improved, and the shooting effect of the RGB-D module is optimized.
Preferably, the step S1 specifically includes: the RGB-D module acquires at least three frames of RGB images and at least three frames of Depth images to calculate and obtain internal parameters of the RGB-D moduleMatrix KRGBAnd KDepthAn extrinsic rotation matrix R and a translational vector T.
Specifically, the method for calculating the external reference rotation matrix R and the translation vector T includes: performing single-point traversal calculation on each point P in the FoV, wherein P is a coordinate of a certain point in a world space coordinate system, and rotating a matrix R under the coordinates of an RGB module and a Depth moduleRGBAnd RDepthAnd translation vector T of RGB module and Depth moduleRGBAnd TDepthAnd obtaining the coordinates of the P points under the RGB module and the Depth module respectively as follows: pRGB＝RRGB*P+TRGB，PDepth＝RDepth*P+TDepth(ii) a Wherein P isRGBIs the coordinate of point P in the RGB image, PDepthCoordinates in the P point Depth image; and (3) obtaining an external reference rotation matrix and a translation vector by coordinate calculation:T＝TRGB-RTDepth。
preferably, the calibration tool of the RGB-D module is used to obtain the internal reference matrix K of the RGB module and the Depth moduleRGBAnd KDepth(ii) a It can be understood that the Zhangyingyou calibration algorithm is a general algorithm for camera calibration calculation, and can directly call a calculation program code to calculate and obtain the internal reference matrix KRGBAnd KDepth。
Preferably, step S2 specifically includes: and shooting and collecting an RGB-D image by the RGB-D module, judging whether the RGB module is aligned with the Depth module or not according to N different collection results, if so, entering the step S3, and otherwise, adjusting the translation vector T.
Specifically, the translation vector micro-values are collected according to N different RGB-D images collected by the RGB-D module shootingSolving forFinally adjusting the translation vector to be T ═ T + Ttuning(ii) a In judging RGB-D image shot by RGB-D moduleAnd when the RGB pixel and the Depth pixel are not aligned, the RGB module and the Depth module are aligned by fine tuning the translation vector to be T.
In order to achieve the above object, the present invention provides a calibration system for RGB-D module, comprising: the main control module is in communication connection with the RGB-D module to control the RGB-D module to shoot and collect images; the calculation module acquires an image shot and collected by the RGB-D module, and calculates and corrects parameters of the RGB-D module; the parameter module is used for writing the parameters calculated by the calculation module into the RGB-D module; and the calibration module moves relative to the RGB-D module under the control of the main control module so as to enable the RGB-D module to acquire images.
According to the calibration system of the RGB-D module, the calibration module moves relative to the RGB-D module under the control of the main control module, so that the RGB-D module can shoot RGB images and Depth images of the calibration module at different angles and different positions; the calculation module acquires an RGB image and a Depth image acquired by the RGB-D module, calculates internal parameters and external parameters of the RGB-D module, calculates when the RGB module and the Depth module are not aligned and corrects the parameters; the parameter module is used for receiving the calculated parameters of the calculation module and writing the parameters into the RGB-D module. The calibration system of the RGB-D module can realize the calibration method of the RGB-D module.
In order to achieve the purpose, the invention provides calibration equipment of an RGB-D module, which comprises a clamp for positioning the RGB-D module, at least 4 calibration modules, a host and a display screen, wherein the calibration equipment comprises a base, a calibration module and a calibration module; the calibration equipment of the RGB-D module further comprises a driving mechanism for driving the clamp and at least 4 calibration modules to move relatively; the host acquires the positions of at least 4 calibration modules and shot images of the RGB-D modules so as to realize calibration of the RGB-D modules; the display screen is in communication connection with the host and displays calibration information.
Compared with the prior art, the calibration equipment for the RGB-D module comprises at least 4 detection calibration templates for calibrating the module, and the driving mechanism drives the at least 4 calibration modules to move relative to the clamp, so that the RGB-D module to be calibrated, which is clamped by the clamp, can acquire a plurality of depth data, and the calibration precision of the calibration equipment for the RGB-D module is improved.
Preferably, at least three of the calibration modules can rotate within a certain angle range along the XY direction.
Preferably, the driving mechanism comprises a first power mechanism for driving the clamp to move and a second power mechanism for driving at least 4 calibration modules to move.
Preferably, the calibration equipment for the RGB-D module further comprises a horizontal control frame and/or a vertical control frame for taking and placing the RGB-D module.
Preferably, the RGB-D module is a mobile communication terminal having a shooting function.
Drawings
FIG. 1 is a flowchart illustrating an RGB-D module calibration method according to the present invention.
FIG. 2 is a schematic diagram of a calibration system of an RGB-D module according to the present invention.
FIG. 3 is a schematic diagram of an RGB-D module calibration apparatus according to the present invention.
FIG. 4 is a schematic diagram of an internal structure of the RGB-D module calibration apparatus according to the present invention.
FIG. 5 is a side view of the internal structure of the RGB-D module calibration apparatus of the present invention.
FIG. 6 is a schematic diagram of a calibration process of the RGB-D module calibration apparatus according to the present invention.
Detailed Description
In order to explain technical contents, structural features, and objects and effects of the present invention in detail, the following detailed description is given with reference to the accompanying drawings in conjunction with the embodiments.
As shown in fig. 1, the calibration method of the RGB-D module provided by the present invention comprises the steps of: s1, shooting and collecting an RGB image and a Depth image by an RGB-D module to calibrate internal parameters and external parameters of the RGB-D module; s2, shooting and collecting an RGB-D image by the RGB-D module, and correcting parameters of the RGB-D module according to N different collection results; and S3, fusing the pixel level parameters of the RGB module and the Depth module to realize the spatial calibration of the RGB-D module. More specifically:
step S1: the RGB-D module shoots and collects RGB images and Depth images to calibrate internal parameters and external parameters of the RGB-D module. Specifically, the RGB-D module respectively acquires at least three frames of RGB images and at least three frames of Depth images, and calculates to obtain an internal reference matrix K of the RGB-D moduleRGBAnd KDepthAn extrinsic rotation matrix R and a translational vector T.
It can be understood that the RGB-D module includes an RGB module and a Depth module, wherein the RGB image taken by the RGB module is a 2D image not containing Depth information, and the Depth image taken by the Depth module contains Depth information. And the RGB-D image with the Depth information is obtained by calculating through fusing the RGB image and the Depth image respectively shot by the RGB module and the Depth module.
In step S1, the RGB module captures an RGB image without Depth information, the Depth module captures a Depth image with Depth information, and the internal parameters and the external parameters of the RGB-D module are acquired and calibrated based on at least three RGB images and at least three Depth images.
Wherein, the internal parameters of the RGB-D module specifically comprise an internal parameter matrix K of the RGB moduleRGBAnd a parameter matrix K of the Depth moduleDepth. The internal reference matrix of the camera module can be obtained by calculation through a Zhangyingyou calibration algorithm, specifically, program codes for calculating Zhangyingyou parameters can be written, and if the calculating program codes for the Zhangyingyou calibration algorithm are arranged in the camera module, the calculating program codes can also be directly called for calculation. Since it is common knowledge in the art to utilize the intrinsic parameters of the image capturing camera module based on the Zhang friend calibration algorithm, no additional description is provided here.
For the RGB-D model, the extrinsic parameters include an extrinsic rotation matrix R and a translational vector T, which may be based on the RGB model extrinsic rotation matrix RRGBAnd translation vector TRGBExternal parameter rotation matrix R of Depth moduleDepthAnd translation vector TDepthAnd (6) performing calculation. The calculation method of the external parameters of the RGB-D module specifically comprises the following steps: performing single-point traversal calculation on each point P in the FoV, wherein P is a coordinate of a certain point in a world space coordinate system, and rotating a matrix R under the coordinates of an RGB module and a Depth moduleRGBAnd RDepthAnd anTranslation vector T of RGB module and Depth moduleRGBAnd TDepthAnd obtaining the coordinates of the P points under the RGB module and the Depth module respectively as follows:
PRGB＝RRGB*P+TRGB，PDepth＝RDepth*P+TDepth；
wherein P isRGBIs the coordinate of point P in the RGB image, PDepthCoordinates in the P point Depth image; and (3) obtaining an external reference rotation matrix and a translation vector by coordinate calculation:
it can be appreciated that the parameters of the camera are currently usually calibrated with 2D planar targets. In the internal parameters and the external parameters of the RGB-D module calibrated based on the calibration method, the Depth related parameters of the Depth module cannot be accurately calculated, so that the imaging effect is influenced because the RGB information and the Depth information in the RGB-D image shot by the RGB-D module are not aligned. Therefore, after step S1, the Depth-related parameters of the Depth module need to be corrected to improve the imaging effect.
Step S2: the RGB-D module shoots and collects RGB-D images and corrects parameters of the RGB-D module according to N different collection results. Specifically, the RGB-D module captures an RGB-D image, and determines whether the RGB module and the Depth module are aligned according to N different capture results, if so, step S3 is performed, and if not, the translation vector T is adjusted.
The following are exemplary: the RGB module and the Depth module shoot N different actual scenes, and based on the internal parameters and the external parameters calibrated in the step S1, the images are fused to obtain N RGB-D images with Depth information; judging the alignment condition of RGB information and Depth information in the N RGB-D images, and further judging whether the RGB module and the Depth module are aligned; if so, the process proceeds to step S3, otherwise, the translation vector T is fine-tuned until the RGB module and the Depth module are aligned.
The specific steps for adjusting the translation vector T are: shooting N different RGB-D images according to the RGB-D module, and collecting translation vector micro-valuesSolving forFinally adjusting the translation vector to be T ═ T + Ttuning(ii) a And when the RGB pixels and Depth pixels in the RGB-D image shot by the RGB-D module are not aligned, the RGB module and the Depth module are aligned by finely adjusting the translation vector to be T.
In step S2, a plurality of actual scenes are captured by the primarily calibrated RGB-D camera, and an RGB-D image with depth information is obtained by fusing the images based on the primarily calibrated internal and external parameters. Whether the RGB module and the Depth module are aligned or not is judged through the actual shooting effect of the RGB-D module, if not, the leveling vector T is adjusted until the RGB module and the Depth module realize pixel-level alignment. It can be understood that, the judgment of whether the RGB information and the Depth information in the RGB-D image are aligned may be made by human eyes, or may be made by comparing the calibration program with the standard image.
And S3, fusing the pixel level parameters of the RGB module and the Depth module to realize the spatial calibration of the RGB-D module. Specifically, the internal parameters and the external parameters preliminarily calibrated in step S1 and corrected in step S2 are written into the RGB-D module, so as to complete the spatial calibration of the RGB-D module. And the corrected RGB-D module can directly use the written internal parameters and external parameters to fuse the RGB image shot by the RGB module and the Depth image shot by the Depth module during shooting, and displays the fused RGB-D image with Depth information for a user.
Compared with the prior art, in the calibration method of the RGB-D module, after the RGB module and the Depth module respectively collect RGB images and Depth images to preliminarily calibrate the internal parameters and the external parameters of the RGB-D module, a plurality of actual scenes are shot, the images are fused based on the preliminarily calibrated internal parameters and the preliminarily calibrated external parameters to obtain the RGB-D images with Depth information, whether the RGB module and the Depth module are aligned or not is judged, if not, the parameters of the RGB-D module are corrected, so that the image fusion of the RGB module and the Depth module at a pixel level is realized, and the imaging effect of the RGB-D module is optimized.
As shown in fig. 2, the present invention provides a calibration system for an RGB-D module to implement the calibration method for the RGB-D module. Specifically, the calibration system for the RGB-D module provided by the present invention comprises: the main control module is in communication connection with the RGB-D module to control the RGB-D module to shoot and collect images; the calculation module acquires an image shot and collected by the RGB-D module, and calculates and corrects parameters of the RGB-D module; the parameter module is used for writing the parameters calculated by the calculation module into the RGB-D module; and the calibration module moves relative to the RGB-D module under the control of the main control module so as to enable the RGB-D module to acquire images.
According to the calibration system of the RGB-D module, the calibration module moves relative to the RGB-D module under the control of the main control module, so that the RGB-D module can shoot RGB images and Depth images of the calibration module at different angles and different positions; the calculation module acquires an RGB image and a Depth image acquired by the RGB-D module, calculates internal parameters and external parameters of the RGB-D module, calculates when the RGB module and the Depth module are not aligned and corrects the parameters; the parameter module is used for receiving the calculated parameters of the calculation module and writing the parameters into the RGB-D module. The calibration system of the RGB-D module can realize the calibration method of the RGB-D module.
In order to realize the calibration method of the RGB-D module, the invention also provides calibration equipment of the RGB-D module. As shown in fig. 3-5, the calibration apparatus for RGB-D module provided in the present invention comprises a fixture 100 for positioning the RGB-D module, at least 4 calibration modules 200, a host and a display end; the calibration equipment of the RGB-D module further comprises a driving mechanism for driving the clamp 100 and at least 4 calibration modules 200 to move relatively; the host computer obtains the positions of at least 4 calibration modules 200 and the shot images of the RGB-D modules so as to realize the calibration of the RGB-D modules; the display end is in communication connection with the host and displays the calibration information. More specifically:
the calibration apparatus for RGB-D module shown in fig. 3-5 has no display host and no display side. It can be understood that the host computer is used for completing calibration of the RGB-D module for the control fixture 100, the calibration module 200, and the like according to a preset process, and may be a PC or other control device; the display end is used for displaying the calibration process or the calibration result, and may be a display screen, or a simple indicator light capable of indicating the calibration state or the calibration result.
As shown in fig. 3 and 4, the calibration apparatus of the RGB-D module includes a bottom chassis 300 and a movable frame 400 that is movable up and down with respect to the bottom chassis 300; the fixture 100 is disposed on the upper side of the bottom frame 300, and at least 4 calibration modules 200 are disposed on the lower side of the movable frame 400 and driven by the movable frame 400 to move up and down relative to the fixture 100.
In this embodiment, as shown in fig. 3, the calibration apparatus for RGB-D module further includes a housing 500, wherein the bottom chassis 300 and the movable chassis 400 are disposed in the housing 500; the side of the housing 500 is provided with a pick-and-place opening 510 for picking and placing the RGB-D module. The arrangement of the housing 500 can reduce the external interference and improve the calibration accuracy in the calibration process of the RGB-D module.
In the calibration method for the RGB-D module, the RGB-D module is required to capture images of the calibration module 200 at different angular positions during the calibration process. For this purpose, the calibration apparatus for RGB-D module provided in the present invention further includes a driving mechanism for driving the fixture 100 and the calibration module 200 to move. The driving mechanism specifically includes a first power mechanism 110 for driving the clamp 100 to move, and a second power mechanism 210 for driving at least 4 calibration modules 200 to move.
As shown in fig. 3 and 4, in the present embodiment, the first power mechanism 110 includes two clamping cylinders for driving the clamp 100 to move. The two clamping cylinders act to drive the clamp 100 to clamp the RGB-D module to be tested, so as to realize the positioning of the RGB-D module.
Further, in the calibration apparatus for RGB-D modules provided in the present invention, at least three calibration modules 200 can rotate within a certain angle range along the XY direction. Specifically, as shown in fig. 3 and 4, the second power mechanism 210 for driving at least 4 calibration modules 200 to move includes a vertical driving unit 211 for driving the movable frame 400 to move up and down relative to the bottom frame 300, and rotation driving units 212 for driving a corresponding calibration module 200 to rotate in the plane direction, respectively. The vertical driving unit 211 drives the movable frame 400 to move up and down relative to the base frame 300 and drives at least 4 calibration modules 200 to move up and down simultaneously, and the vertical driving unit 211 can be a linear driving cylinder or a motor; the number of the rotation driving units 212 is at least 3, and each rotation driving unit 212 drives a corresponding calibration module 200 to rotate in the plane direction by taking the vertical shaft as the center.
Preferably, the calibration apparatus for RGB-D module further comprises a pick-and-place frame (not shown) for picking and placing the RGB-D module, wherein the pick-and-place frame moves between the pick-and-place opening 510 and the fixture 100 to pick up the RGB-D module from the pick-and-place opening 510 and move the RGB-D module to the fixture 100, or pick up the RGB-D module from the fixture 100 and move the RGB-D module to the pick-and-place opening 510. After the pick-and-place frame is arranged, the RGB-D module to be calibrated can be placed at the pick-and-place frame or the calibrated RGB-D module can be taken away from the pick-and-place frame when the pick-and-place frame moves to the pick-and-place opening 510.
Compared with the prior art, the calibration equipment for the RGB-D module comprises at least 4 calibration modules 200, and the driving mechanism drives the at least 4 calibration modules 200 to move relative to the clamp 100, so that the RGB-D module to be calibrated, which is clamped by the clamp 100, can acquire an image meeting the calibration requirement, the calibration method for the RGB-D module is realized, the calibration precision of the RGB-D module is further improved, and the shooting effect of the RGB-D camera is further optimized.
With reference to fig. 1 to 6, a detailed description will be given of a flow of the calibration method for implementing the RGB-D module based on the calibration apparatus for RGB-D module provided by the present invention:
as shown in fig. 5, the calibration process specifically includes the steps of: a1, placing the RGB-D module into the fixture 100 for positioning, and connecting the RGB-D module with the host in a communication way; a2, the host computer controls the clamp 100 to move so as to drive the RGB-D module to move to the detection station; a3, the host computer controls at least 4 calibration modules 200 to move relative to the RGB-D module, and the RGB-D module shoots to complete the parameter correction of the RGB-D module; a4, writing the calibration result into the RGB-D module; a5, the host computer controls a first power mechanism to drive the clamp 100 to drive the RGB-D module to exit the detection station; a6, disconnecting the communication connection between the RGB-D module and the host and taking away the RGB-D module.
Wherein, the step a3 specifically includes: a31, the host controls the RGB-D module to shoot RGB images and Depth images and transmits the RGB images and Depth images back to the host to calibrate the internal parameters and external parameters of the RGB-D module; a32, the host controls the RGB-D module to shoot RGB-D image and transmits the RGB-D image back to the host, judges whether the RGB module and Depth module are aligned, if not, the RGB-D module is corrected; a33, the host controls the RGB-D module to shoot the RGB-D image again and transmits the RGB-D image back to the host until the RGB module is aligned with the Depth module, and the final internal parameters and external parameters of the RGB-D module are obtained.
For step a 1: placing the RGB-D module into the fixture 100 for positioning, and connecting the RGB-D module with a host in a communication way; specifically, the RGB-D module is placed in the fixture 100 for positioning, and the RGB-D module is communicatively connected to the host through a bluetooth or communication connection line. In actual work, step a1 is mainly implemented based on the operation of an operator.
After the operation of step a1 is completed, and the RGB-D module is placed in the fixture 100 and is communicatively connected to the host, the steps a 2-a 5 are mainly automatically implemented under the control of the host, and the specific calculation process is described in the above-mentioned contents of the calibration method for the RGB-D module. Only in step a32, the "determination of whether the RGB module and the Depth module are aligned" may be performed by an operator visually according to the RGB-D image, or by comparing a calibration program loaded in the host with a standard image without manual work, to determine the imaging effect of the RGB-D image and whether the parameters of the RGB-D module need to be calibrated.
It can be understood that the RGB-D module may be a camera module integrated in the mobile communication terminal and having a depth information capturing function. During calibration, the camera module may be calibrated as needed, or the camera module integrated in the mobile communication terminal may be calibrated after the mobile communication terminal is completed.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the scope of the present invention, therefore, the present invention is not limited by the appended claims.

Claims (10)

1. A calibration method for RGB-D module is characterized by comprising the following steps:
s1, shooting and collecting an RGB image and a Depth image by an RGB-D module to calibrate internal parameters and external parameters of the RGB-D module;
s2, shooting and collecting an RGB-D image by the RGB-D module, and correcting parameters of the RGB-D module according to N different collection results;
and S3, fusing the pixel level parameters of the RGB module and the Depth module to realize the spatial calibration of the RGB-D module.
2. The calibration method for the RGB-D module as claimed in claim 1, wherein the step S1 is specifically: the RGB-D module acquires at least three frames of RGB images and at least three frames of Depth images to calculate and obtain an internal parameter matrix K of the RGB-D moduleRGBAnd KDepthAn extrinsic rotation matrix R and a translational vector T.
3. The calibration method for RGB-D module as claimed in claim 2, wherein the extrinsic rotation matrix R and the translation vector T are calculated by: performing single-point traversal calculation on each point P in the FoV, wherein P is a coordinate of a certain point in a world space coordinate system, and rotating a matrix R under the coordinates of an RGB module and a Depth moduleRGBAnd RDepthAnd translation vector T of RGB module and Depth moduleRGBAnd TDepthAnd obtaining the coordinates of the P points under the RGB module and the Depth module respectively as follows:
PRGB＝RRGB*P+TRGB，PDepth＝RDepth*P+TDepth；
wherein P isRGBIs the coordinate of point P in the RGB image, PDepthCoordinates in the P point Depth image;
and (3) obtaining an external reference rotation matrix and a translation vector by coordinate calculation:
T＝TRGB-RTDepth。
4. the method as claimed in claim 2, wherein the calibration tool of the RGB-D module is used to obtain the internal reference matrix K of the RGB module and the Depth moduleRGBAnd KDepth。
5. The calibration method for the RGB-D module as claimed in claim 1, wherein the step S2 is specifically: and shooting and collecting an RGB-D image by the RGB-D module, judging whether the RGB module is aligned with the Depth module or not according to N different collection results, if so, entering the step S3, and otherwise, adjusting the translation vector T.
6. The calibration method for RGB-D module set according to claim 5, wherein the translation vector micro-values are collected according to N different RGB-D images collected by the RGB-D module setSolving forFinally adjusting the translation vector to be T ═ T + Ttuning。
7. A calibration system for an RGB-D module, comprising:
the main control module is in communication connection with the RGB-D module to control the RGB-D module to shoot and collect images;
the calculation module acquires an image shot and collected by the RGB-D module, and calculates and corrects parameters of the RGB-D module;
the parameter module is used for writing the parameters calculated by the calculation module into the RGB-D module;
and the calibration module moves relative to the RGB-D module under the control of the main control module so as to enable the RGB-D module to acquire images.
8. A calibration apparatus for RGB-D module, for implementing the calibration method of RGB-D module as claimed in any one of claims 1-6, wherein the calibration apparatus for RGB-D module comprises a fixture for positioning RGB-D module, at least 4 calibration modules, a host computer and a display screen; the calibration equipment of the RGB-D module further comprises a driving mechanism for driving the clamp and at least 4 calibration modules to move relatively; the host acquires the positions of at least 4 calibration modules and shot images of the RGB-D modules so as to realize calibration of the RGB-D modules; the display screen is in communication connection with the host and displays calibration information.
9. The RGB-D module calibration apparatus according to claim 8, wherein the driving mechanism comprises a first power mechanism for driving the fixture to move, and a second power mechanism for driving at least 4 of the calibration modules to move.
10. The RGB-D module calibration apparatus of claim 8, wherein the calibration apparatus for RGB-D module further comprises a horizontal control rack and/or a vertical control rack for taking and placing the RGB-D module.
CN201910848342.1A 2019-09-09 2019-09-09 Calibration system of RGB-D module and equipment and method thereof Pending CN110599550A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910848342.1A CN110599550A (en) 2019-09-09 2019-09-09 Calibration system of RGB-D module and equipment and method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910848342.1A CN110599550A (en) 2019-09-09 2019-09-09 Calibration system of RGB-D module and equipment and method thereof

Publications (1)

Publication Number Publication Date
CN110599550A true CN110599550A (en) 2019-12-20

Family

ID=68858226

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910848342.1A Pending CN110599550A (en) 2019-09-09 2019-09-09 Calibration system of RGB-D module and equipment and method thereof

Country Status (1)

Country Link
CN (1) CN110599550A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113658134A (en) * 2021-08-13 2021-11-16 安徽大学 Multi-mode alignment calibration RGB-D image salient target detection method

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150023563A1 (en) * 2013-07-16 2015-01-22 Texas Instruments Incorporated Depth Sensor Data with Real-Time Processing of Scene Sensor Data
CN104463880A (en) * 2014-12-12 2015-03-25 中国科学院自动化研究所 RGB-D image acquisition method
KR20170025214A (en) * 2015-08-27 2017-03-08 광주과학기술원 Method for Multi-view Depth Map Generation
CN108921903A (en) * 2018-06-28 2018-11-30 Oppo广东移动通信有限公司 Camera calibration method, device, computer readable storage medium and electronic equipment
CN109040745A (en) * 2018-08-01 2018-12-18 Oppo广东移动通信有限公司 Camera method for self-calibrating and device, electronic equipment, computer storage medium
CN109741405A (en) * 2019-01-21 2019-05-10 同济大学 A kind of depth information acquisition system based on dual structure light RGB-D camera
CN109816731A (en) * 2017-11-21 2019-05-28 西安交通大学 A kind of method of RGB and depth information accuracy registration
CN109978929A (en) * 2017-12-28 2019-07-05 舜宇光学（浙江）研究院有限公司 The RGB-D image synthesis optimizing system and method for depth information camera module
CN110070582A (en) * 2018-01-23 2019-07-30 舜宇光学（浙江）研究院有限公司 Take the photograph mould group parameter self-calibration system and calibration method and its electronic equipment more
CN211403487U (en) * 2019-09-09 2020-09-01 香港光云科技有限公司 Calibration equipment for RGB-D module

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150023563A1 (en) * 2013-07-16 2015-01-22 Texas Instruments Incorporated Depth Sensor Data with Real-Time Processing of Scene Sensor Data
CN104463880A (en) * 2014-12-12 2015-03-25 中国科学院自动化研究所 RGB-D image acquisition method
KR20170025214A (en) * 2015-08-27 2017-03-08 광주과학기술원 Method for Multi-view Depth Map Generation
CN109816731A (en) * 2017-11-21 2019-05-28 西安交通大学 A kind of method of RGB and depth information accuracy registration
CN109978929A (en) * 2017-12-28 2019-07-05 舜宇光学（浙江）研究院有限公司 The RGB-D image synthesis optimizing system and method for depth information camera module
CN110070582A (en) * 2018-01-23 2019-07-30 舜宇光学（浙江）研究院有限公司 Take the photograph mould group parameter self-calibration system and calibration method and its electronic equipment more
CN108921903A (en) * 2018-06-28 2018-11-30 Oppo广东移动通信有限公司 Camera calibration method, device, computer readable storage medium and electronic equipment
CN109040745A (en) * 2018-08-01 2018-12-18 Oppo广东移动通信有限公司 Camera method for self-calibrating and device, electronic equipment, computer storage medium
CN109741405A (en) * 2019-01-21 2019-05-10 同济大学 A kind of depth information acquisition system based on dual structure light RGB-D camera
CN211403487U (en) * 2019-09-09 2020-09-01 香港光云科技有限公司 Calibration equipment for RGB-D module

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113658134A (en) * 2021-08-13 2021-11-16 安徽大学 Multi-mode alignment calibration RGB-D image salient target detection method

Similar Documents

Publication Publication Date Title
TWI547162B (en) Camera system and method for aligning components with footprint on printed circuit board
US8310663B2 (en) Methods and systems for calibrating an adjustable lens
CN108830906B (en) Automatic calibration method for camera parameters based on virtual binocular vision principle
US9438889B2 (en) System and method for improving methods of manufacturing stereoscopic image sensors
CN100590577C (en) Touch screen positioning device and method thereof
EP3316577A1 (en) Autofocus testing device
JP2023509137A (en) Systems and methods for capturing and generating panoramic 3D images
CN111080705B (en) Calibration method and device for automatic focusing binocular camera
CN112543321B (en) Position compensation detection and correction method, camera module and manufacturing method thereof
CN112785952B (en) Detection and correction device and detection and correction method for display screen with under-screen camera
TWI616715B (en) Method and system for multi-lens module alignment
CN111627073B (en) Calibration method, calibration device and storage medium based on man-machine interaction
KR100690172B1 (en) method for extracting 3-dimensional coordinate information from 3-dimensional image using mobile phone with multiple cameras and terminal thereof
CN114577135B (en) 3D detection method and system for chip pin warpage based on single lens
CN211403487U (en) Calibration equipment for RGB-D module
CN110136203B (en) Calibration method and calibration system of TOF equipment
CN110599550A (en) Calibration system of RGB-D module and equipment and method thereof
CN112598751A (en) Calibration method and device, terminal and storage medium
WO2021007944A1 (en) Method for calibrating bracket of 3d structured light module, apparatus, and device
CN116051652A (en) Parameter calibration method, electronic equipment and storage medium
KR101456650B1 (en) Method of alignig optical axis for stereo camera and the camera
TWM561212U (en) Calibration equipment
CN210664375U (en) Antiaircraft gun detection device
CN109584313B (en) Camera calibration method and device, computer equipment and storage medium
KR20120070129A (en) Apparatus and method for photographing stereoscopic image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination