CN113658270B - Method, device, medium and system for multi-vision calibration based on workpiece hole center - Google Patents
Method, device, medium and system for multi-vision calibration based on workpiece hole center Download PDFInfo
- Publication number
- CN113658270B CN113658270B CN202110912869.3A CN202110912869A CN113658270B CN 113658270 B CN113658270 B CN 113658270B CN 202110912869 A CN202110912869 A CN 202110912869A CN 113658270 B CN113658270 B CN 113658270B
- Authority
- CN
- China
- Prior art keywords
- camera
- workpiece
- calibration
- image data
- coordinate system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
Abstract
The invention discloses a method, a device, a medium and a system for multi-vision calibration based on a workpiece hole center, which comprises the following steps: acquiring image data of a calibration plate under the view angles of a plurality of cameras; acquiring initial parameters of a camera according to image data of a calibration plate, wherein the initial parameters of the camera comprise internal parameters of the camera, external parameters of the camera and distortion coefficients of the camera; acquiring sample image data of a calibration workpiece under the view angles of a plurality of cameras; converting a world coordinate system in which initial parameters of a camera are positioned into a workpiece coordinate system based on sample image data of a calibration workpiece, and acquiring camera external parameters after converting the coordinate system; and optimizing initial parameters of the camera based on the calibrated workpiece hole center. The camera initial parameter generalization obtained by the method based on the multi-vision calibration of the workpiece hole center is high, the precision of measuring the workpiece round hole is high, the measurement result can be directly expressed on a workpiece coordinate system, and the measurement is convenient.
Description
Technical Field
The invention relates to the technical field of multi-vision calibration, in particular to a method, a device, a medium and a system for multi-vision calibration based on a workpiece hole center.
Background
In the inspection and measurement industry, three coordinate measuring machines are commonly used for measurement based on industry high precision measurement requirements. However, for a workpiece with multiple characteristic points to be measured, when the three-coordinate measuring machine is used for measuring, each to-be-measured point of the workpiece needs to be measured by controlling the three-coordinate measuring machine to move, the time cost is huge, and the requirement of rapid measurement cannot be met. Therefore, the multi-vision measuring technology can be adopted to realize the rapid measurement of the workpiece with multiple characteristic points.
In the prior art, the industrial measurement technology based on multi-vision usually uses a calibration plate to complete multi-camera calibration, but the obtained camera initial parameters cannot reach very high measurement precision when being directly used for industrial measurement, so that the generalization of the measurement on a workpiece is not high, and the world coordinate system of the calibrated camera is usually a plane coordinate system of the calibration plate or a camera coordinate system of a certain camera, so that the measurement is inconvenient.
Disclosure of Invention
The present invention aims to solve at least one of the technical problems existing in the prior art. Therefore, one of the purposes of the invention is to provide a multi-vision calibration method based on the hole center of a workpiece, which can improve the measurement precision of the calibrated workpiece on the basis of rapid measurement, and can directly express the measurement result on a workpiece coordinate system, thereby facilitating measurement.
The second purpose of the invention is to provide a device for multi-vision calibration based on the hole center of the workpiece.
It is a further object of the invention to provide a non-transitory computer readable storage medium.
The fourth object of the invention is to provide a system for multi-vision calibration based on the hole center of a workpiece.
In order to achieve the above object, a method for multi-vision calibration based on a hole center of a workpiece according to an embodiment of the first aspect of the present invention includes: acquiring image data of a calibration plate under the view angles of a plurality of cameras; acquiring initial parameters of a camera according to the image data of the calibration plate, wherein the initial parameters of the camera comprise internal parameters of the camera, external parameters of the camera and distortion coefficients of the camera; acquiring sample image data of a calibration workpiece under the view angles of the cameras; converting a world coordinate system in which the initial parameters of the camera are positioned into a workpiece coordinate system based on the sample image data of the calibrated workpiece, and obtaining camera external parameters after the coordinate system conversion; and optimizing the initial parameters of the camera based on the calibrated workpiece hole center.
According to the multi-vision calibration method based on the workpiece hole center, based on the camera initial parameters and the sample image data of the calibrated workpiece, the world coordinate system where the camera is located is converted from a certain camera coordinate system to a workpiece coordinate system, namely, the measurement result can be directly represented on the workpiece coordinate system, and the measurement is convenient. And the initial parameters of the camera are optimized based on the hole center of the calibration workpiece, so that the detection precision of the camera is improved, and the generalization of the measurement of the calibration workpiece is improved.
In some embodiments of the present invention, acquiring image data of a calibration plate at a multi-vision camera viewing angle includes: the calibration plates are square calibration plates, and each calibration plate is provided with a plurality of markers; and fixing the positions of the plurality of cameras and moving the calibration plate, and controlling the plurality of cameras to take pictures so as to acquire images of the plurality of calibration plates as image data of the calibration plate.
In some embodiments of the present invention, at least three images of the calibration plate are included in any one of the plurality of cameras at the viewing angle, and a plurality of the markers are included in any one of the images of the calibration plate.
In some embodiments of the present invention, the acquiring sample image data of the calibration workpiece at the plurality of camera perspectives includes: acquiring actual image data of the calibration workpiece; marking the part hole in the actual image data by using an ROI (region of interest ) to obtain marked image data; performing de-distortion processing on the marked image data according to the camera internal parameters and the camera distortion coefficients to obtain processed image data; and fitting an ellipse in the processed image data according to an ellipse detection algorithm to obtain an ellipse center serving as a 2D observation point of the calibration workpiece hole center.
In some embodiments of the present invention, the converting the world coordinate system in which the camera initial parameters are located into the workpiece coordinate system includes: reconstructing 2D observation points of the hole center of the calibration workpiece based on the camera internal reference and the ellipse detection to obtain reconstruction point cloud of the calibration workpiece; and determining the rigid transformation relation of the reconstruction point cloud and the three-coordinate measured point cloud of the calibration workpiece, and applying the rigid transformation relation to the camera external parameters.
In some embodiments of the invention, the optimizing the camera initial parameters based on the calibrated workpiece aperture core comprises: projecting the clear value of the calibration workpiece to an image plane according to the camera internal parameter and the camera external parameter after the coordinate system conversion to obtain a template picture; and matching the marked image data with the template picture to obtain the optimized 2D observation point for calibrating the hole center of the workpiece.
In some embodiments of the present invention, the optimizing the camera initial parameters based on the calibrated workpiece aperture core further comprises: acquiring a three-dimensional measured value of the calibration workpiece as a 3D point; acquiring the camera internal parameters and the camera external parameters after the coordinate system conversion as initial values of the camera; fixing the 3D point according to the optimized 2D observation point of the calibrated workpiece hole center and the initial value of the camera; and optimizing the initial parameters of the camera according to the adjustment of the beam method.
In order to achieve the above object, an apparatus for multi-vision calibration based on a workpiece hole center according to a second aspect of the present invention includes: the image acquisition module is used for acquiring image data of the calibration plates under the view angles of the cameras and acquiring sample image data of the calibration workpieces under the view angles of the cameras; the data generation module is used for acquiring initial parameters of the camera according to the image data of the calibration plate, wherein the initial parameters of the camera comprise internal parameters of the camera, external parameters of the camera and distortion coefficients of the camera; the coordinate system conversion module is used for converting the world coordinate system where the initial parameters of the camera are positioned into a workpiece coordinate system based on the sample image data of the calibration workpiece, and obtaining camera external parameters after the coordinate system conversion; and the parameter optimization module is used for optimizing the initial parameters of the camera based on the calibrated workpiece hole center.
According to the device for multi-vision calibration based on the workpiece hole center, disclosed by the embodiment of the invention, the framework of the image acquisition module, the data generation module, the coordinate system conversion module and the parameter optimization module is adopted, the coordinate system conversion module is used for converting the world coordinate system where the camera is positioned from a certain camera coordinate system into a workpiece coordinate system based on the obtained camera initial parameters and the sample image data of the calibrated workpiece, namely, the measurement result can be directly represented on the workpiece coordinate system, and the measurement is convenient. And the parameter optimization module optimizes the initial parameters of the camera based on the hole center of the calibration workpiece, so that the detection precision of the camera is improved, and the generalization of the measurement of the calibration workpiece is improved.
To achieve the above object, a non-transitory computer readable storage medium according to an embodiment of the third aspect of the present invention has a computer program stored thereon, where the computer program, when executed by a processor, may implement the method for multi-vision calibration based on hole center of a workpiece according to any one of the above embodiments.
According to the non-transitory computer readable storage medium, the computer program is stored on the storage medium, when the computer program runs, the operation parameters of each structure in the device can be obtained for analysis and calculation, in the process of detecting a calibration workpiece, the processor can control the camera to carry out shooting actions, the world coordinate system of the camera is converted from a certain camera coordinate system to a workpiece coordinate system, the measurement is convenient, and the initial parameters of the camera can be optimized based on the calibration workpiece hole center, so that the multi-vision calibration method based on the workpiece hole center in the embodiment is realized.
In order to achieve the above object, a system for multi-vision calibration based on workpiece hole center according to a fourth aspect of the present invention includes: the image acquisition device is used for acquiring image data of the calibration plate and sample image data of the calibration workpiece; the device for multi-vision calibration based on workpiece hole center according to the embodiment of the third aspect is connected to the image acquisition device, and is used for optimizing the initial parameters of the camera.
According to the system for multi-vision calibration based on the workpiece hole center, the hole center detection device based on the deep learning provided by the embodiment of the invention can be used for converting the world coordinate system of a camera from a certain camera coordinate system into a workpiece coordinate system, namely, the measurement result can be directly expressed on the workpiece coordinate system, and the measurement is convenient. The camera initial parameters can be optimized based on the calibrated workpiece hole center, so that the detection precision of the camera is improved, and the generalization of the measurement of the calibrated workpiece is improved.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the invention will become apparent and may be better understood from the following description of embodiments taken in conjunction with the accompanying drawings in which:
FIG. 1 is a flow chart of a method for multi-vision calibration based on workpiece aperture cores, in accordance with one embodiment of the invention;
FIG. 2 is a flow chart of a method for multi-vision calibration based on a workpiece aperture core, in accordance with another embodiment of the invention;
FIG. 3 is a flow chart of a method for multi-vision calibration based on a workpiece aperture core, in accordance with yet another embodiment of the present invention;
FIG. 4 is a flow chart of a method for multi-vision calibration based on a workpiece aperture core, in accordance with yet another embodiment of the present invention;
FIG. 5 is a schematic diagram of a template picture according to one embodiment of the invention;
FIG. 6 is a block diagram of an apparatus for multi-vision calibration based on workpiece aperture cores, in accordance with one embodiment of the invention;
FIG. 7 is a block diagram of a system for multi-vision calibration based on workpiece aperture cores, in accordance with one embodiment of the invention.
Reference numerals:
a system 100 for multi-vision calibration based on the workpiece aperture;
a device 10 for multi-vision calibration based on the hole center of the workpiece and an image acquisition device 20;
an image acquisition module 101, a data generation module 102, a coordinate system conversion module 103 and a parameter optimization module 104.
Detailed Description
Embodiments of the present invention will be described in detail below, by way of example with reference to the accompanying drawings.
In order to solve the above-mentioned problems, an embodiment of the present invention provides a method for multi-vision calibration based on a workpiece hole center, and the method for multi-vision calibration based on a workpiece hole center according to the embodiment of the present invention is described below with reference to fig. 1 to 5, as shown in fig. 1, and is a flowchart of a method for multi-vision calibration based on a workpiece hole center according to an embodiment of the present invention, where the method at least includes steps S1 to S5, and is specifically described below.
S1, acquiring image data of calibration plates under multiple camera visual angles.
In an embodiment, the calibration plate is a square calibration plate, and each calibration plate is provided with a plurality of markers. The Kalibr calibration tool has a multi-camera calibration function, and the square calibration plate can be manufactured according to a calibration plate sample provided by the Kalibr calibration tool.
And fixing the positions of the plurality of cameras and moving the calibration plates, and controlling the plurality of cameras to take pictures so as to acquire images of the plurality of calibration plates as image data of the calibration plates. The plurality of cameras may include n cameras, where n may be set as needed or under laboratory conditions, for example, n may be 50, 66, 80, etc., and the position of each camera is fixed. Under the layout of the fixed cameras, the calibration plates are moved for a plurality of times, so that images of at least three calibration plates are guaranteed to be included under the view angle of any one of the plurality of cameras, and a plurality of markers are included in the images of any one calibration plate. The calibration plate is moved for a plurality of times, and each time the calibration plate is moved, all cameras are controlled to take a picture once, a plurality of markers are contained in one calibration plate image taken under the visual angle of one camera as much as possible, and all or part of image ranges taken by the cameras are overlapped, so that scene splicing is realized during workpiece measurement.
S2, acquiring initial parameters of the camera according to the image data of the calibration plate, wherein the initial parameters of the camera comprise internal parameters of the camera, external parameters of the camera and distortion coefficients of the camera.
In an embodiment, calibration plate images shot by a plurality of cameras are input, and the input calibration plate images are calibrated according to the multi-camera calibration function of the kalibr calibration tool to obtain camera internal parameters, camera external parameters and camera distortion coefficients of the cameras. Taking 66 cameras as an example, the 66 cameras can be respectively numbered according to the setting positions or other characteristics of the cameras, and the like, which are respectively numbered 1-66, and the camera coordinate system of the No. 1 camera can be used as camera external parameters.
S3, sample image data of the calibration workpiece under the view angles of the cameras are obtained.
The calibration workpiece is supported by the support, 66 cameras are respectively arranged at all directions of the calibration workpiece, each hole site on the calibration workpiece is required to be covered by at least two camera fields of view, and the position of the calibration workpiece and the relative positions of the cameras are also fixed. Under the fixed camera layout, all cameras shoot the calibration workpiece once and process the acquired images of the workpiece to be calibrated so as to acquire sample image data of the calibration workpiece.
S4, converting a world coordinate system where initial parameters of the camera are located into a workpiece coordinate system based on sample image data of the calibrated workpiece, and obtaining camera external parameters after the coordinate system conversion.
In an embodiment, the world coordinate system in which the camera is located is the camera coordinate system in which the camera No. 1 is located, where matrix transformation may be performed according to an SVD (Singular Value Decomposition ) algorithm to transform the world coordinate system in which the camera initial parameters are located into the workpiece coordinate system.
S5, optimizing initial parameters of the camera based on the calibrated workpiece hole center.
The method comprises the steps of obtaining three-dimensional measured values of a calibrated workpiece as 3D points, fixing the 3D points based on the Kalibr calibrated camera internal parameters and the camera external parameters after coordinate system conversion, and optimizing initial parameters of the camera to obtain more accurate camera parameters.
According to the multi-vision calibration method based on the workpiece hole center, based on the camera initial parameters and the sample image data of the calibrated workpiece, the world coordinate system where the camera is located is converted from a certain camera coordinate system to a workpiece coordinate system, namely, the measurement result can be directly represented on the workpiece coordinate system, and the measurement is convenient. And the initial parameters of the camera are optimized based on the hole center of the calibration workpiece, so that the detection precision of the camera is improved, and the generalization of the measurement of the calibration workpiece is improved.
In some embodiments of the present invention, as shown in fig. 2, a flowchart of a method for multi-vision calibration based on a hole center of a workpiece according to another embodiment of the present invention is provided, wherein sample image data of a calibrated workpiece at multiple camera angles is obtained, that is, the above step S3 includes steps S31-S34, which is specifically as follows.
S31, acquiring actual image data of the calibration workpiece.
Under the fixed camera layout, all cameras shoot the calibration workpiece once to acquire an image of the calibration workpiece, namely actual image data of the measurement workpiece.
S32, performing ROI labeling on the part holes in the actual image data to obtain labeled image data.
The ROI can be marked on each acquired image of the workpiece to be calibrated manually, namely an image area is selected from the images, the area is focused on the image analysis, the area is marked so as to further process the images, the processing time of image data can be reduced, and the accuracy is improved.
Specifically, the ROI labeling can be represented by coordinates of two points, wherein when the ROI labeling is performed, an image region in the image of the workpiece to be calibrated can be selected, the upper left and lower right points of the image region are labeled and represented as a rectangle, the rectangle is the labeled ROI, and each ROI contains one hole site of the workpiece to be calibrated. The part holes in the actual image data are marked by the ROI, the marked ROI is not changed only when the part holes are first set, and the marked ROI can be directly used for measuring subsequent workpieces.
S33, de-distorting the marked image data according to the camera internal parameters and the camera distortion coefficients to obtain processed image data.
Wherein the camera acquires an image of the calibration artifact that has been substantially transformed by the coordinate system. For example, a point in space is converted from a world coordinate system to a camera coordinate system, then the point is projected onto an imaging plane and then converted into an image physical coordinate system, finally data on the imaging plane is converted into an image pixel coordinate system, the image of a calibrated workpiece can be distorted after being converted into the coordinate system, and a camera can generate some interference after imaging distortion. After the contour marking is carried out on the part hole in the actual image data, the distortion of the marked image data is removed according to the acquired Kalibr calibrated camera internal parameters and the camera distortion coefficients, and the distortion of the marked coordinates in the image data is removed, so that the interference in the image data can be reduced, and the processed image data is obtained.
And S34, fitting an ellipse in the processed image data according to an ellipse detection algorithm to obtain a 2D observation point with the center of the ellipse as the calibration workpiece hole center.
In the embodiment, in the processed image data, an ellipse is fitted in each ROI by adopting an ellipse detection algorithm to obtain an ellipse center, the ellipse center is taken as a 2D observation point for calibrating the hole center of the workpiece, wherein the 2D observation point for calibrating the hole center of the workpiece is recorded as X center 。
In some embodiments of the present invention, as shown in fig. 3, a flowchart of a method for multi-vision calibration based on a workpiece hole center according to another embodiment of the present invention is shown, wherein the world coordinate system where the initial parameters of the conversion camera in the above step S4 are located is the workpiece coordinate system, and the method includes the following steps S41 and S42.
S41, reconstructing 2D observation points of the hole center of the calibration workpiece based on camera internal reference and ellipse detection to obtain reconstruction point cloud of the calibration workpiece.
Specifically, as shown in formula (1-1), based on the principle of triangulation reconstruction, a Kalibr-calibrated camera is utilized to internally reference and calibrate a 2D observation point X of a workpiece hole center center Reconstructing to obtain a reconstruction point cloud of the calibrated workpiece and recording as X CCD1 . Wherein K represents Kalibr-calibrated camera reference, [ R|T ]]Representing Kalibr calibrated camera parameters.
X center ＝K[R|T]X CCD1 (1-1)
S42, determining a rigid transformation relation of the reconstructed point cloud and the three-coordinate measured point cloud of the calibration workpiece, and applying the rigid transformation relation to the camera external parameters.
Wherein, as shown in the formula (1-2), the rigid transformation relation between the point cloud and the three-coordinate measured point cloud of the calibration workpiece can be reconstructed according to the SVD algorithm, and the rigid transformation is applied to the external parameters of the Kalibr calibrated camera, so that the world coordinate system of the camera is converted into the calibration workpiece coordinate system, X CMM Representing a reconstructed point cloud of points,representing a three-coordinate measured point cloud X CMM And reconstructing point cloud X CCD1 Is a rigid transformation relation of->Camera external parameters after converting the world coordinate system.
X center ＝K[R|T][R CMM→CCD1 |T CMM→CCD1 ]X CMM (1-2)
In some embodiments of the present invention, as shown in fig. 4, a flowchart of a method for multi-vision calibration based on a workpiece hole center according to still another embodiment of the present invention is provided, wherein the camera initial parameters are optimized based on the calibrated workpiece hole center, that is, the above step S5 includes at least step S51 and step S52, which is specifically as follows.
S51, projecting the clear and definite value of the calibration workpiece to an image plane according to the camera internal parameter and the camera external parameter after the coordinate system conversion to obtain a template picture.
Specifically, the camera internal parameters calibrated by Kalibr and the camera external parameters after converting the coordinate system are used to project the nominal workpiece definition values to the image plane to obtain a template picture, as shown in fig. 5, which is a schematic diagram of the template picture according to an embodiment of the present invention, wherein the picture includes a hole site of the nominal workpiece in an ROI.
And S52, matching the marked image data with the template picture to obtain the optimized 2D observation point of the calibrated workpiece hole center.
In an embodiment, taking a template picture as shown in fig. 5 as an example, template matching is performed in the ROI to obtain a more accurate 2D observation point for calibrating the hole center of the workpiece, so as to solve the problem that the hole center of the calibrated workpiece is inconsistent with the ellipse center after being projected to the image plane.
In some embodiments of the present invention, as shown in fig. 4, step S5 above optimizes camera initial parameters based on calibrating the workpiece aperture, and further includes steps S53-S56, as follows.
S53, obtaining the three-coordinate measured value of the calibration workpiece as a 3D point.
And taking the three-coordinate measured value of the calibration workpiece as a 3D point, and taking the hole center of the calibration workpiece obtained by template matching as a 2D observation point.
S54, acquiring the camera internal parameters and the camera external parameters after the coordinate system conversion as initial values of the camera. The camera internal parameters calibrated by Kalibr and the camera external parameters after the coordinate system conversion are used as initial values of the camera.
S55, fixing a 3D point according to the optimized 2D observation point of the calibrated workpiece hole center and the initial value of the camera.
S56, optimizing initial parameters of the camera according to the adjustment of the beam method.
Specifically, the beam method adjustment formula is shown in the formula (1-3), wherein j represents the number of a space point, i represents the number of a camera, and 1.ltoreq.i.ltoreq.66. w (w) ij As the weight, if the jth spatial point is not projected on the camera i, the weight is 0, otherwise, the weight is 1.f (P) i ,X j ) Representing the j-th spatial point at the 2D point projected on camera i. X is x ij And (3) representing the observation point of the corresponding j-number space point on the i-number picture, namely obtaining the calibrated workpiece hole center through template matching. Fixed spatial point x= { X 1 ,X 2 ,Λ,X n The three-coordinate measurement value of the calibration workpiece is represented, so that the re-projection error is reduced, and the optimal initial parameter P= { P of the camera is obtained 1 ,P 2 ,Λ,P n }。
Wherein, because the round hole of the workpiece is a hollow circle, the three-coordinate measuring machine is difficult to measure the Z coordinate of the hole center for the round hole on the horizontal plane, and is difficult to measure the Y coordinate of the hole center for the round hole on the side surface, and when measuring and calibrating the three-coordinate measured value of the workpiece, the coordinate values which are difficult to measure can be replaced by the clear value coordinates. In the calibration optimization process, aiming at a round hole on a horizontal plane, the inner parameter and the outer parameter of a camera are optimized, and meanwhile, the X, Y coordinate is fixed and the Z coordinate is optimized. For circular holes on the sides, X, Z coordinates are fixed and Y coordinates are optimized while optimizing the internal and external parameters of the camera.
In some embodiments of the present invention, as shown in fig. 6, a block diagram of an apparatus for multi-vision calibration based on a workpiece hole center according to an embodiment of the present invention is shown, wherein an apparatus 100 for multi-vision calibration based on a workpiece hole center includes an image acquisition module 101, a data generation module 102, a coordinate system conversion module 103, and a parameter optimization module 104.
The image acquisition module 101 is used for acquiring image data of the calibration plate under the view angles of the plurality of cameras and acquiring sample image data of the calibration workpiece under the view angles of the plurality of cameras. The data generating module 102 is configured to obtain initial parameters of the camera according to the image data of the calibration board, where the initial parameters of the camera include internal parameters of the camera, external parameters of the camera, and distortion coefficients of the camera.
The image acquisition module 101 acquires image data of the calibration plates under the view angles of the plurality of cameras, specifically, the calibration plates are moved for a plurality of times under the layout of the fixed cameras, and each time the calibration plates are moved, all the cameras are controlled to take a picture, and one of the captured images of the calibration plates under the view angles of one camera is controlled to contain a plurality of markers as much as possible. The data generating module 102 is configured to obtain initial parameters of the camera according to image data of the calibration plate, and then the image obtaining module 101 photographs the calibration workpiece once to obtain sample image data of the calibration workpiece under multiple camera angles, and de-distorts the actual image data according to the camera internal parameters and the camera distortion coefficients.
The coordinate system conversion module 103 is configured to convert, based on sample image data of the calibration workpiece, a world coordinate system in which initial parameters of the camera are located into a workpiece coordinate system, and obtain camera parameters after the coordinate system conversion. The coordinate system conversion module 103 may perform matrix conversion according to an SVD (Singular Value Decomposition ) algorithm to convert a world coordinate system where the camera initial parameters are located into a workpiece coordinate system.
The parameter optimization module 104 is configured to optimize camera initial parameters based on the calibrated workpiece aperture center. For example, three-coordinate measurements of the calibration workpiece may be obtained as 3D points, and the Kalibr calibration-based camera internal parameters and the camera external parameters after transformation of the coordinate system may be used to fix the 3D points, and the camera initial parameters may be optimized to obtain more accurate camera parameters.
It should be noted that, the specific implementation manner of the apparatus 100 for workpiece hole center based multi-vision calibration according to the embodiment of the present invention is similar to the specific implementation manner of the method for workpiece hole center based multi-vision calibration according to any of the above embodiments of the present invention, and specific reference is made to the description of this method section, and for redundancy reduction, details are not repeated here.
According to the device 10 for multi-vision calibration based on the hole center of the workpiece, which is disclosed by the embodiment of the invention, based on the architectures of the image acquisition module 101, the data generation module 102, the coordinate system conversion module 103 and the parameter optimization module 104, the coordinate system conversion module 103 is used for converting the world coordinate system of the camera from a certain camera coordinate system to a workpiece coordinate system based on the obtained initial parameters of the camera and the sample image data of the calibrated workpiece, namely, the measurement result can be directly represented on the workpiece coordinate system, so that the measurement is convenient. And the parameter optimization module 104 optimizes the initial parameters of the camera based on the hole center of the calibration workpiece, so that the detection precision of the camera is improved, and the generalization of the measurement of the calibration workpiece is improved.
In some embodiments of the present invention, a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of multi-vision calibration based on a workpiece aperture of any of the above embodiments is also presented.
According to the non-transitory computer readable storage medium of the embodiment of the invention, a computer program is stored on the storage medium, when the computer program runs, the operation parameters of each structure in the device 10 can be obtained for analysis and calculation, in the process of detecting a calibration workpiece, a processor can control a camera to carry out shooting actions, a world coordinate system where the camera is located is converted from a certain camera coordinate system to a workpiece coordinate system, measurement is convenient, and the initial parameters of the camera can be optimized based on the calibration workpiece hole center, so that the multi-vision calibration method based on the workpiece hole center of the embodiment is realized.
In some embodiments of the present invention, as shown in fig. 7, a block diagram of a system for multi-vision calibration based on a workpiece hole center according to an embodiment of the present invention is shown, where the system 100 for multi-vision calibration based on a workpiece hole center includes an image acquisition device 20 and the device 10 for multi-vision calibration based on a workpiece hole center provided in the above embodiment.
The image acquisition device 20 is used for acquiring image data of a calibration plate and sample image data of a calibration workpiece; the device 10 based on multi-vision calibration of the hole center of the workpiece is connected with the image acquisition device 20 and is used for optimizing initial parameters of a camera. The image capturing device 20 may include n cameras, and the position of each camera is fixed.
It should be noted that, the specific implementation manner of the system 100 for workpiece hole center based multi-vision calibration according to the embodiment of the present invention is similar to the specific implementation manner of the device 10 for workpiece hole center based multi-vision calibration according to any of the above embodiments of the present invention, please refer to the description of the portion of the device 10 for workpiece hole center based multi-vision calibration specifically, and for redundancy reduction, the description is omitted here.
According to the system 100 for multi-vision calibration based on the hole center of the workpiece, the hole center detection device 10 based on the deep learning provided by the embodiment of the invention can convert the world coordinate system where the camera is located from a certain camera coordinate system into the workpiece coordinate system, namely, the measurement result can be directly represented on the workpiece coordinate system, so that the measurement is convenient. The camera initial parameters can be optimized based on the calibrated workpiece hole center, so that the detection precision of the camera is improved, and the generalization of the measurement of the calibrated workpiece is improved.
In the description of this specification, any process or method description in a flowchart or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing logical functions or steps of the process, and in which the scope of the preferred embodiments of the present invention include additional implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present invention.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As with the other embodiments, if implemented in hardware, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like. While embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the invention.
In the description of the present specification, reference to the terms "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples.
While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the invention, the scope of which is defined by the claims and their equivalents.
Claims (10)
1. A method for multi-vision calibration based on a workpiece hole center, comprising:
acquiring image data of a calibration plate under the view angles of a plurality of cameras;
acquiring initial parameters of a camera according to the image data of the calibration plate, wherein the initial parameters of the camera comprise internal parameters of the camera, external parameters of the camera and distortion coefficients of the camera;
acquiring sample image data of a calibration workpiece under the view angles of the cameras;
converting a world coordinate system in which the initial parameters of the camera are positioned into a workpiece coordinate system based on the sample image data of the calibrated workpiece, and obtaining camera external parameters after the coordinate system conversion;
and optimizing the initial parameters of the camera based on the calibrated workpiece hole center.
2. The method for multi-vision calibration based on workpiece aperture of claim 1, wherein acquiring image data of a calibration plate at a multi-vision camera view angle comprises:
the calibration plate is a square calibration plate, and a plurality of markers are arranged on the calibration plate;
and fixing the positions of the plurality of cameras and moving the calibration plate, and controlling the plurality of cameras to take pictures so as to acquire images of the plurality of calibration plates as image data of the calibration plate.
3. The method of claim 2, wherein any one of the plurality of cameras comprises at least three images of the calibration plate at the view angle, and wherein any one of the plurality of images of the calibration plate comprises a plurality of the markers.
4. The method of claim 1, wherein the obtaining sample image data of the calibrated workpiece at the plurality of camera perspectives comprises:
acquiring actual image data of the calibration workpiece;
performing ROI labeling on the part holes in the actual image data to obtain labeled image data;
performing de-distortion processing on the marked image data according to the camera internal parameters and the camera distortion coefficients to obtain processed image data;
and fitting an ellipse in the processed image data according to an ellipse detection algorithm to obtain an ellipse center serving as a 2D observation point of the calibration workpiece hole center.
5. The method of claim 4, wherein converting the world coordinate system in which the camera initial parameters are located into the workpiece coordinate system comprises:
reconstructing 2D observation points of the hole center of the calibration workpiece based on the camera internal parameters and the ellipse detection algorithm to obtain reconstruction point cloud of the calibration workpiece;
and determining the rigid transformation relation of the reconstruction point cloud and the three-coordinate measured point cloud of the calibration workpiece, and applying the rigid transformation relation to the camera external parameters.
6. The method of workpiece aperture-based multiview calibration of claim 4, wherein the optimizing the camera initial parameters based on calibrating the workpiece aperture comprises:
projecting the nominal value of the calibration workpiece to an image plane according to the camera internal parameter and the camera external parameter after the coordinate system conversion to obtain a template picture;
and matching the marked image data with the template picture to obtain the optimized 2D observation point for calibrating the hole center of the workpiece.
7. The method of workpiece aperture-based multiview calibration of claim 6, wherein the optimizing the camera initial parameters based on calibrating the workpiece aperture further comprises:
acquiring a three-dimensional measured value of the calibration workpiece as a 3D point;
acquiring the camera internal parameters and the camera external parameters after the coordinate system conversion as initial values of the camera;
fixing the 3D point according to the optimized 2D observation point of the calibrated workpiece hole center and the initial value of the camera;
and optimizing the initial parameters of the camera according to the adjustment of the beam method.
8. A device for multi-vision calibration based on a workpiece hole center, comprising:
the image acquisition module is used for acquiring image data of the calibration plates under the view angles of the cameras and acquiring sample image data of the calibration workpieces under the view angles of the cameras;
the data generation module is used for acquiring initial parameters of the camera according to the image data of the calibration plate, wherein the initial parameters of the camera comprise internal parameters of the camera, external parameters of the camera and distortion coefficients of the camera;
the coordinate system conversion module is used for converting the world coordinate system where the initial parameters of the camera are positioned into a workpiece coordinate system based on the sample image data of the calibration workpiece, and obtaining camera external parameters after the coordinate system conversion;
and the parameter optimization module is used for optimizing the initial parameters of the camera based on the calibrated workpiece hole center.
9. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, is operable to implement the method of workpiece aperture core based multi-vision calibration of any one of claims 1-7.
10. A system for multi-vision calibration based on a workpiece aperture, comprising:
the image acquisition device is used for acquiring image data of the calibration plate and sample image data of the calibration workpiece;
the device for multi-vision calibration based on workpiece hole center of claim 8, which is connected with the image acquisition device for optimizing the initial parameters of the camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110912869.3A CN113658270B (en) | 2021-08-10 | 2021-08-10 | Method, device, medium and system for multi-vision calibration based on workpiece hole center |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110912869.3A CN113658270B (en) | 2021-08-10 | 2021-08-10 | Method, device, medium and system for multi-vision calibration based on workpiece hole center |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113658270A CN113658270A (en) | 2021-11-16 |
| CN113658270B true CN113658270B (en) | 2023-09-29 |
Family
ID=78491128
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110912869.3A Active CN113658270B (en) | 2021-08-10 | 2021-08-10 | Method, device, medium and system for multi-vision calibration based on workpiece hole center |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113658270B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116878381B (en) * | 2023-08-01 | 2024-01-30 | 湖南视比特机器人有限公司 | Online full-size detection method and system based on multi-eye vision |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1251329A2 (en) * | 2001-04-16 | 2002-10-23 | VMT Bildverarbeitungssysteme GmbH | Procedure to stabilise the viewed point by 3D-positiondetermination of an object with a digital camera |
| CN108288294A (en) * | 2018-01-17 | 2018-07-17 | 视缘(上海)智能科技有限公司 | A kind of outer ginseng scaling method of a 3D phases group of planes |
| CN109489591A (en) * | 2018-12-17 | 2019-03-19 | 吉林大学 | Plane scratch length non-contact measurement method based on machine vision |
| CN110954067A (en) * | 2019-12-28 | 2020-04-03 | 长安大学 | Monocular vision excavator pose measurement system and method based on target |
| CN111308448A (en) * | 2018-12-10 | 2020-06-19 | 杭州海康威视数字技术股份有限公司 | Image acquisition equipment and radar external parameter determination method and device |
-
2021
- 2021-08-10 CN CN202110912869.3A patent/CN113658270B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1251329A2 (en) * | 2001-04-16 | 2002-10-23 | VMT Bildverarbeitungssysteme GmbH | Procedure to stabilise the viewed point by 3D-positiondetermination of an object with a digital camera |
| CN108288294A (en) * | 2018-01-17 | 2018-07-17 | 视缘(上海)智能科技有限公司 | A kind of outer ginseng scaling method of a 3D phases group of planes |
| CN111308448A (en) * | 2018-12-10 | 2020-06-19 | 杭州海康威视数字技术股份有限公司 | Image acquisition equipment and radar external parameter determination method and device |
| CN109489591A (en) * | 2018-12-17 | 2019-03-19 | 吉林大学 | Plane scratch length non-contact measurement method based on machine vision |
| CN110954067A (en) * | 2019-12-28 | 2020-04-03 | 长安大学 | Monocular vision excavator pose measurement system and method based on target |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113658270A (en) | 2021-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111505606B (en) | Method and device for calibrating relative pose of multi-camera and laser radar system | |
| US9965870B2 (en) | Camera calibration method using a calibration target | |
| CN104713885B (en) | A kind of structure light for pcb board on-line checking aids in binocular measuring method | |
| WO2012053521A1 (en) | Optical information processing device, optical information processing method, optical information processing system, and optical information processing program | |
| CN105716542B (en) | A kind of three-dimensional data joining method based on flexible characteristic point | |
| CN110728715A (en) | Camera angle self-adaptive adjusting method of intelligent inspection robot | |
| CN106625713A (en) | Method of improving gumming accuracy of gumming industrial robot | |
| WO2018201677A1 (en) | Bundle adjustment-based calibration method and device for telecentric lens-containing three-dimensional imaging system | |
| CN110044262B (en) | Non-contact precision measuring instrument based on image super-resolution reconstruction and measuring method | |
| TW201525415A (en) | Method and system for calibrating laser measuring apparatus | |
| CN112815843B (en) | On-line monitoring method for printing deviation of workpiece surface in 3D printing process | |
| CN113920205B (en) | Calibration method of non-coaxial camera | |
| JPWO2017056473A1 (en) | Camera calibration board, camera calibration device, camera calibration method, and camera calibration program recording medium | |
| US9157874B2 (en) | System and method for automated x-ray inspection | |
| CN112229323B (en) | Six-degree-of-freedom measurement method of checkerboard cooperative target based on monocular vision of mobile phone and application of six-degree-of-freedom measurement method | |
| CN114283203A (en) | Calibration method and system of multi-camera system | |
| CN113658270B (en) | Method, device, medium and system for multi-vision calibration based on workpiece hole center | |
| WO2018168757A1 (en) | Image processing device, system, image processing method, article manufacturing method, and program | |
| JP5599849B2 (en) | Lens inspection apparatus and method | |
| CN113724337A (en) | Camera dynamic external parameter calibration method and device without depending on holder angle | |
| CN112308930A (en) | Camera external parameter calibration method, system and device | |
| CN104167001A (en) | Large-visual-field camera calibration method based on orthogonal compensation | |
| CN115187612A (en) | Plane area measuring method, device and system based on machine vision | |
| CN116625258A (en) | Chain spacing measuring system and chain spacing measuring method | |
| CN114792345A (en) | Calibration method based on monocular structured light system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |