CN117557731A - Forging temperature field distribution and three-dimensional size integrated reconstruction method - Google Patents
Forging temperature field distribution and three-dimensional size integrated reconstruction method Download PDFInfo
- Publication number
- CN117557731A CN117557731A CN202311670133.5A CN202311670133A CN117557731A CN 117557731 A CN117557731 A CN 117557731A CN 202311670133 A CN202311670133 A CN 202311670133A CN 117557731 A CN117557731 A CN 117557731A
- Authority
- CN
- China
- Prior art keywords
- image
- forging
- dimensional
- infrared
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005242 forging Methods 0.000 title claims abstract description 83
- 238000000034 method Methods 0.000 title claims abstract description 74
- 238000009826 distribution Methods 0.000 title claims abstract description 34
- 238000012545 processing Methods 0.000 claims abstract description 40
- 238000004364 calculation method Methods 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 9
- 238000001514 detection method Methods 0.000 claims abstract description 8
- 230000005855 radiation Effects 0.000 claims abstract description 8
- 230000009466 transformation Effects 0.000 claims abstract description 5
- 230000004927 fusion Effects 0.000 claims abstract description 4
- 238000009529 body temperature measurement Methods 0.000 claims description 25
- 238000001914 filtration Methods 0.000 claims description 15
- 238000005259 measurement Methods 0.000 claims description 15
- 230000005457 Black-body radiation Effects 0.000 claims description 11
- 230000001131 transforming effect Effects 0.000 claims description 11
- 239000011159 matrix material Substances 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 238000013507 mapping Methods 0.000 claims description 9
- 239000013598 vector Substances 0.000 claims description 9
- 238000012937 correction Methods 0.000 claims description 8
- 238000013519 translation Methods 0.000 claims description 7
- 238000007476 Maximum Likelihood Methods 0.000 claims description 6
- 238000007670 refining Methods 0.000 claims description 5
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 230000036760 body temperature Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 4
- 230000000007 visual effect Effects 0.000 abstract 1
- 238000005516 engineering process Methods 0.000 description 6
- 238000003384 imaging method Methods 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000010438 heat treatment Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000001931 thermography Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Geometry (AREA)
- Computer Graphics (AREA)
- Radiation Pyrometers (AREA)
Abstract
The invention discloses a forge piece temperature field distribution and three-dimensional size integrated reconstruction method, which comprises the following steps: calibrating a binocular camera, projecting cross laser to the surface of a forging to be measured, respectively acquiring images of the forging to be measured by a left camera and a right camera, acquiring a first image and a second image, preprocessing the images to acquire point cloud information of the surface of the forging, splicing the point cloud, and measuring the size of the forging by utilizing three-dimensional reconstruction information; performing radiation calibration on the infrared camera through a blackbody source, and performing enhancement processing and super-resolution processing on the thermal infrared image; and the pose relation of the binocular camera and the infrared camera is determined through calibration, the space points are subjected to coordinate transformation, and fusion registration of the infrared image and the three-dimensional point cloud is carried out, so that the integrated reconstruction of the temperature and the size of the forge piece is realized. The invention simplifies visual matching calculation, enhances the three-dimensional display effect of the temperature field, can realize quick detection and reconstruction of the surface of the forging piece, and has better matching precision.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a forge piece temperature field distribution and three-dimensional size integrated reconstruction method.
Background
The online dimension measurement of the large-sized forging is to measure the main dimension of the forging in the manufacturing process of the forging so as to judge whether the dimension of the forging meets the process requirement, adjust forging equipment in time and decide whether to terminate forging, however, the online dimension measurement of the high-temperature forging has a plurality of defects. The traditional measuring method adopts a manual measuring method, and uses an artificial naked eye or a caliper to measure, so that the manual contact type measuring method has large measuring error and low efficiency, and can have great harm to personal safety under the severe forging production environment. Many non-contact measuring methods are also widely used at present, such as: the research of the non-contact measuring method at the present stage mainly focuses on the laser measuring technology and the computer vision technology, wherein the binocular vision measuring technology in the computer vision measuring technology has the advantages of non-contact, wide range, high precision, quick dynamic response and the like.
In the forging process of a large-sized forging, the change condition of a temperature field of the forging in the heat treatment process is known in time, and the method is important to ensure the correct implementation of the heat treatment process of the forging. At present, the infrared temperature measurement is the most widely applied temperature field measurement method because of the advantages of convenience, rapidness, flexibility, accuracy, non-contact and the like. However, how to obtain the temperature field of the measured object and ensure higher temperature measurement accuracy is still an urgent problem to be solved.
Two-dimensional infrared projection imaging can lead to inaccurate temperature measurement, establishes the three-dimensional temperature field of high temperature forging and helps reducing the temperature distribution of forging in three-dimensional space to improve the richness of temperature information, carry out the integration reconfiguration of temperature and three-dimensional size of forging simultaneously, grasp the temperature and the size information of forging in real time, have important guide effect to the production manufacturing of forging. Therefore, the research of the temperature field and the three-dimensional dimension integrated reconstruction technology has important significance for forging the high-temperature forge piece.
Disclosure of Invention
The invention aims to provide a binocular vision and infrared thermal imaging-based forge piece temperature field distribution and three-dimensional size integrated reconstruction method, which is used for solving the problems of low efficiency, large measurement error, harm to human bodies and the like in the measurement of the size of a large-sized high-temperature forge piece in the prior art, solving the problem that two-dimensional infrared projection imaging can cause inaccurate temperature measurement, carrying out integrated reconstruction of the temperature and the three-dimensional size of the forge piece, and grasping the temperature and the size information of the forge piece in real time.
In order to achieve the above object, the present invention provides the following technical solutions:
a forge piece temperature field distribution and three-dimensional size integrated reconstruction method is characterized by comprising the following steps of: the method comprises the following steps:
s1: three-dimensional reconstruction is carried out on the forge piece through a binocular camera, and the size of the forge piece is measured; comprising the following steps:
s11: calibrating the binocular cameras to obtain internal parameters and external parameters of the left camera and the right camera;
s12: projecting cross laser to the surface of the forging to be detected, and acquiring a first image and a second image of the forging to be detected by a left camera and a right camera;
s13: carrying out three-dimensional correction on the image to enable the image to meet epipolar constraint;
s14: a preprocessing step of binarizing, closing and refining the image;
s15: positioning the center of the laser cross;
s16: converting the cross center coordinates into three-dimensional coordinates in a world coordinate system;
s17: acquiring point cloud information on the surface of the forging piece, splicing the acquired point cloud information, and performing three-dimensional reconstruction;
s18: realizing the dimension measurement of the forging by utilizing the three-dimensional reconstruction information;
s2: acquiring a thermal infrared image of the forging piece by using an infrared camera, and measuring the temperature of the forging piece; comprising the following steps:
s21: performing radiation calibration on the infrared camera by using a blackbody radiation source, fitting a temperature measurement curve, and correcting a temperature measurement result;
s22: performing thermal infrared image enhancement processing to enhance image contrast;
s23: performing image super-resolution processing to further improve the quality of the infrared image;
s3: registering the infrared image and the three-dimensional point cloud in a fusion way, so as to realize the integral reconstruction of the temperature and the size of the forging piece; comprising the following steps:
s31: determining the pose relation of the binocular vision camera and the infrared camera;
s32: transforming the space point from the binocular vision camera coordinate system to the world coordinate system, and then converting the space point to the infrared camera coordinate system;
s33: calculating gray values of projection points, and completing solving of the corresponding relation between infrared information and the three-dimensional point cloud;
s34: and correcting errors and compensating offset according to the condition that registration offset occurs in the temperature assignment of the three-dimensional points.
As a further preferred embodiment of the present invention, in the step S11, the calibration of the binocular camera, obtaining the internal parameters and the external parameters of the left and right cameras includes:
calibrating the binocular camera by adopting a Zhang calibration method;
the acquired internal parameters of the left and right cameras at least comprise distortion coefficients;
the acquired external parameters of the left and right cameras at least comprise a rotation matrix R and a translation vector T.
As a further preferred embodiment of the present invention, in the step S13, the acquired image is stereo-corrected using a Bouguet algorithm so as to satisfy epipolar constraint.
As a further preferred embodiment of the present invention, the preprocessing step of performing binarization, closing operation and refinement processing on the image in the step S14 includes:
when the acquired image is subjected to binarization processing, a scale factor R is introduced, pixel points in the original image are ordered according to the size of an R channel value, the pixel point value of the first R percent is set as 1, and the rest is set as 0;
when the image subjected to binarization processing is subjected to closed operation processing, firstly, expanding the binarization image, filling holes in the image, reconnecting the disconnected image, and then, corroding the image to ensure that the whole image is changed back to the size before expansion;
when the image subjected to the closed operation processing is subjected to the refinement processing, an algorithm for refining the binary region is used to simplify the original pixel block into a binary image in which single pixels are connected.
As a further preferred embodiment of the present invention, in the step S15, the positioning of the center of the laser cross includes a linear filtering process and a contour detection, wherein,
the linear filtering processing is carried out by designing a filtering operator, and the filtering operator is used for filtering the image, wherein the coordinates of the pixels which are larger than the expected threshold value and smaller than 0 pixel point in the image are the coordinates of the potential laser cross center;
the contour detection is carried out by setting a rectangular area with a potential laser cross center as a center and a side length of b as an interested area, searching a contour in the area, discarding the point if 4 blank areas cannot be found, judging the next coordinate point until the rectangular area meeting the requirement is found, obtaining a contour point set of the 4 blank areas, further obtaining the relative positions of the 4 contours, calculating the shortest distance between the diagonal areas, obtaining two points of the shortest distance, obtaining the midpoint of the two points, and regarding the coordinates of the point as the coordinates of the laser cross center.
As a further preferred embodiment of the present invention, in the step S17, obtaining point cloud information of the surface of the forging, splicing the obtained point cloud information, and performing three-dimensional reconstruction includes:
s171: coarsely splicing the acquired point cloud information based on Super-4pcs algorithm;
s172: finely splicing the acquired point cloud information based on an ICP algorithm;
s173: and rotationally translating the plurality of pieces of point cloud data to a unified coordinate system.
As a further preferred embodiment of the present invention, in the step S21, the infrared camera is calibrated by using a blackbody radiation source, a temperature measurement curve is fitted, and the correcting the temperature measurement result includes:
preheating a blackbody radiation source and an infrared camera to a stable state, placing the infrared camera in a radiation field of the blackbody radiation source, and measuring the relationship between the infrared camera and the surface temperature of the blackbody;
the measurement data is specified to follow a gaussian distribution around the true value, and the calculation formula is as follows:
wherein,
f(V i ) Is a probability density function;
V(k 1 ,k 2 ,b；T i ) Is a temperature measurement function;
k 1 ，k 2 primary and secondary coefficients that are arguments;
b is the curve offset;
T i is black body temperature;
V i to measure gray scale;
sigma is the standard deviation;
e is a natural constant;
constructing a maximum likelihood function, wherein the calculation formula is as follows:
wherein,
l is a likelihood function;
n is a natural number set;
to maximize the maximum likelihood function, the formula is requiredAnd obtaining a minimum value, using a gradient descent method to find a global minimum value fitting temperature measurement curve of the Loss function, and determining a temperature measurement correction function.
As a further preferred embodiment of the present invention, in the step S22, performing the thermal infrared image enhancement process to enhance the image contrast includes:
s221: the image is subjected to gray mapping by a linear mapping method based on local gray level, and the calculation formula is as follows:
wherein,
I out to output an image;
I in is an input image;
delta is an adjustable parameter;
s222: the image is subjected to contrast enhancement processing through a self-adaptive histogram equalization algorithm, and the calculation formula is as follows:
y＝F CLAHE [f CLAHE (x)]
wherein,
y is the final output image;
x is the input infrared original image;
F CLAHE is a global CLAHE function;
f CLAHE is a local CLAHE function;
in the step S23, performing image super-resolution processing, further improving the quality of the infrared image includes:
the method comprises the steps of performing double resolution reconstruction processing on an input image through convolutional neural network super-resolution image processing, performing bicubic interpolation preprocessing on the input image to reach the target resolution, and performing image block extraction and encoding, nonlinear mapping of encoding vectors and calculation on a high-resolution image.
As a further preferred embodiment of the present invention, the transforming the spatial point from the binocular vision camera coordinate system to the world coordinate system and then converting it to the infrared camera coordinate system in the step S32 includes:
and transforming the space point P from the binocular vision camera coordinate system to the world coordinate system to obtain P ', and then transforming the space point P into the infrared camera coordinate system to obtain P ', wherein P ' is a new three-dimensional coordinate of the P point under the infrared camera coordinate system after coordinate transformation.
As a further preferred embodiment of the present invention, in the step S33, calculating the gray value of the projection point, and completing the solution of the correspondence between the infrared information and the three-dimensional point cloud includes:
projecting a point P' to an infrared camera pixel plane to obtain an infrared pixel point (u, v) corresponding to the point P, calculating a gray value of the projected point based on a bilinear interpolation method, and completing solving of the corresponding relation between infrared information and three-dimensional point cloud to realize the integrated reconstruction of the temperature and the size of the forging;
in the step S34, for the case that the registration offset occurs in the three-dimensional point temperature assignment, the error is corrected, and the offset compensation includes:
and converting the feature points corresponding to the depth image and the infrared image respectively into the same coordinate system, calculating the mean square error of the distance between the feature points of the converted depth image and the feature points of the infrared image, and correcting the error of registration offset condition of three-dimensional point temperature assignment.
Compared with the prior art, the invention has the beneficial effects that:
1) The invention provides an integrated reconstruction method for temperature field distribution and three-dimensional size of a forging, which can be used for carrying out non-contact measurement on the three-dimensional size of a large-scale high-temperature forging, effectively solving the forging size in a larger range, simplifying vision matching calculation, realizing good three-dimensional reconstruction effect, high measurement precision and quick dynamic response, having better matching precision, realizing rapid detection and reconstruction of the surface of the forging, effectively solving the problems of low efficiency, large error and the like of the traditional forging measurement, realizing real-time detection and visualization of the temperature of the high-temperature forging, realizing integrated reconstruction of the temperature and the size of the forging, and enhancing the three-dimensional display effect of the temperature field.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present invention, and other drawings may be obtained according to these drawings for a person having ordinary skill in the art.
FIG. 1 is a flow chart of steps of a forging temperature field distribution and three-dimensional dimension integrated reconstruction method provided by the invention;
FIG. 2 is a flow chart of forging dimension measurement steps of a forging temperature field distribution and three-dimensional dimension integrated reconstruction method provided by the invention;
FIG. 3 is a flow chart of forging temperature measurement steps of a forging temperature field distribution and three-dimensional dimension integrated reconstruction method provided by the invention;
fig. 4 is a flowchart of an infrared image and three-dimensional point cloud fusion step of a forging temperature field distribution and three-dimensional size integrated reconstruction method provided by the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper", "lower", "inner", "outer", "front", "rear", "both ends", "one end", "the other end", etc. are based on the directions or positional relationships shown in the drawings, are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific direction, be configured and operated in the specific direction, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that, unless explicitly specified and limited otherwise, the terms "mounted," "provided," "connected," and the like are to be construed broadly, and may be fixedly connected, detachably connected, or integrally connected, for example; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
First embodiment
Fig. 1 is a flow chart showing steps of a forging temperature field distribution and three-dimensional dimension integrated reconstruction method according to a first embodiment of the present invention, where the embodiment is based on binocular vision and infrared thermal imaging technology, and performs integrated reconstruction on forging temperature and dimension.
As shown in fig. 1, the forging temperature field distribution and three-dimensional dimension integrated reconstruction method provided by the embodiment includes the following steps:
s1: three-dimensional reconstruction is carried out on the forge piece through a binocular camera, and the size of the forge piece is measured; as shown in fig. 2, the method specifically comprises the following steps:
s11: calibrating the binocular camera by adopting a Zhang calibration method to obtain internal parameters and external parameters of the left camera and the right camera;
when the Zhang's calibration method is adopted to calibrate the binocular cameras, the two cameras are calibrated respectively, the checkerboard template is used, the two cameras are used for collecting images of the checkerboard template at least two positions, the unity matrix between the template and the images can be obtained by matching points in the collected images and points on the template, and then the distortion coefficient and other internal parameters of the cameras are obtained by using a linear solving method according to the obtained unity matrix.
In addition to calibrating the two cameras separately, the relationship between the positions of the two cameras needs to be obtained. The rotation matrix and translation vector of the left and right cameras are respectively: r is R l 、T l And R is r 、T r Suppose that any spatial point P is at C l Coordinate system, C r The non-homogeneous coordinates in the coordinate system and the world coordinate system are respectively: x is x cl 、x cr 、x w The formula can be found:
elimination of x w The method can obtain the following steps:
the positional relationship of the left and right cameras can be expressed by the following formula:
where R, T represents the rotation matrix and translation vector between the left and right camera coordinate systems, respectively.
That is, the Zhang's calibration method is to calibrate the internal parameters of the left and right cameras obtained by the binocular camera to at least include distortion coefficients, and the external parameters of the left and right cameras obtained by the Zhang's calibration method at least include rotation matrix R and translation vector T.
S12: the cross laser is projected to the surface of the forging to be detected, and an automatic or manual scanning mode can be adopted to control the left camera and the right camera to respectively acquire images of the forging to be detected, so that a first image and a second image are acquired;
s13: and the acquired image is subjected to three-dimensional correction by using a Bouguet algorithm, so that errors caused by unparallel optical axes of cameras in the system are eliminated, and the epipolar constraint is met.
The Bouguet algorithm is a multi-step algorithm for achieving camera pose estimation. The Bouguet algorithm is a step-by-step process, and gradually optimizes camera pose parameters from feature point back projection to an image plane, including rotation, scaling, translation and the like.
The Bouguet algorithm can reserve the common area of the left image and the right image to the greatest extent on the premise of ensuring that the distortion caused by the reprojection of the left image and the right image is minimum.
The method comprises the following specific steps: decomposing the rotation matrix R into left and right rotation matrices R l 、R r And R is l 、R r The inverse matrix is given by:
r is then added with l 、R r And the left camera and the right camera rotate by half to ensure that the optical axes of the left camera and the right camera are parallel to the left imaging plane and the right imaging plane, and then construct a transformation matrix R through a translation vector T rect And (3) enabling the connecting lines of the left camera and the right camera to be parallel to an imaging plane, and finally obtaining final rotation matrixes of the left camera and the right camera respectively, wherein the formula is as follows:
s14: performing preprocessing steps such as binarization, closing operation, refinement treatment and the like on the image; in this step:
when the acquired image is subjected to binarization processing, a scale factor R is introduced, pixel points in the original image are ordered according to the size of an R channel value, the pixel point value of the first R percent is set as 1, and the rest is set as 0;
when the image subjected to binarization processing is subjected to closed operation processing, firstly, expanding the binarization image, filling holes in the image, reconnecting the disconnected image, and then, corroding the image to ensure that the whole image is changed back to the size before expansion;
when the image subjected to the closed-loop processing is subjected to the refinement processing, an algorithm for refining the binary region, such as a Zhang-Suen algorithm and a Guo-Hall algorithm, is used to simplify the original pixel block into a binary image in which single pixels are connected.
S15: positioning the center of the laser cross; in this step, the positioning of the laser cross center includes a linear filtering process and contour detection, wherein,
the linear filtering processing is carried out by designing a filtering operator, and the filtering operator is used for filtering the image, wherein the coordinates of the pixels which are larger than the expected threshold value and smaller than 0 pixel point in the image are the coordinates of the potential laser cross center;
the contour detection is carried out by setting a rectangular area with a potential laser cross center as a center and a side length of b as an interested area, searching a contour in the area, discarding the point if 4 blank areas cannot be found, judging the next coordinate point until the rectangular area meeting the requirement is found, obtaining a contour point set of the 4 blank areas, further obtaining the relative positions of the 4 contours, calculating the shortest distance between the diagonal areas, obtaining two points of the shortest distance, obtaining the midpoint of the two points, and regarding the coordinates of the point as the coordinates of the laser cross center.
S16: and converting the determined laser cross center coordinates into three-dimensional coordinates in a world coordinate system, so as to realize the conversion of the coordinate system and facilitate the point cloud splicing and the three-dimensional reconstruction.
S17: acquiring point cloud information on the surface of the forging piece, splicing the acquired point cloud information, and performing three-dimensional reconstruction; the method specifically comprises the following steps: s171: because the initial positions of the two given relevant frames in the point cloud data are far apart, the acquired point cloud information needs to be roughly spliced based on the Super-4pcs algorithm; s172: finely splicing the acquired point cloud information by using an ICP algorithm, and further improving the accuracy and the splicing speed of the point cloud by adopting a Super-4pcs point cloud information coarse registration algorithm and an ICP refinement algorithm; s173: and then rotating and translating the plurality of pieces of point cloud data to a unified coordinate system so as to facilitate the follow-up three-dimensional reconstruction of the forge piece.
S18: realizing the dimension measurement of the forging by utilizing the three-dimensional reconstruction information;
s2: acquiring a thermal infrared image of the forging piece by using an infrared camera, and realizing temperature measurement of the forging piece; as shown in fig. 3, the method specifically comprises the following steps:
s21: performing radiation calibration on the infrared camera by using a blackbody radiation source, fitting a temperature measurement curve, and correcting a temperature measurement result; the method comprises the following steps:
preheating a blackbody radiation source and an infrared camera to a stable state, placing the infrared camera in a radiation field of the blackbody radiation source, and measuring the relationship between the infrared camera and the surface temperature of the blackbody;
the measurement data is specified to follow a gaussian distribution around the true value, and the calculation formula is as follows:
wherein,
f(V i ) Is a probability density function;
V(k 1 ,k 2 ,b；T i ) Is a temperature measurement function;
k 1 ，k 2 primary and secondary coefficients that are arguments;
b is the curve offset;
ti is the blackbody temperature;
vi is the measurement gray;
sigma is the standard deviation;
e is a natural constant;
constructing a maximum likelihood function, wherein the calculation formula is as follows:
wherein,
l is a likelihood function;
n is a natural number set;
to maximize the maximum likelihood function, the formula is requiredAnd obtaining a minimum value, using a gradient descent method to find a global minimum value fitting temperature measurement curve of the Loss function, and determining a temperature measurement correction function.
S22: performing thermal infrared image enhancement processing to enhance image contrast; the method specifically comprises the following steps:
s221: the image is subjected to gray mapping by a linear mapping method based on local gray level, and the calculation formula is as follows:
wherein,
I out to output an image;
I in is an input image;
delta is an adjustable parameter;
s222: the image is subjected to contrast enhancement processing through a self-adaptive histogram equalization algorithm, and the calculation formula is as follows:
y＝F CLAHE [f CLAHE (x)]
wherein,
y is the final output image;
x is the input infrared original image;
F CLAHE is a global CLAHE function;
f CLAHE is a local CLAHE function;
s23: performing image super-resolution processing to further improve the quality of the infrared image; the specific process is as follows:
the method comprises the steps of performing double resolution reconstruction processing on an input image through convolutional neural network super-resolution image processing, performing bicubic interpolation preprocessing on the input image to reach the target resolution, and performing image block extraction and encoding, nonlinear mapping of encoding vectors and calculation on a high-resolution image.
S3: realizing the integral reconstruction of the temperature and the size of the forging; as shown in fig. 4, the method specifically comprises the following steps:
s31: determining the pose relation of the binocular vision camera and the infrared camera;
s32: transforming the space point from the binocular vision camera coordinate system to the world coordinate system, and then converting the space point to the infrared camera coordinate system; the method comprises the following steps: and transforming the space point P from the binocular vision camera coordinate system to the world coordinate system to obtain P ', and then transforming the space point P into the infrared camera coordinate system to obtain P ', wherein P ' is a new three-dimensional coordinate of the P point under the infrared camera coordinate system after coordinate transformation.
S33: calculating gray values of projection points, and completing solving of the corresponding relation between infrared information and the three-dimensional point cloud; the method comprises the following steps: projecting a point P' to an infrared camera pixel plane to obtain an infrared pixel point (u, v) corresponding to the point P, calculating a gray value of the projected point based on a bilinear interpolation method, and completing solving of the corresponding relation between infrared information and three-dimensional point cloud to realize the integrated reconstruction of the temperature and the size of the forging;
s34: and correcting errors and compensating offset according to the condition that registration offset occurs in the temperature assignment of the three-dimensional points.
In the step, feature points corresponding to the depth image and the infrared image are converted into the same coordinate system, the mean square error of the distance between the feature points of the converted depth image and the feature points of the infrared image is calculated, and the error of registration offset condition of the three-dimensional point temperature assignment is corrected.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (10)
1. A forge piece temperature field distribution and three-dimensional size integrated reconstruction method is characterized by comprising the following steps of: the method comprises the following steps:
s1: three-dimensional reconstruction is carried out on the forge piece through a binocular camera, and the size of the forge piece is measured; comprising the following steps:
s11: calibrating the binocular cameras to obtain internal parameters and external parameters of the left camera and the right camera;
s12: projecting cross laser to the surface of the forging to be detected, and acquiring a first image and a second image of the forging to be detected by a left camera and a right camera;
s13: carrying out three-dimensional correction on the image to enable the image to meet epipolar constraint;
s14: a preprocessing step of binarizing, closing and refining the image;
s15: positioning the center of the laser cross;
s16: converting the cross center coordinates into three-dimensional coordinates in a world coordinate system;
s17: acquiring point cloud information on the surface of the forging piece, splicing the acquired point cloud information, and performing three-dimensional reconstruction;
s18: realizing the dimension measurement of the forging by utilizing the three-dimensional reconstruction information;
s2: acquiring a thermal infrared image of the forging piece by using an infrared camera, and measuring the temperature of the forging piece; comprising the following steps:
s21: performing radiation calibration on the infrared camera by using a blackbody radiation source, fitting a temperature measurement curve, and correcting a temperature measurement result;
s22: performing thermal infrared image enhancement processing to enhance image contrast;
s23: performing image super-resolution processing to further improve the quality of the infrared image;
s3: registering the infrared image and the three-dimensional point cloud in a fusion way, so as to realize the integral reconstruction of the temperature and the size of the forging piece; comprising the following steps:
s31: determining the pose relation of the binocular vision camera and the infrared camera;
s32: transforming the space point from the binocular vision camera coordinate system to the world coordinate system, and then converting the space point to the infrared camera coordinate system;
s33: calculating gray values of projection points, and completing solving of the corresponding relation between infrared information and the three-dimensional point cloud;
s34: and correcting errors and compensating offset according to the condition that registration offset occurs in the temperature assignment of the three-dimensional points.
2. The method for reconstructing the temperature field distribution and the three-dimensional size of the forging according to claim 1, wherein in the step S11, the binocular camera is calibrated, and obtaining the internal parameters and the external parameters of the left and right cameras comprises:
calibrating the binocular camera by adopting a Zhang calibration method;
the acquired internal parameters of the left and right cameras at least comprise distortion coefficients;
the acquired external parameters of the left and right cameras at least comprise a rotation matrix R and a translation vector T.
3. The forging temperature field distribution and three-dimensional size integrated reconstruction method as recited in claim 1, wherein the forging temperature field distribution and three-dimensional size integrated reconstruction method is characterized by comprising the following steps of: in the step S13, a Bouguet algorithm is used to perform stereo correction on the acquired image, so that the acquired image meets epipolar constraint.
4. The forging temperature field distribution and three-dimensional size integrated reconstruction method as recited in claim 1, wherein the forging temperature field distribution and three-dimensional size integrated reconstruction method is characterized by comprising the following steps of: in the step S14, the preprocessing step of performing binarization, closing operation and refinement processing on the image includes:
when the acquired image is subjected to binarization processing, a scale factor R is introduced, pixel points in the original image are ordered according to the size of an R channel value, the pixel point value of the first R percent is set as 1, and the rest is set as 0;
when the image subjected to binarization processing is subjected to closed operation processing, firstly, expanding the binarization image, filling holes in the image, reconnecting the disconnected image, and then, corroding the image to ensure that the whole image is changed back to the size before expansion;
when the image subjected to the closed operation processing is subjected to the refinement processing, an algorithm for refining the binary region is used to simplify the original pixel block into a binary image in which single pixels are connected.
5. The forging temperature field distribution and three-dimensional size integrated reconstruction method as recited in claim 1, wherein the forging temperature field distribution and three-dimensional size integrated reconstruction method is characterized by comprising the following steps of: in the step S15, the positioning of the laser cross center includes linear filtering and contour detection, wherein,
the linear filtering processing is carried out by designing a filtering operator, and the filtering operator is used for filtering the image, wherein the coordinates of the pixels which are larger than the expected threshold value and smaller than 0 pixel point in the image are the coordinates of the potential laser cross center;
the contour detection is carried out by setting a rectangular area with a potential laser cross center as a center and a side length of b as an interested area, searching a contour in the area, discarding the point if 4 blank areas cannot be found, judging the next coordinate point until the rectangular area meeting the requirement is found, obtaining a contour point set of the 4 blank areas, further obtaining the relative positions of the 4 contours, calculating the shortest distance between the diagonal areas, obtaining two points of the shortest distance, obtaining the midpoint of the two points, and regarding the coordinates of the point as the coordinates of the laser cross center.
6. The forging temperature field distribution and three-dimensional size integrated reconstruction method as recited in claim 1, wherein the forging temperature field distribution and three-dimensional size integrated reconstruction method is characterized by comprising the following steps of: in the step S17, obtaining point cloud information of the surface of the forging, splicing the obtained point cloud information, and performing three-dimensional reconstruction includes:
s171: coarsely splicing the acquired point cloud information based on Super-4pcs algorithm;
s172: finely splicing the acquired point cloud information based on an ICP algorithm;
s173: and rotationally translating the plurality of pieces of point cloud data to a unified coordinate system.
7. The forging temperature field distribution and three-dimensional size integrated reconstruction method as recited in claim 1, wherein the forging temperature field distribution and three-dimensional size integrated reconstruction method is characterized by comprising the following steps of: in the step S21, the blackbody radiation source is used to perform radiation calibration on the infrared camera, a temperature measurement curve is fitted, and the correction of the temperature measurement result includes:
preheating a blackbody radiation source and an infrared camera to a stable state, placing the infrared camera in a radiation field of the blackbody radiation source, and measuring the relationship between the infrared camera and the surface temperature of the blackbody;
the measurement data is specified to follow a gaussian distribution around the true value, and the calculation formula is as follows:
wherein,
f(V i ) Is a probability density function;
V(k 1 ,k 2 ,b；T i ) Is a temperature measurement function;
k 1 ，k 2 primary and secondary coefficients that are arguments;
b is the curve offset;
T i is black body temperature;
V i to measure gray scale;
sigma is the standard deviation;
e is a natural constant;
constructing a maximum likelihood function, wherein the calculation formula is as follows:
wherein,
l is a likelihood function;
n is a natural number set;
to maximize the maximum likelihood function, the formula is requiredAnd obtaining a minimum value, using a gradient descent method to find a global minimum value fitting temperature measurement curve of the Loss function, and determining a temperature measurement correction function.
8. The forging temperature field distribution and three-dimensional size integrated reconstruction method as recited in claim 1, wherein the forging temperature field distribution and three-dimensional size integrated reconstruction method is characterized by comprising the following steps of:
in the step S22, performing the thermal infrared image enhancement process to enhance the image contrast includes:
s221: the image is subjected to gray mapping by a linear mapping method based on local gray level, and the calculation formula is as follows:
wherein,
I out to output an image;
I in is an input image;
delta is an adjustable parameter;
s222: the image is subjected to contrast enhancement processing through a self-adaptive histogram equalization algorithm, and the calculation formula is as follows:
y＝F CLAHE [f CLAHE (x)]
wherein,
y is the final output image;
x is the input infrared original image;
F CLAHE is a global CLAHE function;
f CLAHE is a local CLAHE function;
in the step S23, performing image super-resolution processing, further improving the quality of the infrared image includes:
the method comprises the steps of performing double resolution reconstruction processing on an input image through convolutional neural network super-resolution image processing, performing bicubic interpolation preprocessing on the input image to reach the target resolution, and performing image block extraction and encoding, nonlinear mapping of encoding vectors and calculation on a high-resolution image.
9. The forging temperature field distribution and three-dimensional size integrated reconstruction method as recited in claim 1, wherein the forging temperature field distribution and three-dimensional size integrated reconstruction method is characterized by comprising the following steps of: in the step S32, transforming the spatial point from the binocular vision camera coordinate system to the world coordinate system and then to the infrared camera coordinate system includes:
and transforming the space point P from the binocular vision camera coordinate system to the world coordinate system to obtain P ', and then transforming the space point P into the infrared camera coordinate system to obtain P ', wherein P ' is a new three-dimensional coordinate of the P point under the infrared camera coordinate system after coordinate transformation.
10. The forging temperature field distribution and three-dimensional size integrated reconstruction method as recited in claim 9, wherein:
in the step S33, calculating the gray value of the projection point, and completing the solution of the correspondence between the infrared information and the three-dimensional point cloud includes:
projecting a point P' to an infrared camera pixel plane to obtain an infrared pixel point (u, v) corresponding to the point P, calculating a gray value of the projected point based on a bilinear interpolation method, and completing solving of the corresponding relation between infrared information and three-dimensional point cloud to realize the integrated reconstruction of the temperature and the size of the forging;
in the step S34, for the case that the registration offset occurs in the three-dimensional point temperature assignment, the error is corrected, and the offset compensation includes:
and converting the feature points corresponding to the depth image and the infrared image respectively into the same coordinate system, calculating the mean square error of the distance between the feature points of the converted depth image and the feature points of the infrared image, and correcting the error of registration offset condition of three-dimensional point temperature assignment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311670133.5A CN117557731A (en) | 2023-12-07 | 2023-12-07 | Forging temperature field distribution and three-dimensional size integrated reconstruction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311670133.5A CN117557731A (en) | 2023-12-07 | 2023-12-07 | Forging temperature field distribution and three-dimensional size integrated reconstruction method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117557731A true CN117557731A (en) | 2024-02-13 |
Family
ID=89810912
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311670133.5A Pending CN117557731A (en) | 2023-12-07 | 2023-12-07 | Forging temperature field distribution and three-dimensional size integrated reconstruction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117557731A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117740186A (en) * | 2024-02-21 | 2024-03-22 | 微牌科技(浙江)有限公司 | Tunnel equipment temperature detection method and device and computer equipment |
-
2023
- 2023-12-07 CN CN202311670133.5A patent/CN117557731A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117740186A (en) * | 2024-02-21 | 2024-03-22 | 微牌科技(浙江)有限公司 | Tunnel equipment temperature detection method and device and computer equipment |
| CN117740186B (en) * | 2024-02-21 | 2024-05-10 | 微牌科技(浙江)有限公司 | Tunnel equipment temperature detection method and device and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108520537B (en) | Binocular depth acquisition method based on luminosity parallax | |
| CN111784778B (en) | Binocular camera external parameter calibration method and system based on linear solving and nonlinear optimization | |
| CN117557731A (en) | Forging temperature field distribution and three-dimensional size integrated reconstruction method | |
| CN110009690A (en) | Binocular stereo vision image measuring method based on polar curve correction | |
| CN101448065A (en) | Method of correcting image distortion and apparatus for processing image using the method | |
| CN113360964A (en) | Convergence type binocular vision guided robot positioning method under high dynamic range | |
| CN112561807B (en) | End-to-end radial distortion correction method based on convolutional neural network | |
| CN115930828A (en) | Method and device for detecting contour dimension of surface coating of planar plate | |
| CN116740048A (en) | Lithium battery surface defect detection method based on fusion target detection technology | |
| CN114998448A (en) | Method for calibrating multi-constraint binocular fisheye camera and positioning space point | |
| Liu et al. | Convergent binocular vision algorithm for guiding machining robot under extended imaging dynamic range | |
| CN112833821B (en) | Differential geometric three-dimensional micro-vision detection system and method for high-density IC welding spots | |
| CN112017259B (en) | Indoor positioning and image building method based on depth camera and thermal imager | |
| CN110910457B (en) | Multispectral three-dimensional camera external parameter calculation method based on angular point characteristics | |
| CN116067506A (en) | Method and system for constructing three-dimensional temperature field on material surface of industrial furnace | |
| CN112927304B (en) | Fish-eye lens calibration method based on convolutional neural network | |
| Zhu et al. | Distortion correction method of a zoom lens based on the vanishing point geometric constraint | |
| CN116297181A (en) | Method for rapidly and accurately correcting laser scanning galvanometer through calibration plate | |
| CN115375762A (en) | Three-dimensional reconstruction method for power line based on trinocular vision | |
| CN115272569A (en) | Generation method and generation device of deviation and color difference graph of three-dimensional data and electronic equipment | |
| CN114782540B (en) | Three-dimensional reconstruction camera attitude estimation method considering mechanical arm motion constraint | |
| CN112017108A (en) | Satellite ortho-image color relative correction method based on independent model method block adjustment | |
| CN117593309B (en) | Method, system and storage medium for detecting chip welding spots based on stereo reconstruction | |
| Wang et al. | Computer vision methods for automating high temperature steel section sizing in thermal images | |
| CN113674192B (en) | Method, system and device for fusing infrared video image and visible light video image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |