CN111531203B - Online visual detection method and system for depth of aircraft panel countersink - Google Patents
Online visual detection method and system for depth of aircraft panel countersink Download PDFInfo
- Publication number
- CN111531203B CN111531203B CN202010395981.XA CN202010395981A CN111531203B CN 111531203 B CN111531203 B CN 111531203B CN 202010395981 A CN202010395981 A CN 202010395981A CN 111531203 B CN111531203 B CN 111531203B
- Authority
- CN
- China
- Prior art keywords
- dimple
- dimple hole
- depth
- ellipse
- equation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23B—TURNING; BORING
- B23B41/00—Boring or drilling machines or devices specially adapted for particular work; Accessories specially adapted therefor
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23B—TURNING; BORING
- B23B47/00—Constructional features of components specially designed for boring or drilling machines; Accessories therefor
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23Q—DETAILS, COMPONENTS, OR ACCESSORIES FOR MACHINE TOOLS, e.g. ARRANGEMENTS FOR COPYING OR CONTROLLING; MACHINE TOOLS IN GENERAL CHARACTERISED BY THE CONSTRUCTION OF PARTICULAR DETAILS OR COMPONENTS; COMBINATIONS OR ASSOCIATIONS OF METAL-WORKING MACHINES, NOT DIRECTED TO A PARTICULAR RESULT
- B23Q17/00—Arrangements for observing, indicating or measuring on machine tools
- B23Q17/20—Arrangements for observing, indicating or measuring on machine tools for indicating or measuring workpiece characteristics, e.g. contour, dimension, hardness
Abstract
The invention relates to an on-line visual inspection method and system for the depth of a countersink of an aircraft panel, and relates to the field of visual inspection of the depth of the countersink of the aircraft panel. The method comprises the following steps: automatically manufacturing a dimple hole on a wall plate of an airplane; acquiring a first dimple hole image at a first set position; acquiring a second expanding dimple hole image at a second set position; processing the two images to obtain an ellipse equation of the inner edge and the outer edge of the first dimple hole image and an ellipse equation of the inner edge and the outer edge of the second dimple hole image; respectively determining inner and outer ellipse characteristic points of the first dimple hole image and inner and outer ellipse characteristic points of the second dimple hole image according to the ellipse equation; determining the depth of the dimple hole according to the characteristic points; acquiring a set dimple hole depth; and correcting the depth of the dimple hole according to the depth of the dimple hole and the set depth of the dimple hole. The method can improve the accuracy and efficiency of the on-line detection of the depth of the aircraft wallboard countersink.
Description
Technical Field
The invention relates to the field of visual detection of depth of a countersink of an aircraft panel, in particular to an online visual detection method and system for the depth of the countersink of the aircraft panel.
Background
The stealthy nature and the appearance design of aircraft have proposed stricter requirement to the quality of aircraft wallboard surface connection structure, and the system hole counter sink when mainly assembling through the aircraft at present adopts counter sink bolt or rivet to fasten the connection in order to improve surface quality, and the counter sink hole degree of depth deviation can lead to the arch of aircraft wallboard connection structure department or sunken, becomes the important factor that influences aircraft wallboard surface quality and structural strength. At present, automatic assembly equipment is widely applied to aircraft assembly, but the depth of a dimple hole still adopts a rivet or dimple gauge plugging mode, and the development requirements of digitization and intellectualization of an aircraft automatic assembly technology cannot be met. To the above situation, how to realize the non-contact, high-precision and real-time online dimple hole depth detection during the automatic assembly of the airplane is a problem which needs to be solved urgently in the field of airplane assembly.
Disclosure of Invention
The invention aims to provide an on-line visual detection method and system for the depth of a countersink of an aircraft panel, which can improve the accuracy and efficiency of the on-line detection of the depth of the countersink of the aircraft panel.
In order to achieve the purpose, the invention provides the following scheme:
an on-line visual inspection method for depth of a countersink of an aircraft panel comprises the following steps:
automatically manufacturing a dimple hole on a wall plate of an airplane;
acquiring a first dimple hole image at a first set position;
acquiring a second ream socket hole image at a second set position, wherein the second set position is a position which is parallel moved for a set distance on the basis of the first set position;
processing the first dimple hole image and the second dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the first dimple hole image and an ellipse equation of the inner edge and the outer edge of the second dimple hole image;
determining inner and outer elliptical feature points of the first dimple hole image according to an elliptical equation of the inner edge and the outer edge of the first dimple hole image;
determining inner and outer elliptical feature points of the second dimple hole image according to an elliptical equation of the inner edge and the outer edge of the second dimple hole image;
determining the depth of the dimple hole according to the inner and outer elliptical feature points of the first dimple hole image and the inner and outer elliptical feature points of the second dimple hole image;
acquiring a set dimple hole depth;
and correcting the depth of the dimple hole according to the depth of the dimple hole and the set depth of the dimple hole.
Optionally, the processing the first dimple hole image and the second dimple hole image to obtain an ellipse equation of the inner and outer edges of the first dimple hole image and an ellipse equation of the inner and outer edges of the second dimple hole image specifically includes:
graying, denoising, edge detection and ellipse fitting processing are carried out on the first dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the first dimple hole image;
and carrying out graying, denoising, edge detection and ellipse fitting treatment on the second dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the second dimple hole image.
Optionally, the determining, according to the ellipse equation of the inner edge and the outer edge of the first dimple hole image, an inner ellipse feature point and an outer ellipse feature point of the first dimple hole image specifically includes:
acquiring an ellipse equation of the inner edge and the outer edge of the first dimple hole image, wherein the ellipse equation comprises a first inner ellipse equation and a first outer ellipse equation;
making a straight line through the center of the inner ellipse of the first dimple hole image to obtain a first linear equation;
and determining inner and outer ellipse characteristic points of the first dimple hole image according to the first inner ellipse equation, the first outer ellipse equation and the first linear equation.
Optionally, the determining, according to the ellipse equation of the inner edge and the outer edge of the second dimple hole image, an inner ellipse feature point and an outer ellipse feature point of the second dimple hole image specifically includes:
acquiring an ellipse equation of the inner edge and the outer edge of the second dimple hole image, wherein the ellipse equation comprises a second inner ellipse equation and a second outer ellipse equation;
making a straight line through the center of the inner ellipse of the second dimple hole image to obtain a second linear equation;
and determining inner and outer ellipse characteristic points of the second spot-facing hole image according to the second inner ellipse equation, the second outer ellipse equation and the second linear equation.
Optionally, determining the dimple hole depth according to the inner and outer elliptical feature points of the first dimple hole image and the inner and outer elliptical feature points of the second dimple hole image, specifically including:
adopting a formula according to the inner and outer ellipse characteristic points of the first dimple hole image and the inner and outer ellipse characteristic points of the second dimple hole image
Determining the depth of the dimple hole;
wherein HdDepth of dimple hole, f actual focal length of camera, luIs the physical size of the pixel in the transverse direction, d is the translation distance of the camera from the first set position to the second set position, u1iFor the abscissa, u, of the outer elliptical dimple depth feature point of the first dimple hole image1i+1For the abscissa, u, of the feature point of the depth of the elliptical dimple in the first dimple hole image2iFor the abscissa, u, of the outer elliptical dimple depth feature point of the first dimple hole image2i+1And the horizontal coordinate of the feature point of the depth of the oval dimple in the first dimple hole image is shown.
An online visual inspection system of aircraft panel dimple hole depth, includes:
the spot facing hole manufacturing module is used for automatically manufacturing a spot facing hole on a wall plate of an airplane;
the first dimple hole image acquisition module is used for acquiring a first dimple hole image at a first set position;
the second reaming socket hole image acquisition module is used for acquiring a second reaming socket hole image at a second set position, and the second set position is a position which is parallel moved by a set distance on the basis of the first set position;
the image processing module is used for processing the first dimple hole image and the second dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the first dimple hole image and an ellipse equation of the inner edge and the outer edge of the second dimple hole image;
the first characteristic point determining module is used for determining inner and outer elliptical characteristic points of the first dimple hole image according to an elliptical equation of the inner edge and the outer edge of the first dimple hole image;
the second characteristic point determining module is used for determining inner and outer elliptical characteristic points of the second dimple hole image according to an elliptical equation of the inner edge and the outer edge of the second dimple hole image;
the dimple depth determining module is used for determining the dimple depth according to the inner and outer elliptical feature points of the first dimple image and the inner and outer elliptical feature points of the second dimple image;
the set dimple hole depth obtaining module is used for obtaining the set dimple hole depth;
and the dimple hole depth correction module is used for correcting the dimple hole depth according to the dimple hole depth and the set dimple hole depth.
Optionally, the image processing module specifically includes:
the first image processing unit is used for carrying out graying, denoising, edge detection and ellipse fitting processing on the first dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the first dimple hole image;
and the second image processing unit is used for carrying out graying, denoising, edge detection and ellipse fitting processing on the second dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the second dimple hole image.
Optionally, the first feature point determining module specifically includes:
the first ellipse equation determining unit is used for acquiring ellipse equations of the inner edge and the outer edge of the first dimple hole image, and the ellipse equations comprise a first inner ellipse equation and a first outer ellipse equation;
the first linear equation determining unit is used for making a straight line through the center of the inner ellipse of the first dimple hole image to obtain a first linear equation;
and the first characteristic point determining unit is used for determining the inner and outer elliptical characteristic points of the first dimple hole image according to the first inner elliptical equation, the first outer elliptical equation and the first linear equation.
Optionally, the second feature point determining module specifically includes:
the second ellipse equation determining unit is used for acquiring ellipse equations of the inner edge and the outer edge of the second dimple hole image, and the ellipse equations comprise a second inner ellipse equation and a second outer ellipse equation;
the second linear equation determining unit is used for making a straight line through the center of the inner ellipse of the second dimple hole image to obtain a second linear equation;
and the second characteristic point determining unit is used for determining the inner and outer elliptical characteristic points of the second open-pocket hole image according to the second inner elliptical equation, the second outer elliptical equation and the second linear equation.
Optionally, the dimple hole depth determining module specifically includes:
a dimple hole depth determining unit for adopting a formula according to the inner and outer ellipse feature points of the first dimple hole image and the inner and outer ellipse feature points of the second dimple hole image
Determining the depth of the dimple hole;
wherein HdDepth of dimple hole, f actual focal length of camera, luIs the physical size of the pixel in the transverse direction, d is the translation distance of the camera from the first set position to the second set position, u1iFor the abscissa, u, of the outer elliptical dimple depth feature point of the first dimple hole image1i+1For the abscissa, u, of the feature point of the depth of the elliptical dimple in the first dimple hole image2iFor the abscissa, u, of the outer elliptical dimple depth feature point of the first dimple hole image2i+1And the horizontal coordinate of the feature point of the depth of the oval dimple in the first dimple hole image is shown.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the invention can realize the on-line detection of the depth of the aircraft wall panel countersink only by translation shooting, has the characteristics of high efficiency, high precision, low cost and the like, and can be used for realizing the full-automatic operation of manufacturing-detection integration in various automatic assembly equipment.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a flow chart of an on-line visual inspection method for the depth of a countersink of an aircraft panel;
FIG. 2 is a diagram showing the relationship between an end effector and a camera of the automatic hole drilling robot according to the present invention;
FIG. 3 is a schematic diagram of feature point extraction according to the present invention;
FIG. 4 is a schematic diagram of dimple hole depth calculation according to the present invention;
FIG. 5 is a diagram of a system data transmission according to the present invention;
FIG. 6 is a structural diagram of an on-line visual inspection system for the depth of a countersink of an aircraft panel.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide an on-line visual detection method and system for the depth of a countersink of an aircraft panel, which can improve the accuracy and efficiency of the on-line detection of the depth of the countersink of the aircraft panel.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Example 1:
according to the invention, the monocular camera is integrated on the end effector of the automatic assembly equipment, the dimple hole image is shot twice through the translation of the robot, the depth data of the dimple hole of the aircraft panel can be obtained, and the detection data is fed back to the automatic assembly equipment. FIG. 1 is a flow chart of an on-line visual inspection method for the depth of a countersink of an aircraft panel. As shown in fig. 1, an on-line visual inspection method for the depth of a countersink of an aircraft panel comprises the following steps:
step 101: automatically manufacturing the socket hole on the aircraft wall plate. The automatic assembly equipment completes automatic hole-making and spot-facing on an airplane wall plate through an end effector arranged at the tail end of an A6 shaft of the robot.
Step 102: a first dimple hole image is acquired at a first set position. Specifically, the end effector is driven to move to the position 1 according to the offline point position, and the first dimple hole image is shot by the industrial camera integrated on the end effector.
Step 103: and acquiring a second expanding dimple hole image at a second set position, wherein the second set position is a position after the first set position is subjected to parallel movement for a set distance. Specifically, drive end effector parallel translation and set for the distance, the industry camera shoots second expand dimple hole image.
Step 104: processing the first dimple hole image and the second dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the first dimple hole image and an ellipse equation of the inner edge and the outer edge of the second dimple hole image, and the method specifically comprises the following steps:
and carrying out graying, denoising, edge detection and ellipse fitting treatment on the first dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the first dimple hole image.
And carrying out graying, denoising, edge detection and ellipse fitting treatment on the second dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the second dimple hole image.
Step 105: determining inner and outer ellipse feature points of the first dimple hole image according to an ellipse equation of the inner edge and the outer edge of the first dimple hole image, and specifically comprises the following steps:
and acquiring an ellipse equation of the inner edge and the outer edge of the first dimple hole image, wherein the ellipse equation comprises a first inner ellipse equation and a first outer ellipse equation.
And making a straight line through the center of the inner ellipse of the first dimple hole image to obtain a first linear equation.
And determining inner and outer ellipse characteristic points of the first dimple hole image according to the first inner ellipse equation, the first outer ellipse equation and the first linear equation.
Step 106: determining inner and outer ellipse characteristic points of the second dimple hole image according to an ellipse equation of the inner edge and the outer edge of the second dimple hole image, and specifically comprises the following steps:
and acquiring an ellipse equation of the inner edge and the outer edge of the second dimple hole image, wherein the ellipse equation comprises a second inner ellipse equation and a second outer ellipse equation.
And making a straight line through the center of the inner ellipse of the second dimple hole image to obtain a second linear equation.
And determining inner and outer ellipse characteristic points of the second spot-facing hole image according to the second inner ellipse equation, the second outer ellipse equation and the second linear equation.
Intersecting any linear equation passing through the center of the inner ellipse with the equation of the inner ellipse and the outer ellipse in the first dimple hole, establishing length equations of two points where the straight line intersects with the inner ellipse and the outer ellipse on the same side, and respectively setting the two points where the inner ellipse and the outer ellipse intersect with the straight line when the maximum length is obtained as P1i(u1i,v1i) And P1i+1(u1i+1,v1i+1) That is, the characteristic point is the inner and outer ellipse characteristic point of the second dimple hole image is P2i(u2i,v2i) And P2i+1(u2i+1,v2i+1)。
Step 107: determining the depth of the dimple hole according to the inner and outer oval feature points of the first dimple hole image and the inner and outer oval feature points of the second dimple hole image, and specifically comprising the following steps:
adopting a formula according to the inner and outer ellipse characteristic points of the first dimple hole image and the inner and outer ellipse characteristic points of the second dimple hole image
And determining the depth of the dimple hole.
Wherein HdDepth of dimple hole, f actual focal length of camera, luIs the physical size of the pixel in the transverse direction, d is the translation distance of the camera from the first set position to the second set position, u1iFor the abscissa, u, of the outer elliptical dimple depth feature point of the first dimple hole image1i+1For the first dimple hole-facing imageAbscissa, u, of characteristic point of depth of elliptical countersink2iFor the abscissa, u, of the outer elliptical dimple depth feature point of the first dimple hole image2i+1And the horizontal coordinate of the feature point of the depth of the oval dimple in the first dimple hole image is shown.
Step 108: and acquiring the set dimple hole depth.
Step 109: and correcting the depth of the dimple hole according to the depth of the dimple hole and the set depth of the dimple hole.
And feeding the dimple depth data back to a control system of the automation equipment, and correcting the dimple depth by the control system according to the detected dimple depth data and the acquired deviation of the set dimple depth data.
The invention can realize the on-line detection of the depth of the aircraft wallboard countersink only by the translation shooting of the monocular camera, has the characteristics of high efficiency, high precision, low cost and the like, and can be integrated in various automatic assembly devices to realize the full-automatic operation of manufacturing-detection integration.
Example 2:
the embodiment of the invention describes in detail the derivation process of the on-line visual inspection method for the depth of the aircraft panel socket hole and the related calculation method, wherein the on-line visual inspection method for the depth of the aircraft panel socket hole specifically comprises the following steps:
(1) and (5) hole making and dimple forming. The specific process is that the robot moves an end effector fixedly mounted at the tail end of a mechanical arm of the robot to a point position of a to-be-drilled hole of an aircraft panel according to DELMIA pre-planned point position, a pressure angle module presses the aircraft panel, the drilling module feeds a drilling counter sink, and finally the pressure angle module and the drilling module return to initial positions.
(2) An image is captured. The robot moves the camera integrally installed on the end effector, so that the aircraft wallboard countersink hole completed in the previous step is completely and clearly imaged in the camera, the PLC controls the camera to shoot a first image, and the object distance d between the camera and the aircraft wallboard1And according to the feedback control of the laser ranging sensor, the robot moves the distance d in parallel, and the PLC controls the camera to shoot a second image. FIG. 2 is a diagram of the relationship between the end effector and the camera of the automatic hole drilling robot of the present invention, upper levelThe system sends out an instruction through the PLC, so that the robot can shoot the same hole position at different positions by moving for a fixed distance, and then two pictures are obtained. And (4) processing and calculating photos to obtain the depth information of the counter boring of the hole, and feeding back the information to an upper system.
(3) And (5) processing the image. Denoising is carried out firstly, median filtering is carried out during denoising, then bilateral filtering is carried out, effective denoising and smoothing can be carried out on the dimple hole image, then inner and outer edge detection is carried out on the dimple hole image based on a morphological sub-pixel edge detection method, the precision can reach 0.1 pixel, and finally oval fitting is carried out on the contour of the inner edge and the outer edge of the dimple hole based on a least square method.
(4) And (5) calculating the depth of the dimple hole. Fig. 3 is a schematic diagram of feature point extraction according to the present invention. In FIG. 3, OXY is the image plane, C1、C2An ellipse equation is obtained by image processing and fitting the inner edge and the outer edge of the countersink hole, and L is an inner ellipse C2Straight line in the center, P13、P16Is a straight line L and an outer ellipse C1Point of intersection of (P)14、P15Is a straight line L and an ellipse C2The intersection point of (a).
And setting the ellipse equation of the inner edge and the outer edge of the dimple hole of the first image obtained by the image processing in the previous step as follows:
wherein A is11、B11、C11、D11、E11、F11、A22、B22、C22、D22、E22、F22Is an ellipse C1、C2The equation constant parameters.
Cross ellipse C2The straight-line equation for an arbitrary slope k at the center is:
L:v＝ku+(v20-ku20)
wherein k is the slope of the linear equation, u20、v20Is an ellipse C2The horizontal and vertical coordinates of the center point.
Simultaneous straight lineEquation L ellipse C1And C2The following two equation sets can be obtained:
wherein f is1(u) is a linear equation L-elliptic equation C1，f2(u) is a linear equation L-elliptic equation C2。
Let a quadratic equation of one1(u)＝0，f2Solutions with (u) being 0 are each P13(u13,v13)、P16(u16,v16) And P14(u14,v14)、P15(u15,v15) Wherein u is13<u16，u14<u15Oval C1And C2Coefficient of equation and ellipse C2The coordinates of the center point of (a) are known quantities, so that the four solutions are directly related to the slope k of the straight line, and for solving the value of the slope k of the straight line at the maximum value, the following are provided:
if f '(k)' is 0, the straight line L and the ellipse C are obtained1Oval C2K-k when the length of the intersecting segment reaches the maximum valuemaxFurther, when k is equal to k, the value of k can be obtainedmaxTime P13(u13,v13)，P14(u14,v14) And P15(u15,v15)，P16(u16,v16) When the value of | u13-u14|>|u15-u16If | then there are:
in the formula:
P1i(u1i,v1i) -first pictureDimple-like hole outer edge ellipse C1And calculating the edge point of the depth of the upper socket hole, namely the outer oval feature point of the first socket hole image.
P1i+1(u1i+1,v1i+1) -ellipse C of inner edge of dimple hole in first image2And calculating the edge point of the depth of the upper socket hole, namely the inner ellipse feature point of the first socket hole image.
When u13-u14|>|u15-u16If | then there are:
by semi-globalBlockA matching algorithm (SGBM) is used for solving matching characteristic points of the first image and the second image, and the edge point for detecting the depth of the inner hole and the outer hole of the first image is set as P1i(u1i,v1i)，P1i+1(u1i+1,v1i+1) Then, detecting the edge matching point as P at the depth detection edge of the dimple hole corresponding to the second image2i(u2i,v2i)，P2i+1(u2i+1,v2i+1)。
FIG. 4 is a schematic diagram of the calculation of the depth of the countersink. The robot drives two shooting points of the camera at the position 1 and the position 2, the first picture is obtained by shooting at the position 1, the second picture is obtained by shooting at the position 2, and P is1、P2Two arbitrary points of the inner and outer edges of the countersink hole, P11(u11，v11) And P21(u21，v21) Are respectively P1Projection points of points on two images, P12(u12，v12) And P22(u22，v22) Are respectively P2Projection points of points on two images, P1The perpendicular distance from the point to the optical center of the camera is H1，P2The perpendicular distance from the point to the optical center of the camera is H2The actual focal length of the camera is f, and the horizontal physical size of the pixel is luThe robot drives the camera to translate a distance d from position 1 to position 2.
By perspective projection principleAs can be seen, Δ P1P11P′11～ΔP1O11N1、ΔP1P21P′11～ΔP1O12N1。
According to the triangle similarity theorem, the following exist:
the two equations above are combined:
wherein, O11N1、P1N1Is DeltaP1O11N1Side length of (P)1P11、P11P11Is' Δ P1P11P11Length of side of `, O12N1、P1N1Is DeltaP1O12N1Side length of (P)21P11′、P11P11Is' Δ P1P21P11' side length.
The projection relation of the inner and outer edge points of the counter sink hole can be used for knowing O11N1+O12N1＝d，P1N1＝H1，P1P′11＝H1-f，P11P′11+P21P′11＝d+lu(u21-u11) Substituting the formula as follows:
in the formula:
H1——P1the perpendicular distance of the point to the camera optical center in the optical axis direction.
u11——P1Projection point P11The horizontal pixel coordinates of (a).
u21——P1Projection point P21The horizontal pixel coordinates of (a).
P on the outer edge can be calculated by the above formula1Perpendicular distance of point to camera optical center in optical axis direction:
thus, P on the inner edge can be calculated in the same way2Perpendicular distance of point to camera optical center in optical axis direction:
then there is H1And H2Is the difference on the outer edge P1Point and P on inner edge2The depth difference of the points, namely the depth H of the dimple hole is as follows:
replacing the coordinates of the above formula with P1i(u1i,v1i)，P1i+1(u1i+1,v1i+1) And P2i(u2i,v2i)，P2i+1(u2i+1,v2i+1) And finally obtaining a calculation formula of the depth of the aircraft panel countersink:
wherein the actual focal length f of the camera and the physical size l of the pixel in the transverse directionuSolving by a Zhang Zhengyou scaling method.
(5) And (4) online feedback. In the invention, the PLC, the camera, the upper computer, the execution motor driver and the robot are mainly connected into a Profibet bus system in a network port mode and communicate with each other by a TCP/IP protocol. And the PLC sends the photographing instruction to the robot and the camera through the TCP/IP. And after the robot finishes a photographing task according to the displacement instruction and the photographing instruction, the camera transmits the picture to the upper computer through the TCP/IP. The host computer calculates the dimple depth data that one-step calculation obtained according to the photo to carry out the contrast judgement with the dimple result data of calculating and dimple depth standard and required precision, according to different judgement results, specifically have following three kinds of condition:
the first condition is as follows: the depth of the dimple hole meets the requirements of indexes and errors. Under this condition, the host computer is direct for issuing PLC system hole counter sinking instruction next time, and PLC does not change present system hole counter sinking process parameter and assigns the required displacement instruction of system hole counter sinking next time to the robot, assigns the feeding instruction of system hole counter sinking to terminal executive motor, and feed volume and speed are unchangeable, and the main shaft rotational speed is unchangeable yet. And after the hole making and the dimple forming are finished, continuously sending a photographing instruction to the camera, and carrying out online detection and feedback.
Case two: the depth of the countersink is greater than the index and the error requirement. In this case, the upper computer calculates the difference Δ H ═ Hd–HG，HGIs a standard value and is based on the current execution unit feed quantity HnCalculating the next feeding amount Hm＝Hn- Δ H. According to HmThe upper computers are interconnected through an open database, and are connected with the database through codes compiled by a software platform and SQL statements, and similar feeding quantity H is selected in a process parameter databaseoRule is | Ho-HmAnd selecting a smaller value when the | is less than or equal to the delta, wherein the delta is an error requirement. According to HoCorresponding process parameters, such as spindle speed, etc., may also be changed. The upper computer is according to HoAnd after selecting proper process parameters, transmitting the data to the PLC. PLC gives end actuating motor with data transmission, and data are including feeding volume, rotational speed etc. and assign the required displacement instruction of system hole counter sinking next time to the robot simultaneously to assign the instruction of shooing to the camera after the counter sinking is accomplished, the on-line measuring feedback.
Case three: countersinkThe hole depth is less than the index and error requirements. In this case, the upper computer calculates the difference Δ H ═ Hd–HGIn this case, Δ H is a negative value. According to the current execution unit feed amount HnCalculating the next feeding amount Hm＝Hn- Δ H. According to HmThe upper computers are interconnected through an open database, and are connected with the database through codes compiled by a software platform and SQL statements, and similar feeding quantity H is selected in a process parameter databaseoRule is | Ho-HmAnd selecting a larger value if the | is less than or equal to delta. Then, as the case two, the upper computer is according to HoAnd after selecting proper process parameters, transmitting the data to the PLC. PLC gives end actuating motor with data transmission, and data are including feeding volume, rotational speed etc. and assign the required displacement instruction of system hole counter sinking next time to the robot simultaneously to assign the instruction of shooing to the camera after the counter sinking is accomplished, the on-line measuring feedback.
The feedback is carried out by the bottom layer execution unit, the calculation and judgment are carried out by the upper layer management unit, and finally the judgment result is sent to the bottom layer execution unit by a new execution instruction, so that the online feedback and adjustment are completed.
FIG. 5 is a diagram of system data transmission according to the present invention. The camera, the upper computer, the PLC, the robot and the tail end execution unit are connected into a Profibet bus system through network ports, and data are transmitted through a TCP/IP protocol. And the upper computer and the process parameter library exchange data through a common ODBC interface.
FIG. 6 is a structural diagram of an on-line visual inspection system for the depth of a countersink of an aircraft panel. As shown in fig. 6, an on-line visual inspection system for the depth of a countersink of an aircraft panel comprises:
dimple hole making module 201 for automatically making dimple holes on an aircraft panel.
A first dimple hole image acquisition module 202 configured to acquire a first dimple hole image at a first set position.
And the second reaming socket hole image acquisition module 203 is used for acquiring a second reaming socket hole image at a second set position, wherein the second set position is a position after the first set position is subjected to parallel movement for a set distance.
The image processing module 204 is configured to process the first dimple hole image and the second dimple hole image to obtain an ellipse equation of the inner and outer edges of the first dimple hole image and an ellipse equation of the inner and outer edges of the second dimple hole image.
The first feature point determining module 205 is configured to determine inner and outer elliptical feature points of the first dimple hole image according to an ellipse equation of the inner and outer edges of the first dimple hole image.
And a second feature point determining module 206, configured to determine inner and outer elliptical feature points of the second dimple image according to an elliptical equation of the inner and outer edges of the second dimple image.
And the dimple depth determining module 207 is used for determining the dimple depth according to the inner and outer elliptical feature points of the first dimple image and the inner and outer elliptical feature points of the second dimple image.
And a set dimple depth acquisition module 208 for acquiring the set dimple depth.
And the dimple depth correction module 209 is used for correcting the dimple depth according to the dimple depth and the set dimple depth.
The image processing module 204 specifically includes:
and the first image processing unit is used for carrying out graying, denoising, edge detection and ellipse fitting processing on the first dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the first dimple hole image.
And the second image processing unit is used for carrying out graying, denoising, edge detection and ellipse fitting processing on the second dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the second dimple hole image.
The first feature point determining module 205 specifically includes:
the first ellipse equation determining unit is used for obtaining ellipse equations of the inner edge and the outer edge of the first dimple hole image, and the ellipse equations comprise a first inner ellipse equation and a first outer ellipse equation.
And the first linear equation determining unit is used for making a straight line through the center of the inner ellipse of the first dimple hole image to obtain a first linear equation.
And the first characteristic point determining unit is used for determining the inner and outer elliptical characteristic points of the first dimple hole image according to the first inner elliptical equation, the first outer elliptical equation and the first linear equation.
The second feature point determining module 206 specifically includes:
and the second ellipse equation determining unit is used for acquiring ellipse equations of the inner edge and the outer edge of the second dimple hole image, and the ellipse equations comprise a second inner ellipse equation and a second outer ellipse equation.
And the second linear equation determining unit is used for making a straight line through the center of the inner ellipse of the second dimple hole image to obtain a second linear equation.
And the second characteristic point determining unit is used for determining the inner and outer elliptical characteristic points of the second open-pocket hole image according to the second inner elliptical equation, the second outer elliptical equation and the second linear equation.
The dimple hole depth determining module 207 specifically includes:
a dimple hole depth determining unit for adopting a formula according to the inner and outer ellipse feature points of the first dimple hole image and the inner and outer ellipse feature points of the second dimple hole image
And determining the depth of the dimple hole.
Wherein HdDepth of dimple hole, f actual focal length of camera, luIs the physical size of the pixel in the transverse direction, d is the translation distance of the camera from the first set position to the second set position, u1iFor the abscissa, u, of the outer elliptical dimple depth feature point of the first dimple hole image1i+1For the abscissa, u, of the feature point of the depth of the elliptical dimple in the first dimple hole image2iFor the abscissa, u, of the outer elliptical dimple depth feature point of the first dimple hole image2i+1And the horizontal coordinate of the feature point of the depth of the oval dimple in the first dimple hole image is shown.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (8)
1. The utility model provides an online visual inspection method of aircraft wallboard dimple hole degree of depth which characterized in that includes:
automatically manufacturing a dimple hole on a wall plate of an airplane;
acquiring a first dimple hole image at a first set position;
acquiring a second ream socket hole image at a second set position, wherein the second set position is a position which is parallel moved for a set distance on the basis of the first set position;
processing the first dimple hole image and the second dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the first dimple hole image and an ellipse equation of the inner edge and the outer edge of the second dimple hole image;
determining inner and outer elliptical feature points of the first dimple hole image according to an elliptical equation of the inner edge and the outer edge of the first dimple hole image;
determining inner and outer elliptical feature points of the second dimple hole image according to an elliptical equation of the inner edge and the outer edge of the second dimple hole image;
determining the depth of the dimple hole according to the inner and outer elliptical feature points of the first dimple hole image and the inner and outer elliptical feature points of the second dimple hole image;
acquiring a set dimple hole depth;
correcting the depth of the dimple hole according to the depth of the dimple hole and the set depth of the dimple hole;
according to the inner and outer oval feature points of the first dimple hole image and the inner and outer oval feature points of the second dimple hole image, dimple hole depth is determined, and the method specifically comprises the following steps:
adopting a formula according to the inner and outer ellipse characteristic points of the first dimple hole image and the inner and outer ellipse characteristic points of the second dimple hole image
Determining the depth of the dimple hole;
wherein HdDepth of dimple hole, f actual focal length of camera, luIs the physical size of the pixel in the transverse direction, d is the translation distance of the camera from the first set position to the second set position, u1iFor the abscissa, u, of the outer elliptical dimple depth feature point of the first dimple hole image1i+1For the abscissa, u, of the feature point of the depth of the elliptical dimple in the first dimple hole image2iFor the abscissa, u, of the outer elliptical dimple depth feature point of the first dimple hole image2i+1And the horizontal coordinate of the feature point of the depth of the oval dimple in the first dimple hole image is shown.
2. The on-line visual inspection method for the depth of a dimple of an aircraft panel according to claim 1, wherein the processing of the first and second images of the dimple to obtain an ellipse equation of the inner and outer edges of the first image of the dimple and an ellipse equation of the inner and outer edges of the second image of the dimple specifically comprises:
graying, denoising, edge detection and ellipse fitting processing are carried out on the first dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the first dimple hole image;
and carrying out graying, denoising, edge detection and ellipse fitting treatment on the second dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the second dimple hole image.
3. The on-line visual inspection method for the depth of the dimple hole of the aircraft panel according to claim 1, wherein the determining the inner and outer elliptical feature points of the first dimple hole image according to the elliptical equation of the inner and outer edges of the first dimple hole image specifically comprises:
acquiring an ellipse equation of the inner edge and the outer edge of the first dimple hole image, wherein the ellipse equation comprises a first inner ellipse equation and a first outer ellipse equation;
making a straight line through the center of the inner ellipse of the first dimple hole image to obtain a first linear equation;
and determining inner and outer ellipse characteristic points of the first dimple hole image according to the first inner ellipse equation, the first outer ellipse equation and the first linear equation.
4. The on-line visual inspection method for the depth of the countersink of the aircraft panel according to claim 1, wherein the determining of the inner and outer elliptical feature points of the second countersink image according to the elliptical equation of the inner and outer edges of the second countersink image specifically comprises:
acquiring an ellipse equation of the inner edge and the outer edge of the second dimple hole image, wherein the ellipse equation comprises a second inner ellipse equation and a second outer ellipse equation;
making a straight line through the center of the inner ellipse of the second dimple hole image to obtain a second linear equation;
and determining inner and outer ellipse characteristic points of the second spot-facing hole image according to the second inner ellipse equation, the second outer ellipse equation and the second linear equation.
5. The utility model provides an online visual inspection system of aircraft wallboard dimple hole degree of depth which characterized in that includes:
the spot facing hole manufacturing module is used for automatically manufacturing a spot facing hole on a wall plate of an airplane;
the first dimple hole image acquisition module is used for acquiring a first dimple hole image at a first set position;
the second reaming socket hole image acquisition module is used for acquiring a second reaming socket hole image at a second set position, and the second set position is a position which is parallel moved by a set distance on the basis of the first set position;
the image processing module is used for processing the first dimple hole image and the second dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the first dimple hole image and an ellipse equation of the inner edge and the outer edge of the second dimple hole image;
the first characteristic point determining module is used for determining inner and outer elliptical characteristic points of the first dimple hole image according to an elliptical equation of the inner edge and the outer edge of the first dimple hole image;
the second characteristic point determining module is used for determining inner and outer elliptical characteristic points of the second dimple hole image according to an elliptical equation of the inner edge and the outer edge of the second dimple hole image;
the dimple depth determining module is used for determining the dimple depth according to the inner and outer elliptical feature points of the first dimple image and the inner and outer elliptical feature points of the second dimple image;
the set dimple hole depth obtaining module is used for obtaining the set dimple hole depth;
the dimple hole depth correction module is used for correcting the dimple hole depth according to the dimple hole depth and the set dimple hole depth;
the dimple hole depth determination module specifically comprises:
a dimple hole depth determining unit for adopting a formula according to the inner and outer ellipse feature points of the first dimple hole image and the inner and outer ellipse feature points of the second dimple hole image
Determining the depth of the dimple hole;
wherein HdDepth of dimple hole, f actual focal length of camera, luIs the physical size of the pixel in the transverse direction, d is the translation distance of the camera from the first set position to the second set position, u1iFor the abscissa, u, of the outer elliptical dimple depth feature point of the first dimple hole image1i+1For the first dimple hole image inner ellipseAbscissa of dimple depth characteristic point, u2iFor the abscissa, u, of the outer elliptical dimple depth feature point of the first dimple hole image2i+1And the horizontal coordinate of the feature point of the depth of the oval dimple in the first dimple hole image is shown.
6. The on-line visual inspection system for the depth of the countersink of the aircraft panel according to claim 5, wherein the image processing module specifically comprises:
the first image processing unit is used for carrying out graying, denoising, edge detection and ellipse fitting processing on the first dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the first dimple hole image;
and the second image processing unit is used for carrying out graying, denoising, edge detection and ellipse fitting processing on the second dimple hole image to obtain an ellipse equation of the inner edge and the outer edge of the second dimple hole image.
7. The on-line visual inspection system for the depth of the countersink of the aircraft panel as claimed in claim 5, wherein the first feature point determining module specifically comprises:
the first ellipse equation determining unit is used for acquiring ellipse equations of the inner edge and the outer edge of the first dimple hole image, and the ellipse equations comprise a first inner ellipse equation and a first outer ellipse equation;
the first linear equation determining unit is used for making a straight line through the center of the inner ellipse of the first dimple hole image to obtain a first linear equation;
and the first characteristic point determining unit is used for determining the inner and outer elliptical characteristic points of the first dimple hole image according to the first inner elliptical equation, the first outer elliptical equation and the first linear equation.
8. The on-line visual inspection system for the depth of the countersink of the aircraft panel as claimed in claim 5, wherein the second feature point determining module specifically comprises:
the second ellipse equation determining unit is used for acquiring ellipse equations of the inner edge and the outer edge of the second dimple hole image, and the ellipse equations comprise a second inner ellipse equation and a second outer ellipse equation;
the second linear equation determining unit is used for making a straight line through the center of the inner ellipse of the second dimple hole image to obtain a second linear equation;
and the second characteristic point determining unit is used for determining the inner and outer elliptical characteristic points of the second open-pocket hole image according to the second inner elliptical equation, the second outer elliptical equation and the second linear equation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010395981.XA CN111531203B (en) | 2020-05-12 | 2020-05-12 | Online visual detection method and system for depth of aircraft panel countersink |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010395981.XA CN111531203B (en) | 2020-05-12 | 2020-05-12 | Online visual detection method and system for depth of aircraft panel countersink |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111531203A CN111531203A (en) | 2020-08-14 |
| CN111531203B true CN111531203B (en) | 2021-03-05 |
Family
ID=71968086
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010395981.XA Active CN111531203B (en) | 2020-05-12 | 2020-05-12 | Online visual detection method and system for depth of aircraft panel countersink |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111531203B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113369990B (en) * | 2021-07-06 | 2022-05-10 | 成都飞机工业（集团）有限责任公司 | On-line detection device for non-contact measuring hole and use method thereof |
| CN113945169A (en) * | 2021-10-21 | 2022-01-18 | 西北工业大学 | System and method for measuring key sizes of straight holes and nest holes |
| CN114193231B (en) * | 2022-02-16 | 2022-06-14 | 成都飞机工业（集团）有限责任公司 | Bottom hole orifice measuring method for numerical control countersink |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003014657A (en) * | 2001-07-03 | 2003-01-15 | Dainippon Printing Co Ltd | Method and apparatus for inspecting cyclic pattern |
| CN107192340A (en) * | 2017-07-26 | 2017-09-22 | 广东理工学院 | A kind of NI Vision Builder for Automated Inspection of measured hole, the position of groove and physical dimension |
| CN109741314A (en) * | 2018-12-29 | 2019-05-10 | 广州博通信息技术有限公司 | A kind of visible detection method and system of part |
| CN110076350B (en) * | 2019-04-30 | 2020-05-29 | 北京航空航天大学 | Ultrasonic stamping type cutting and extruding integrated processing method |
| CN111006608B (en) * | 2019-12-17 | 2021-04-27 | 湖南科创信息技术股份有限公司 | Transparent plate through hole visual detection system and method |
-
2020
- 2020-05-12 CN CN202010395981.XA patent/CN111531203B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111531203A (en) | 2020-08-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111531203B (en) | Online visual detection method and system for depth of aircraft panel countersink | |
| CN111775146B (en) | Visual alignment method under industrial mechanical arm multi-station operation | |
| US11225018B2 (en) | 3D printing method and device with multi-axis mechanical system and visual surveillance | |
| CN108326850B (en) | Method and system for robot to accurately move mechanical arm to reach specified position | |
| CN107588721A (en) | The measuring method and system of a kind of more sizes of part based on binocular vision | |
| CN107218930B (en) | Monocular-hand-eye-system-based active measurement method for six-dimensional position-posture of space circle | |
| JPH08210816A (en) | Coordinate system connection method for determining relationship between sensor coordinate system and robot tip part in robot-visual sensor system | |
| CN104819707A (en) | Polyhedral active cursor target | |
| CN109341559A (en) | A kind of aero-engine casing deformation measurement method based on Binocular Vision Principle | |
| CN112658643B (en) | Connector assembly method | |
| CN109940606B (en) | Robot guiding system and method based on point cloud data | |
| CN111912335A (en) | Airplane surface datum hole identification method suitable for robot drilling and riveting system | |
| CN112362034B (en) | Solid engine multi-cylinder section butt joint guiding measurement method based on binocular vision | |
| CN105436252A (en) | Bending machine machining control method and device based on vision measurement | |
| US20220080597A1 (en) | Device and method for calibrating coordinate system of 3d camera and robotic arm | |
| CN113516716A (en) | Monocular vision pose measuring and adjusting method and system | |
| CN114001651B (en) | Large-scale slender barrel type component pose in-situ measurement method based on binocular vision measurement and priori detection data | |
| CN112958960B (en) | Robot hand-eye calibration device based on optical target | |
| CN109940604B (en) | Workpiece three-dimensional positioning system and method based on point cloud data | |
| CN110992416A (en) | High-reflection-surface metal part pose measurement method based on binocular vision and CAD model | |
| CN110475627A (en) | Deformation processing auxiliary system and deformation processing householder method | |
| CN209459599U (en) | A kind of aero-engine casing deformation measuring device based on Binocular Vision Principle | |
| CN111768383A (en) | Three-dimensional target and method for recovering working function of visual sensor by using same | |
| Zhu et al. | Development of a monocular vision system for robotic drilling | |
| JPH06243236A (en) | Setting device for coordinate system correction parameter of visual recognizer |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |