CN111145154A - Machine vision-based serial steel wire anti-loosening structure detection method - Google Patents

Machine vision-based serial steel wire anti-loosening structure detection method Download PDF

Info

Publication number
CN111145154A
CN111145154A CN201911361531.2A CN201911361531A CN111145154A CN 111145154 A CN111145154 A CN 111145154A CN 201911361531 A CN201911361531 A CN 201911361531A CN 111145154 A CN111145154 A CN 111145154A
Authority
CN
China
Prior art keywords
image
theta
max
line segments
steel wire
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911361531.2A
Other languages
Chinese (zh)
Other versions
CN111145154B (en
Inventor
孙惠斌
王静
王展
李宽宽
宋屹桐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northwestern Polytechnical University
Original Assignee
Northwestern Polytechnical University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northwestern Polytechnical University filed Critical Northwestern Polytechnical University
Priority to CN201911361531.2A priority Critical patent/CN111145154B/en
Publication of CN111145154A publication Critical patent/CN111145154A/en
Application granted granted Critical
Publication of CN111145154B publication Critical patent/CN111145154B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/20—Administration of product repair or maintenance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Economics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a machine vision-based method for detecting a serial steel wire anti-loose structure of an aircraft engine. The invention improves the detection efficiency of the anti-loose structure of the series steel wire on the engine, replaces manpower with a machine, and improves the intelligent level of the detection of the anti-loose structure of the series steel wire in the assembly of the engine.

Description

Machine vision-based serial steel wire anti-loosening structure detection method
Technical Field
The invention relates to the field of assembly and maintenance of aero-engines, in particular to a detection method of an aero-engine series steel wire anti-loose structure based on machine vision.
Background
In order to ensure that the connection structure of the aero-engine is reliable and safe, a plurality of anti-loosening structures with series steel wires are required to be installed on the aero-engine to ensure the tightness of threads, so the steel wires are also called as fuses, and the structure is shown as a schematic diagram 1. During the flying process of the airplane, the threaded fasteners such as bolts tend to loosen due to the influence of factors such as vibration of an engine and an airplane body, temperature change and the like. Therefore, in addition to the tightening torque requirements for some fasteners on an aircraft, it is necessary to add tandem wires to prevent loosening.
At present, the installation process of the steel wire anti-loose structure of the aircraft engine requires four checks, namely checking whether threaded fasteners are neglected for installation, checking whether steel wires among the fasteners are wound, checking whether the winding direction of series steel wires is consistent with the screwing direction of threads, and checking whether the deflection of the steel wires exceeds the standard (is loosened). In the traditional method, technicians carry out manual detection, the risks of neglected loading and wrong loading exist, and the manual detection efficiency is relatively low especially for large aircraft engines.
The machine vision technology is applied to various fields, the core of the machine vision technology is image preprocessing and target feature extraction, and the methods are divided into two categories, namely machine learning and traditional algorithms. The machine learning method has strong adaptability and recognition accuracy under the support of sufficient effective image data, and has the prominent defects that the interpretability is poor, and a large amount of effective image data is difficult to obtain. In addition, the traditional method has definite calculation logic and better detection precision under definite characteristics and scenes, and has the defects of complex algorithm and poor multi-scene adaptability. In combination with the background, the traditional method has high accuracy for a specific scene due to strong interpretability, does not need a large amount of data support, and has applicability in bolt and series steel wire detection.
Disclosure of Invention
The invention provides a steel wire anti-loose structure detection technology based on machine vision, aiming at the defects of the existing steel wire anti-loose structure detection method, through an image preprocessing method, the interference of scratches, light spots, shadows and edge profiles is eliminated, a fastening bolt and a steel wire are identified, whether the steel wire is neglected to be installed or not is checked, the consistency of the winding direction and the screwing direction of a series steel wire is judged, and whether the steel wire is tensioned or not is judged. The invention improves the detection efficiency of the anti-loose structure of the series steel wire on the engine, replaces manpower with a machine, and improves the intelligent level of the detection of the anti-loose structure of the series steel wire in the assembly of the engine.
The technical scheme of the invention is as follows:
the serial steel wire anti-loosening structure detection method based on machine vision is characterized by comprising the following steps: the method comprises the following steps:
step 1: and (3) checking neglected installation of the fastening bolt:
acquiring an anti-loosening structure image of the series steel wire, converting the image into a gray image, and performing reverse binarization processing on the gray image according to a set threshold value to obtain a binary image; setting an inner core area to perform morphological closed operation on the binary image, then performing corrosion and expansion processing to eliminate scratch influence, performing Hough circle detection on the bolt, and storing pixel coordinate data of the circle center into a dynamic array FasteningCenters; comparing the number of circles obtained by Hough circle detection with the number of fastening bolts required in the series steel wire anti-loosening structure corresponding to the image, and judging whether the bolt fasteners are neglected to be installed in the series steel wire anti-loosening structure corresponding to the image; after determining that the circle obtained by Hough circle detection corresponds to the fastening bolt required in the serial steel wire anti-loosening structure corresponding to the image one by one, entering step 2;
step 2: and (3) steel wire neglected loading inspection:
introducing the gray level image obtained in the step 1, performing Gaussian filtering processing, then performing reverse binary and morphological closed operation processing to obtain a binary image with connected high-value points, and then performing corrosion processing, wherein the number of corrosion processing iterations is less than that in the step 1;
calling the FasteningCenters dynamic arrays in the step 1, and respectively extracting the most significant values U of the circle center coordinates on the U axis and the V axis of the image coordinate system by comparing the pixel coordinate values of the circle center point of the Hough circlemin，umax，vmin，vmaxCombining the two maximum values to obtain A (u)min,vmin)、B(umin,vmax)、C(umax,vmax) And D (u)max,vmin) Four points, connecting the four ABCD points in sequence by line segments to form a rectangular bounding box, then expanding each boundary of the bounding box to completely surround the bolt, wherein the vertex coordinate is A' (u)min-l,vmin-l)，B′(umin-l,vmax+l)，C′(umax+l,vmax+l)，D′(umax+l,vmin-l)；
Taking rectangular outlines which are sequentially connected with four vertexes A 'B' C 'D' as boundaries, introducing the outlines into a mask function, covering areas outside the rectangles A 'B' C 'D', using Canny operators to extract edges of the rectangular inner outlines, then executing Hough line detection, and storing detected line segment endpoint coordinates into LinesPoints dynamic arrays;
traversing all line segment data in the LinesPoints dynamic array, and deleting redundant line segments according to the following two rules:
rule 1: calculating the inclination angles corresponding to all the line segment slopes by using the pixel coordinates of the line segment end points, and when the difference between the inclination angles of the two line segments is in the range
Figure BDA0002337276690000031
Considering the two line segments as the line segments with similar inclination angles, and entering the rule 2 to continue judging;
the way in which the tilt angle is calculated from the slope is:
the slope k is in the range of (-infinity, + ∞) and the inverse tangent of the slope k is calculated in the range of
Figure BDA0002337276690000032
The range of the tilt angle is [0,. pi.), so when arctank > 0, θ is arctank, when arctank < 0, take
Figure BDA0002337276690000033
Making the inclination angle correspond to theta; for the limit case, when
Figure BDA0002337276690000034
When the range of the inclination angle close to theta is defined as
Figure BDA0002337276690000035
When in use
Figure BDA0002337276690000036
Then, the tilt angle (theta, pi) ∪ [0, pi-theta) is defined to be close to theta;
rule 2: calculating the distance between the end points on the left side of the line segments with the similar inclination angles determined in the rule 1, setting a threshold value M, and if the distance exceeds the threshold value, determining that the two steel wires are two steel wires, but the pixel inclination angles are similar; if the distance is less than M, determining that redundant line segments exist and only keeping one of the redundant line segments;
comparing the number of the remaining line segments after the redundant line segments are deleted with the number of steel wires to be installed required in the serial steel wire anti-loosening structure corresponding to the image, and judging whether the steel wires are neglected to be installed or not; and storing the coordinates of the end points of the rest line segments into a ValidLinesP dynamic array; after determining that the residual line segments correspond to the steel wires to be additionally installed required in the serial steel wire anti-loosening structure corresponding to the image one by one, entering step 3;
and step 3: judging the consistency of the winding direction of the series steel wires and the screwing direction of the bolts:
traversing pixel coordinates of center points in fasteningcenter, connecting different center points pairwise by using nested circulation, then performing redundancy removing operation on the obtained line segments by using a rule 1 and a rule 2 to obtain the connected line segments with the same number as the number of steel wires to be added required in the anti-loosening structure of the series steel wires corresponding to the image, calculating the inclination angles of the connected line segments and storing the inclination angles into a Bolt _ theta dynamic array; simultaneously calculating the inclination angle of the line segment in the ValidLinesP and storing the inclination angle into a Fuse _ theta array;
for a certain tilt angle theta in Bolt _ thetaBFinding the tilt angle theta close to the Fuse _ theta arrayFThen, the following rules are adopted for judgment:
rule 3: thetaF＞θBOr is or
Figure BDA0002337276690000037
And is
Figure BDA0002337276690000038
Simultaneously, the two steps are carried out;
rule 4: read out θ separatelyF,θBCorresponding line segment endpoint coordinates F1(uF1,vF1)，F2(uF2,vF2) And B1(uB1,vB1)，B2(uB2,vB2) Let endpoint F1F2And B1B2Respectively substituting the linear equations to obtain a two-point linear equation about the variables (u, v):
Figure BDA0002337276690000041
solving a system of equations of a first order of two for (u, v), in the interval of the variables u (u)F1,uF2) The internal equation set has a solution;
if a set of thetaF,θBIf the rule 3 and the rule 4 are met simultaneously, judging that the connecting direction of the series steel wires is consistent with the bolt tightening direction, otherwise, judging that the connecting direction of the series steel wires is inconsistent;
and 4, step 4: judging whether the deflection of the series steel wires exceeds the standard or not:
for a certain tilt angle theta in Bolt _ thetaBObtaining the coordinates P of two end points of the corresponding connected line segment1(uP1,vP1) And P2(uP2,vP2) And rotating the binary image in the step 1 by-thetaBTo make
Figure BDA0002337276690000042
Keeping the horizontal in the image coordinate system and obtaining the coordinate P after rotation1′(u′P1,v′P)，P′2(u′P2,v′P)；
Get umin＝min{u′P1,u′P2}，umax＝max{u′P1,u′P2And taking the vertex coordinate of the rectangular mask as M1(umin+W,v′P-H)，M2(umin+W,v′P+H)，M3(umax-W,v′P+H)，M4(umax-W,v′P-H) performing rectangular edge mask processing on the rotated binary image, wherein W and H are set offset amounts;
traversing all pixel points in the binary image mask, and carrying out comparison on all high-value pixel points Qi(ui,vi) And then a least square regression line fitting is carried out,the straight line equation au + bv + c is 0, and the distances of all the high-value pixel points relative to the regression line are calculated:
Figure BDA0002337276690000043
to diTaking the mean value to obtain
Figure BDA0002337276690000044
And will be
Figure BDA0002337276690000045
Comparing with the set deflection standard exceeding threshold S if
Figure BDA0002337276690000046
Judging that the tandem wire is not tensioned, if
Figure BDA0002337276690000047
The tandem wire is determined to be in a tensioned state.
Further preferred scheme, said a series connection steel wire anti loosening structure detection method based on machine vision, its characterized in that: in step 2, the number of iterations of the etching treatment is smaller than that in step 1
Figure BDA0002337276690000048
Advantageous effects
The invention provides a machine vision-based method for detecting a serial steel wire anti-loose structure of an aircraft engine, which improves the detection efficiency and the intelligent level through an image processing method, can effectively prevent the serial steel wire from being neglected and wrongly installed, and ensures the safety of the aircraft engine.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1: a schematic diagram of a serial steel wire anti-loose structure;
FIG. 2: step 1, a bolt fastener neglected loading inspection flow chart;
FIG. 3: step 2, a serial steel wire neglected loading inspection flow chart;
FIG. 4: step 3, a series steel wire misloading inspection flow chart;
FIG. 5: the relation between the bolt center connecting line and the steel wire is shown schematically;
FIG. 6: step 4, a flow chart of detecting the deflection of the series steel wires;
FIG. 7: step 4, setting a schematic diagram of parameters of a mask boundary W and a mask boundary H;
FIG. 8: a schematic diagram of an original captured image;
FIG. 9: a bolt fastener center detection schematic diagram;
FIG. 10: the binary image schematic diagram is obtained after edge mask processing;
FIG. 11: generating a redundant line segment schematic diagram by Hough line detection;
FIG. 12: simplifying the schematic diagram of the treated Hough line;
FIG. 13: a schematic diagram of a winding direction misloading inspection result;
FIG. 14: schematic diagram of identification results of forward and reverse installation of the series steel wires;
FIG. 15: the characteristic schematic diagram of local gray scale amplification of the steel wire after the mask is covered;
FIG. 16: a binary image high-value point least square fitting straight line schematic diagram;
FIG. 17: schematic diagram for judging the deflection of the series steel wires.
Detailed Description
The invention provides a steel wire anti-loosening structure detection method based on machine vision, aiming at the defects of the existing steel wire anti-loosening structure detection method, which comprises the following steps:
step 1: and (3) checking neglected installation of the fastening bolt, wherein the flow is shown in FIG. 2:
image data of the serial steel wire anti-loosening structure acquired by the industrial camera are numbered in sequence, and the numbering process is automatically generated by a program according to the shooting sequence and stored in a storage device;
the collected image is imported into an image processing program by a storage device, and is converted into a gray image, and the gray image is subjected to reverse binarization processing according to a set threshold value;
setting an inner core area to perform morphological closed operation on the binary image, then performing corrosion and expansion treatment to eliminate scratch influence, performing Hough circle detection on the bolt, storing pixel coordinate data of a circle center into a dynamic array named as FasteningCenters, and performing circular ring marking through a circle center pixel point;
and comparing the number obtained by detecting the Hough circle with the number corresponding to the numbering process requirement, and judging whether the bolt fastener is neglected to be installed at the corresponding position of the image by the program.
Step 2: and (3) checking the neglected loading of the steel wire, wherein the flow is shown in figure 3:
and (3) introducing the gray level image obtained in the step 1, and performing Gaussian filtering processing, wherein the resolution of the camera image for experiment is 6240 × 4160, and the resolution is higher, so that the filtering kernel Size adopted here is (5,5), and if the filtering kernel is too small, the filtering effect is not obvious, and certain influence is caused on detection.
Then, reverse binary and morphological closed operation processing is carried out to obtain a binary image with high-value point communication, then corrosion processing is carried out, and since the pixel characteristics of the steel wire are slender and easy to corrode compared with the bolt, in order to keep the serial steel wire outline of the collected image, the iteration times are fewer than that in the step 1
Figure BDA0002337276690000061
And (4) etching treatment.
Calling the FasteningCenters dynamic arrays in the step 1, and respectively extracting the most significant values U of the circle center coordinates on the U axis and the V axis of the image coordinate system by comparing the pixel coordinate values of the circle center point of the Hough circlemin，umax，vmin，vmaxUnder an image coordinate system, an upper left corner point of an image is a coordinate origin, a horizontal axis is a U axis, a vertical axis is a V axis, and coordinates on the axes are positive integers and represent pixel positions.
Combining the two maximum values to obtain A (u)min,vmin)、B(umin,vmax)、C(umax,vmax) AndD(umax,vmin) Four points, connecting the four ABCD points in sequence by line segments to form a rectangular bounding box, then expanding each boundary of the bounding box to completely surround the bolt, wherein the vertex coordinate is A' (u)min-l,vmin-l)，B′(umin-l,vmax+l)，C′(umax+l,vmax+l)，D′(umax+l,vmin-l)。
Taking the rectangular contour with the four vertexes A 'B' C 'D' connected in sequence as a boundary, introducing the contour into a mask function, wherein the mask function is as follows: traversing all pixel point coordinates of the image, and when the pixel points are positioned inside or on a rectangular boundary, keeping pixel point color information, or else, removing the pixel point color information; covering the area outside the rectangle A 'B' C 'D' by a mask function, extracting the edge of the inner contour of the rectangle by using a Canny operator, then executing Hough line detection, and storing the detected line segment endpoint coordinates into a LinesPoints dynamic array.
Because the Hough line detection is used for generating the redundant line segment, all line segment data in the LinesPoints dynamic array are traversed, and the redundant line segment is deleted according to the following two rules:
rule 1: and eliminating principles with similar inclination angles. Calculating the inclination angles corresponding to all the line segment slopes by using the pixel coordinates of the line segment end points, and when the difference between the inclination angles of the two line segments is in the range
Figure BDA0002337276690000071
Considering the two line segments as the line segments with similar inclination angles, and entering the rule 2 to continue judging;
the way in which the tilt angle is calculated from the slope is:
the slope k is in the range of (-infinity, + ∞) and the inverse tangent of the slope k is calculated in the range of
Figure BDA0002337276690000072
The range of the tilt angle is [0,. pi.), so when arctank > 0, θ is arctank, when arctank < 0, take
Figure BDA0002337276690000073
Make the slopeThe angle corresponds to theta; for the limit case, when
Figure BDA0002337276690000074
When the range of the inclination angle close to theta is defined as
Figure BDA0002337276690000075
When in use
Figure BDA0002337276690000076
Then, the tilt angle (theta, pi) ∪ [0, pi-theta) is defined to be close to theta;
rule 2: and (5) an endpoint-close rejection principle. Calculating the distance between the end points on the left side of the line segments with the similar inclination angles determined in the rule 1, setting a threshold value M, and if the distance exceeds the threshold value, determining that the two steel wires are two steel wires, but the pixel inclination angles are similar; if the distance is less than M, determining that redundant line segments exist and only keeping one of the redundant line segments;
traversing all line segment data in the LinesPoints dynamic array, comparing the number of the remaining line segments of which the redundant line segments are deleted by the two judgment rules with the number of steel wires to be additionally arranged required in the serial steel wire anti-loosening structure corresponding to the image, and judging whether the steel wires are neglected to be arranged; and storing the coordinates of the end points of the rest line segments into a ValidLinesP dynamic array; and (3) determining that the residual line segments correspond to the additionally-installed steel wires required in the series steel wire anti-loosening structure corresponding to the image one by one, and then entering the step (3).
And step 3: judging the consistency of the winding direction of the series steel wires and the screwing direction of the bolts:
traversing pixel coordinates of center points in fasteningcenter, connecting different center points pairwise by using nested circulation, then performing redundancy removing operation on the obtained line segments by using a rule 1 and a rule 2 to obtain the connected line segments with the same number as the number of steel wires to be added required in the anti-loosening structure of the series steel wires corresponding to the image, calculating the inclination angles of the connected line segments and storing the inclination angles into a Bolt _ theta dynamic array; simultaneously calculating the inclination angle of the line segment in the ValidLinesP and storing the inclination angle into a Fuse _ theta array;
for a certain tilt angle theta in Bolt _ thetaBFinding the tilt angle theta close to the Fuse _ theta arrayFThen using the following ruleAnd (4) line judgment:
rule 3: thetaF＞θBOr is or
Figure BDA0002337276690000081
And is
Figure BDA0002337276690000082
Simultaneously, the two steps are carried out;
rule 4: read out θ separatelyF,θBCorresponding line segment endpoint coordinates F1(uF1,vF1)，F2(uF2,vF2) And B1(uB1,vB1)，B2(uB2,vB2) Let endpoint F1F2And B1B2Respectively substituting the linear equations to obtain a two-point linear equation about the variables (u, v):
Figure BDA0002337276690000083
solving a system of equations of a first order of two for (u, v), in the interval of the variables u (u)F1,uF2) The internal equation set has a solution;
if a set of thetaF,θBIf the rule 3 and the rule 4 are met simultaneously, the connection direction of the series steel wires is judged to be consistent with the bolt tightening direction, and the program marks a plus sign at the center of the bolt connecting line; otherwise, the connection direction is wrong, and the program marks a negative sign at the center of the bolt connecting line.
And 4, step 4: judging whether the deflection of the series steel wires exceeds the standard or not:
for a certain tilt angle theta in Bolt _ thetaBObtaining the coordinates P of two end points of the corresponding connected line segment1(uP1,vP1) And P2(uP2,vP2) And rotating the binary image in the step 1 by-thetaBTo make
Figure BDA0002337276690000084
Keeping the horizontal in the image coordinate system and obtaining the coordinate P after rotation1′(u′P1,v′P)，P2′(u′P2,v′P)。
Get umin＝min{u′P1,u′P2}，umax＝max{u′P1,u′P2And taking the vertex coordinate of the rectangular mask as M1(umin+W,v′P-H)，M2(umin+W,v′P+H)，M3(umax-W,v′P+H)，M4(umax-W,v′P-H) performing rectangular edge mask processing on the rotated binary image, wherein W and H are set offset amounts.
Traversing all pixel points in the binary image mask, and carrying out comparison on all high-value pixel points Qi(ui,vi) And performing least square regression line fitting to obtain a line equation au + bv + c which is 0, and calculating the distance between all high-value pixel points and the regression line:
Figure BDA0002337276690000091
to diTaking the mean value to obtain
Figure BDA0002337276690000092
And will be
Figure BDA0002337276690000093
Comparing with the set deflection standard exceeding threshold S if
Figure BDA0002337276690000094
Judging that the tandem wire is not tensioned, if
Figure BDA0002337276690000095
The tandem wire is determined to be in a tensioned state.
Embodiments of the present invention will be described in detail below with reference to fig. 6-16, which are exemplary and intended to be illustrative of the invention and should not be construed as limiting the invention.
This embodiment is described by taking a group of bolt fasteners wound around tandem steel wires as an example, and as shown in fig. 8, the results of the first 3 steps are demonstrated, wherein there are 5 bolt fasteners, and the 5 tandem steel wires are connected two by two to form a pentagon, and obvious scratches and partial shadows can be observed in the pentagon, and alternatively, steps 3 and 4 are additionally described by using fig. 13 and 16, respectively.
Step 1, detecting missing bolt installation:
in the image acquisition stage, the resolution of an image acquired by an industrial camera is 6240 multiplied by 4160, and the diameter of a bolt pixel is kept between 100 and 500 pixels. According to the flow shown in fig. 2, the filtering is converted into a single-channel gray image, the threshold value threshold of inverse binarization is 80, the kernel of closed operation is a pixel matrix of 50 × 50, and the corrosion degree just completely corrodes the steel wire contour and the scratch for high-value point corrosion and expansion respectively is 30.
The result of the Hough circle detection is shown in FIG. 8, and the center of the circle is offset from the center of the bolt but within the allowable range. And if the number of Hough circles detected by the corresponding numbered position images does not accord with the known number, sending a prompt.
Step 2, detecting missing of the series steel wire:
according to the pixel coordinates of the 5 central points detected in the step 1, the maximum value and the minimum value on each axis are searched for the coordinate components on the U axis and the V axis in a traversing manner, l is set to be 200, four vertexes of the mask are obtained, so that the edge interference is removed, and then the binary image is copied to the internal area of the mask, as shown in fig. 9, the mask can cover the edge interference caused by illumination and shooting position conditions.
The masked binary image was subjected to morphological close operation with a kernel of 40 × 40, and slightly eroded with an erosion iteration number iterative of 10.
Edge detection is carried out by using Canny operator, parameters are set to be threshold 1 ═ 150 and threshold2 ═ 500, then Hough line detection is carried out, and accumulator distance resolution rho ═ 0.5 and accumulator angle resolution are set
Figure BDA0002337276690000101
The accumulator threshold is 100, the minimum line length is 1000, and the connection point is maximumThe gap is maxLineGap of 200, and the detection result is as shown in fig. 10, and a large number of redundant line segments are detected in the detection result, and 227 lines are detected.
The redundant line segments are judged by using the rules 1 and 2, the threshold value M in the rule 2 is set to be 500, and the simplified Hough line segments correspond to the steel wires one by one as shown in fig. 11.
And (4) checking the detection result of the series steel wires and the position due number, and outputting the information without missing installation.
Step 3, checking the winding direction of the series steel wires:
and importing the 5 Hough circle center points identified in the step 1 and the end point data of the Hough line screened in the step 2. The 5 circle centers are connected pairwise to generate 10 connecting lines, and redundancy exists.
Calculating the inclination angles of circle center connecting lines and Hough lines by using a rule 1, and then screening 5 circle center connecting lines associated with the Hough lines according to the rule 1 and the rule 2, wherein each circle center connecting line and the corresponding Hough line are compiled into a group.
According to the illustration in fig. 5, whether the position relationship between each group of Hough lines and the circle center connecting line is correct is judged according to the rules 3 and 4, if the judgment is correct, the program marks a positive sign at the center of the circle center connecting line, and as a result, the connection relationship between 5 groups is correct as illustrated in fig. 12. In addition, the control group shown in FIG. 13 was connected in reverse on the left and in forward on the right.
And outputting information corresponding to the positive and negative connection of the picture according to the measured result, and sending out reminding information if the picture is wrongly installed.
Step 4, detecting the deflection of the series steel wire:
and (4) importing circle center connecting lines related to the serial anti-loosening steel wires in the step (3) and an inclination angle theta corresponding to the circle center connecting lines. To obtain the circle center of P'1(u′P1,v′P) And P'2(u′P2,v′P) For example, let W be 200 pixels and H be 100 pixels, and obtain pixel coordinate values of four vertices, and mask-process the pixel coordinate values to obtain the local enlarged feature shown in fig. 14. Wherein, the left image mask vertex pixel coordinate is: (1923,2203), (1923,2403), (4077,2403), (4077,2203); the coordinates of the vertices of the mask pixels of the right image are: (2213,2061),(2213,2261),(4001,2261),(4001,2061).
For the image binary processing in the mask, traverse all high-value points in the search area by row and column, and then fit a straight line by the least square method, as shown in fig. 15.
Calculating the average deviation of all high-value points relative to the straight line by using a point-to-straight line distance formula, and setting a threshold value S to be 20, wherein the average deviation on the left side is 6.157068, the deviation is within the threshold value, and the deviation is in a tensioning state; the right mean deviation is 46.205240, exceeding the threshold, and is relaxed.
The program marks the image according to the judgment as shown in fig. 16, and sends out a prompt according to the relaxation information, and the content contains the value of which the average deviation exceeds the standard.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made in the above embodiments by those of ordinary skill in the art without departing from the principle and spirit of the present invention.

Claims (2)

1. A serial steel wire anti-loosening structure detection method based on machine vision is characterized in that: the method comprises the following steps:
step 1: and (3) checking neglected installation of the fastening bolt:
acquiring an anti-loosening structure image of the series steel wire, converting the image into a gray image, and performing reverse binarization processing on the gray image according to a set threshold value to obtain a binary image; setting an inner core area to perform morphological closed operation on the binary image, then performing corrosion and expansion processing to eliminate scratch influence, performing Hough circle detection on the bolt, and storing pixel coordinate data of the circle center into a dynamic array FasteningCenters; comparing the number of circles obtained by Hough circle detection with the number of fastening bolts required in the series steel wire anti-loosening structure corresponding to the image, and judging whether the bolt fasteners are neglected to be installed in the series steel wire anti-loosening structure corresponding to the image; after determining that the circle obtained by Hough circle detection corresponds to the fastening bolt required in the serial steel wire anti-loosening structure corresponding to the image one by one, entering step 2;
step 2: and (3) steel wire neglected loading inspection:
introducing the gray level image obtained in the step 1, performing Gaussian filtering processing, then performing reverse binary and morphological closed operation processing to obtain a binary image with connected high-value points, and then performing corrosion processing, wherein the number of corrosion processing iterations is less than that in the step 1;
calling the FasteningCenters dynamic arrays in the step 1, and respectively extracting the most significant values U of the circle center coordinates on the U axis and the V axis of the image coordinate system by comparing the pixel coordinate values of the circle center point of the Hough circlemin，umax，vmin，vmaxCombining the two maximum values to obtain A (u)min,vmin)、B(umin,vmax)、C(umax,vmax) And D (u)max,vmin) Four points, connecting the four ABCD points in sequence by line segments to form a rectangular bounding box, then expanding each boundary of the bounding box to completely surround the bolt, wherein the vertex coordinate is A' (u)min-l,vmin-l)，B′(umin-l,vmax+l)，C′(umax+l,vmax+l)，D′(umax+l,vmin-l)；
Taking rectangular outlines which are sequentially connected with four vertexes A 'B' C 'D' as boundaries, introducing the outlines into a mask function, covering areas outside the rectangles A 'B' C 'D', using Canny operators to extract edges of the rectangular inner outlines, then executing Hough line detection, and storing detected line segment endpoint coordinates into LinesPoints dynamic arrays;
traversing all line segment data in the LinesPoints dynamic array, and deleting redundant line segments according to the following two rules:
rule 1: calculating the inclination angles corresponding to all the line segment slopes by using the pixel coordinates of the line segment end points, and when the difference between the inclination angles of the two line segments is in the range
Figure FDA0002337276680000011
Considering the two line segments as the line segments with similar inclination angles, and entering the rule 2 to continue judging;
the way in which the tilt angle is calculated from the slope is:
the slope k ranges from (— infinity, + ∞) and is invertedTangent calculation in the range of
Figure FDA0002337276680000021
The range of the tilt angle is [0,. pi.), so when arctank > 0, θ is arctank, when arctank < 0, take
Figure FDA0002337276680000022
Making the inclination angle correspond to theta; for the limit case, when
Figure FDA0002337276680000023
When the range of the inclination angle close to theta is defined as
Figure FDA0002337276680000024
When in use
Figure FDA0002337276680000025
Then, the tilt angle (theta, pi) ∪ [0, pi-theta) is defined to be close to theta;
rule 2: calculating the distance between the end points on the left side of the line segments with the similar inclination angles determined in the rule 1, setting a threshold value M, and if the distance exceeds the threshold value, determining that the two steel wires are two steel wires, but the pixel inclination angles are similar; if the distance is less than M, determining that redundant line segments exist and only keeping one of the redundant line segments;
comparing the number of the remaining line segments after the redundant line segments are deleted with the number of steel wires to be installed required in the serial steel wire anti-loosening structure corresponding to the image, and judging whether the steel wires are neglected to be installed or not; and storing the coordinates of the end points of the rest line segments into a ValidLinesP dynamic array; after determining that the residual line segments correspond to the steel wires to be additionally installed required in the serial steel wire anti-loosening structure corresponding to the image one by one, entering step 3;
and step 3: judging the consistency of the winding direction of the series steel wires and the screwing direction of the bolts:
traversing pixel coordinates of center points in fasteningcenter, connecting different center points pairwise by using nested circulation, then performing redundancy removing operation on the obtained line segments by using a rule 1 and a rule 2 to obtain the connected line segments with the same number as the number of steel wires to be added required in the anti-loosening structure of the series steel wires corresponding to the image, calculating the inclination angles of the connected line segments and storing the inclination angles into a Bolt _ theta dynamic array; simultaneously calculating the inclination angle of the line segment in the ValidLinesP and storing the inclination angle into a Fuse _ theta array;
for a certain tilt angle theta in Bolt _ thetaBFinding the tilt angle theta close to the Fuse _ theta arrayFThen, the following rules are adopted for judgment:
rule 3: thetaF＞θBOr is or
Figure FDA0002337276680000026
And is
Figure FDA0002337276680000027
Simultaneously, the two steps are carried out;
rule 4: read out θ separatelyF,θBCorresponding line segment endpoint coordinates F1(uF1,vF1)，F2(uF2,vF2) And B1(uB1,vB1)，B2(uB2,vB2) Let endpoint F1F2And B1B2Respectively substituting the linear equations to obtain a two-point linear equation about the variables (u, v):
Figure FDA0002337276680000031
solving a system of equations of a first order of two for (u, v), in the interval of the variables u (u)F1,uF2) The internal equation set has a solution;
if a set of thetaF,θBIf the rule 3 and the rule 4 are met simultaneously, judging that the connecting direction of the series steel wires is consistent with the bolt tightening direction, otherwise, judging that the connecting direction of the series steel wires is inconsistent;
and 4, step 4: judging whether the deflection of the series steel wires exceeds the standard or not:
for a certain tilt angle theta in Bolt _ thetaBObtaining the coordinates P of two end points of the corresponding connected line segment1(uP1,vP1) And P2(uP2,vP2) And combining the binary image in the step 1Rotation-thetaBTo make
Figure FDA0002337276680000032
Keeping the image horizontal in the image coordinate system, and obtaining the coordinate after rotation as P'1(u′P1,v′P)，P′2(u′P2,v′P)；
Get umin＝min{u′P1,u′P2}，umax＝max{u′P1,u′P2And taking the vertex coordinate of the rectangular mask as M1(umin+W,v′P-H)，M2(umin+W,v′P+H)，M3(umax-W,v′P+H)，M4(umax-W,v′P-H) performing rectangular edge mask processing on the rotated binary image, wherein W and H are set offset amounts;
traversing all pixel points in the binary image mask, and carrying out comparison on all high-value pixel points Qi(ui,vi) And performing least square regression line fitting to obtain a line equation au + bv + c which is 0, and calculating the distance between all high-value pixel points and the regression line:
Figure FDA0002337276680000033
to diTaking the mean value to obtain
Figure FDA0002337276680000034
And will be
Figure FDA0002337276680000035
Comparing with the set deflection standard exceeding threshold S if
Figure FDA0002337276680000036
Judging that the tandem wire is not tensioned, if
Figure FDA0002337276680000037
Judging the steel in seriesThe wire is under tension.
2. The machine vision-based serial steel wire anti-loosening structure detection method as claimed in claim 1, wherein: in step 2, the number of iterations of the etching treatment is smaller than that in step 1
Figure FDA0002337276680000038
CN201911361531.2A 2019-12-25 2019-12-25 Machine vision-based serial steel wire anti-loosening structure detection method Active CN111145154B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911361531.2A CN111145154B (en) 2019-12-25 2019-12-25 Machine vision-based serial steel wire anti-loosening structure detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911361531.2A CN111145154B (en) 2019-12-25 2019-12-25 Machine vision-based serial steel wire anti-loosening structure detection method

Publications (2)

Publication Number Publication Date
CN111145154A true CN111145154A (en) 2020-05-12
CN111145154B CN111145154B (en) 2022-04-01

Family

ID=70520363

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911361531.2A Active CN111145154B (en) 2019-12-25 2019-12-25 Machine vision-based serial steel wire anti-loosening structure detection method

Country Status (1)

Country Link
CN (1) CN111145154B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113469966A (en) * 2021-06-25 2021-10-01 西南交通大学 Train bolt looseness detection method based on anti-loosening line identification

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160231253A1 (en) * 2011-05-17 2016-08-11 Gii Acquisition, Llc Dba General Inspection, Llc Method and system for optically inspecting parts
CN106340019A (en) * 2016-08-31 2017-01-18 西南交通大学 Method for detecting adverse state of high-speed rail overhead line system inclined cable fixing hook
CN106372667A (en) * 2016-08-31 2017-02-01 西南交通大学 Method for detecting adverse state of inclined sleeve part screws of high-speed train overhead line system
CN106570857A (en) * 2016-09-11 2017-04-19 西南交通大学 High-speed railway overhead contact system lateral conductor fixation hook nut falling bad state detection method based on HOG features
CN109141232A (en) * 2018-08-07 2019-01-04 常州好迪机械有限公司 A kind of circle plate casting online test method based on machine vision
CN109389566A (en) * 2018-10-19 2019-02-26 辽宁奇辉电子系统工程有限公司 Subway height adjusting valve fastening nut defective mode detection method based on boundary characteristic
CN110097536A (en) * 2019-04-10 2019-08-06 东南大学 Hexagon bolt looseness detection method based on deep learning and Hough transformation
CN110276747A (en) * 2019-06-06 2019-09-24 中南大学 A kind of insulator breakdown detection and failure ranking method based on image analysis

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160231253A1 (en) * 2011-05-17 2016-08-11 Gii Acquisition, Llc Dba General Inspection, Llc Method and system for optically inspecting parts
CN106340019A (en) * 2016-08-31 2017-01-18 西南交通大学 Method for detecting adverse state of high-speed rail overhead line system inclined cable fixing hook
CN106372667A (en) * 2016-08-31 2017-02-01 西南交通大学 Method for detecting adverse state of inclined sleeve part screws of high-speed train overhead line system
CN106570857A (en) * 2016-09-11 2017-04-19 西南交通大学 High-speed railway overhead contact system lateral conductor fixation hook nut falling bad state detection method based on HOG features
CN109141232A (en) * 2018-08-07 2019-01-04 常州好迪机械有限公司 A kind of circle plate casting online test method based on machine vision
CN109389566A (en) * 2018-10-19 2019-02-26 辽宁奇辉电子系统工程有限公司 Subway height adjusting valve fastening nut defective mode detection method based on boundary characteristic
CN110097536A (en) * 2019-04-10 2019-08-06 东南大学 Hexagon bolt looseness detection method based on deep learning and Hough transformation
CN110276747A (en) * 2019-06-06 2019-09-24 中南大学 A kind of insulator breakdown detection and failure ranking method based on image analysis

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
JAE-HYUNG PARK 等: "Novel bolt-loosening detection technique using image processing for bolt joints in steel bridges", 《ADVANCES IN STRUCTURAL ENGINEERING AND MECHANICS》 *
JAE-HYUNG PARK 等: "Vision-based technique for bolt-loosening detection in wind turbine tower", 《WIND AND STRUCTURES》 *
YOUNG-JIN CHA 等: "Vision-based detection of loosened bolts using the Hough transform and support vector machines", 《AUTOMATION IN CONSTRUCTION》 *
袁云峰: "在线螺栓拧紧设备的检测和评定方法", 《柴油机设计与制造》 *
黄潜 等: "基于三维扫描的机车走行部螺栓识别与定位", 《光电工程》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113469966A (en) * 2021-06-25 2021-10-01 西南交通大学 Train bolt looseness detection method based on anti-loosening line identification
CN113469966B (en) * 2021-06-25 2023-04-18 西南交通大学 Train bolt looseness detection method based on anti-loosening line identification

Also Published As

Publication number Publication date
CN111145154B (en) 2022-04-01

Similar Documents

Publication Publication Date Title
CN106934803B (en) method and device for detecting surface defects of electronic device
CN108369650B (en) Method for identifying possible characteristic points of calibration pattern
CN115082464B (en) Method and system for identifying weld data in welding process of dust remover
US20210027475A1 (en) Vision-based fastener loosening detection
CN108491498B (en) Bayonet image target searching method based on multi-feature detection
CN111369516A (en) Transformer bushing heating defect detection method based on infrared image recognition
CN112750107A (en) Spherical workpiece identification method and system based on contour edge number fitting and convex hull processing
CN108827979A (en) A kind of module group lens appearance detecting method
CN113469971B (en) Image matching method, detection device and storage medium
CN111145154B (en) Machine vision-based serial steel wire anti-loosening structure detection method
US20220230301A1 (en) Manufacturing Method And Image Processing Method and System For Quality Inspection Of Objects Of A Manufacturing Method
CN113516619B (en) Product surface flaw identification method based on image processing technology
US11508052B2 (en) Systems and methods for quantifying light flares in images
CN114024503A (en) Solar cell color separation and defect detection system and method thereof
CN116883446A (en) Real-time monitoring system for grinding degree of vehicle-mounted camera lens
CN112288680B (en) Automatic defect area extraction method and system for automobile hub X-ray image
CN117333696A (en) Bolt detection method and device based on machine vision
CN116664512A (en) VEH connector assembly safety monitoring alarm system based on visual identification
CN114266748B (en) Method and device for judging surface integrity of process board in field of rail traffic overhaul
CN113902667B (en) Thread turning identification method and system for machine vision
CN115880481A (en) Curve positioning algorithm and system based on edge profile
CN115619796A (en) Method and device for obtaining photovoltaic module template and nonvolatile storage medium
CN114359414A (en) Lens smudge identification method and device, computer equipment and storage medium
CN114399479A (en) Detection method and device for bolt cotter pin of railway vehicle and computer equipment
Kim et al. Camera module defect detection using gabor filter and convolutional neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant