CN117593515A - Bolt loosening detection system and method for railway vehicle and storage medium - Google Patents
Bolt loosening detection system and method for railway vehicle and storage medium Download PDFInfo
- Publication number
- CN117593515A CN117593515A CN202410068789.8A CN202410068789A CN117593515A CN 117593515 A CN117593515 A CN 117593515A CN 202410068789 A CN202410068789 A CN 202410068789A CN 117593515 A CN117593515 A CN 117593515A
- Authority
- CN
- China
- Prior art keywords
- bolt
- image
- mark line
- bolts
- representing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 41
- 238000003860 storage Methods 0.000 title claims abstract description 9
- 238000000034 method Methods 0.000 title claims description 26
- 230000011218 segmentation Effects 0.000 claims description 45
- 238000004364 calculation method Methods 0.000 claims description 20
- 238000007781 pre-processing Methods 0.000 claims description 19
- 238000004458 analytical method Methods 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 17
- 230000003993 interaction Effects 0.000 claims description 14
- 238000013075 data extraction Methods 0.000 claims description 13
- 238000004519 manufacturing process Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 claims description 8
- 239000003550 marker Substances 0.000 claims description 7
- 238000005286 illumination Methods 0.000 claims description 4
- 238000005520 cutting process Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 208000003464 asthenopia Diseases 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/20—Administration of product repair or maintenance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Medical Informatics (AREA)
- Entrepreneurship & Innovation (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- Databases & Information Systems (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of railway track detection, and discloses a bolt loosening detection system, a bolt loosening detection method and a storage medium for a railway vehicle.
Description
Technical Field
The invention relates to the technical field of railway track detection, in particular to a bolt loosening detection system and method for a railway vehicle and a storage medium.
Background
Bolts play an important role in rail vehicles such as motor train units, urban rails, subways and the like, and are used for fastening various components, however, during operation, the bolts are easy to cause problems including deformation, loosening, breakage and even falling off due to the effects of various factors such as corrosion, vibration and impact, and the problems can lead to equipment failure, so that an important process in the security inspection of trains in the rail transportation industry is to detect loosening of the bolts.
Typically, there are hundreds or even more bolts on each car in a train, and many trains need maintenance every day, so detecting bolt looseness is a very heavy task. The traditional bolt loosening detection method is a manual inspection method, and mainly comprises two methods: one method is a regular tightening method, in which a worker regularly tightens bolts using a wrench, whether they have been loosened or not, to ensure that all bolts remain tightened; the other method is a marking method, after the first tightening, workers can mark the tightening state on the bolts and nuts respectively, lines are usually used for marking, in the subsequent checking, the workers only need to check whether the marking lines on the bolts and the nuts are aligned or not, and then whether the bolts are loosened or not is judged, but the regular tightening method needs the workers to spend a great deal of time and effort, has huge workload and low efficiency, lacks pertinence, and the marking method reduces unnecessary tightening work, but is easy to cause visual fatigue of the workers, and particularly when a large number of bolts with various types and different sizes need to be checked, the workers are easy to fatigue, and the tiny problem is difficult to find under the conditions of insufficient illumination and fatigue and potential safety hazards possibly happen.
Disclosure of Invention
In order to overcome the above-mentioned drawbacks of the prior art, the present invention provides a bolt loosening detection system and method for a railway vehicle and a storage medium, so as to solve the above-mentioned problems in the prior art.
The invention provides the following technical scheme: a bolt looseness detection system for a railway vehicle comprises an image acquisition module, a template matching module, an image processing module, a data extraction module, a contour detection module, a looseness analysis module and an output interaction module;
the image acquisition module comprises a marking unit and an acquisition unit, wherein the marking unit is used for drawing bolt loosening marking lines on bolts, the acquisition unit is used for acquiring all images of the part to be detected of the vehicle through image acquisition equipment, marking all bolts to be detected and manufacturing a template diagram;
the template matching module acquires an image containing a bolt and a mark line by using a camera and a light source device, performs template matching operation on the acquired image and the template map, and cuts out the image containing the bolt and the mark line;
the image processing module comprises an image preprocessing unit and an image semantic segmentation unit, wherein the image preprocessing unit performs image preprocessing on an image containing bolts and mark lines, and the image semantic segmentation unit performs bolt mark line semantic segmentation on the image subjected to the image preprocessing to obtain an image which is equal to the image subjected to the image preprocessing and contains only mark lines in size and channel number;
the data extraction module is used for extracting characteristic data of the image which only comprises the mark line and is obtained after the semantic segmentation of the bolt mark line in the image processing module, wherein the characteristic data comprise bolt offset parameters, bolt loosening parameters and bolt deformation parameters;
the contour detection module is used for receiving the characteristic data of the image which only contains the mark line and is extracted by the data extraction module, calculating the mark line contour index, and transmitting the calculated mark line contour index to the looseness analysis module;
the loosening analysis module is used for receiving the marking line profile index calculated by the profile detection module, comparing the marking line profile index with a preset threshold value, counting the number of marking line profiles with marking line profile indexes smaller than the preset threshold value, and judging whether the bolt is loosened;
the output interaction module is used for outputting the loosening bolt image detected by the loosening analysis module to the man-machine interaction end.
Preferably, the template matching operation in the template matching module adopts a normalized cross-correlation method, and the matching degree is determined by establishing cross-correlation coefficients of a coordinate system calculation template and each position in the image, wherein the calculation formula is as follows:wherein->Representing the cross-correlation coefficients of the template with various locations in the image,covariance representing coordinates of each position in the template and the image,/->Representing the variance of the coordinates of each position in the template, +.>And (5) representing the variance of each position coordinate in the image, wherein the position corresponding to the maximum cross correlation coefficient is the matching position.
Preferably, the image preprocessing operation in the image processing module includes that the image including the bolt and the mark line is proportionally adjusted to be 80 pixels in size, semantic segmentation of the bolt mark line is performed by making a semantic segmentation data set and building a semantic segmentation network, and then training out a semantic segmentation model of the bolt mark line, and the specific process is as follows: bolt mark line semantic segmentation data set manufacturing: under different illumination conditions, bolt images of all shapes and specifications on a train and with mark lines are acquired at various shooting angles, labelme software is used for manufacturing a semantic segmentation data set by taking the mark lines as the foreground and the rest as the background, and a semantic segmentation network is trained through the semantic segmentation data set to obtain a mark line semantic segmentation model.
Preferably, the bolt deflection parameters in the data extraction module comprise deflection amount, diameter and pretightening force of the bolt, the bolt loosening parameters comprise current length, initial length and elastic restoring force of the bolt, and the bolt deformation parameters comprise maximum bearing capacity, rigidity and cross-sectional area of the bolt.
Preferably, the calculating of the mark line profile index in the profile detection module includes the following steps:
step S01: marking the marking line images of all the bolts as 1, 2 and 3 … … n in sequence, calculating the contour indexes of the marking lines in sequence, directly applying force to the bolts by using a tension measuring instrument, measuring the applied force, and determining the pretightening force of the bolts, the elastic restoring force of the bolts and the rigidity of the bolts;
step S02: and calculating bolt offset coefficients of all bolts based on the bolt offset parameters, wherein the calculation formula is as follows:wherein->Representing the bolt offset coefficient of the respective bolts, +.>Indicating the offset of each bolt->Error values representing the respective bolt offsets +.>Representing the bolt diameter of the respective bolt, < > and->Representing the bolt pretightening force of each bolt;
step S03: bolt loosening coefficients of all bolts are calculated based on the bolt loosening parameters, and the calculation formula is as follows:wherein->Representing the bolt loosening coefficient of the respective bolts, +.>Indicating the current length of each bolt,representing the initial length of the respective bolt, < > and->Representing the elastic restoring force of the bolt;
step S04: and calculating the bolt deformation coefficient of each bolt based on the bolt deformation parameters, wherein the calculation formula is as follows:wherein->Representing the deformation coefficient of the individual bolts, +.>Indicating the maximum bearing capacity of the individual bolts, +.>Representing the bolt stiffness of the individual bolts,/>Representing the cross-sectional area of each bolt;
step S05: and calculating the outline index of each bolt mark line based on the bolt offset coefficient, the bolt loosening coefficient and the bolt deformation coefficient, wherein the calculation formula is as follows:wherein->Marking line profile index for each bolt, +.>Representing the bolt offset coefficient of the respective bolts, +.>Representing the bolt loosening coefficient of the respective bolts, +.>Representing the deformation coefficient of the individual bolts, +.>、/>And->Representing the corresponding weights of the coefficients.
Preferably, the loosening analysis module compares the marking line profile index with a preset threshold value, counts the number N of marking line profiles with marking line profile indexes smaller than the preset threshold value, if N is smaller than 2, indicates that only one marking line is provided or no marking line is detected, judges that the bolt is not loosened, and if N is greater than or equal to 2, judges whether the bolt is loosened according to the center line number M.
A bolt looseness detection method for a railway vehicle comprises the following steps:
step S11: the template diagram is manufactured through a marking unit and an acquisition unit: the marking unit is used for drawing a bolt loosening marking line on the bolt, the acquisition unit is used for acquiring all images of the part to be detected of the vehicle through the image acquisition equipment, and all bolts to be detected are marked;
step S12: performing template matching on the acquired image and a template map: acquiring an image containing a bolt and a mark line by using a camera and a light source device, performing template matching operation on the acquired image and the template map, and cutting out the image containing the bolt and the mark line;
step S13: the image which is equal to the image after the image pretreatment and only contains the mark line is obtained through the processing and the segmentation;
step S14: extracting feature data of an image containing only the marker lines: extracting feature data of an image only containing the mark line, which is obtained after semantic segmentation of the bolt mark line;
step S15: calculating a mark line profile index: receiving characteristic data of an image only containing the mark line, and calculating a mark line contour index;
step S16: judging whether the bolt is loosened according to the outline index of the mark line;
step S17: and outputting the image of the loose bolt as a judging result to the man-machine interaction end.
A non-transitory computer-readable storage medium storing computer instructions that cause the computer to execute a bolt looseness detection system for a rail vehicle of any of the above.
The invention has the technical effects and advantages that: the template image is manufactured through the marking unit and the collecting unit by the image collecting module, the template matching module performs template matching operation on the obtained image and the template image, the image containing bolts and the marking lines is cut out, the image processing module obtains the image which is equal to the image subjected to image pretreatment and only contains the marking lines, the data extracting module extracts characteristic data of the image, the contour detecting module establishes a model to calculate the contour index of the marking lines, the loosening analyzing module compares the contour index of the marking lines with a preset threshold value, the contour index of the marking lines is counted to be smaller than the number N of the marking lines of the preset threshold value, if N is smaller than 2, only one marking line or no marking line is detected, whether the bolts are loosened is judged, if N is larger than or equal to 2, whether the bolts are loosened is judged according to the number M of the center lines, the loosening bolt image detected by the loosening analyzing module is output to the man-machine interaction end, the state of the bolts can be monitored in real time, the condition of the bolts can be found and alarmed in time, the detection accuracy is improved, and measures are taken to avoid accidents and faults.
Drawings
Fig. 1 is a flowchart of a bolt looseness detection system for a rail vehicle.
Fig. 2 is a flowchart of a bolt loosening detection method for a railway vehicle.
Detailed Description
The embodiments of the present invention will be described more fully with reference to the drawings in the present invention, and the configurations of the structures described in the following embodiments are merely examples, and the bolt loosening detection system, method and storage medium for a railway vehicle according to the present invention are not limited to the structures described in the following embodiments, but all other embodiments obtained by a person skilled in the art without making any creative effort are within the scope of the present invention.
As shown in fig. 1, the invention provides a bolt looseness detection system for a railway vehicle, which comprises an image acquisition module, a template matching module, an image processing module, a data extraction module, a contour detection module, a looseness analysis module and an output interaction module;
the image acquisition module comprises a marking unit and an acquisition unit, wherein the marking unit is used for drawing bolt loosening marking lines on bolts, the acquisition unit is used for acquiring all images of the part to be detected of the vehicle through image acquisition equipment, marking all bolts to be detected and manufacturing a template diagram;
the template matching module acquires an image containing a bolt and a mark line by using a camera and a light source device, performs template matching operation on the acquired image and the template map, and cuts out the image containing the bolt and the mark line;
the image processing module comprises an image preprocessing unit and an image semantic segmentation unit, wherein the image preprocessing unit performs image preprocessing on an image containing bolts and mark lines, and the image semantic segmentation unit performs bolt mark line semantic segmentation on the image subjected to the image preprocessing to obtain an image which is equal to the image subjected to the image preprocessing and contains only mark lines in size and channel number;
the data extraction module is used for extracting characteristic data of the image which only comprises the mark line and is obtained after the semantic segmentation of the bolt mark line in the image processing module, wherein the characteristic data comprise bolt offset parameters, bolt loosening parameters and bolt deformation parameters;
the contour detection module is used for receiving the characteristic data of the image which only contains the mark line and is extracted by the data extraction module, calculating the mark line contour index, and transmitting the calculated mark line contour index to the looseness analysis module;
the loosening analysis module is used for receiving the marking line profile index calculated by the profile detection module, comparing the marking line profile index with a preset threshold value, counting the number of marking line profiles with marking line profile indexes smaller than the preset threshold value, and judging whether the bolt is loosened;
the output interaction module is used for outputting the loosening bolt image detected by the loosening analysis module to the man-machine interaction end.
In this embodiment, it needs to be specifically described that, the template matching operation in the template matching module adopts a normalized cross-correlation method, and the matching degree is determined by establishing cross-correlation coefficients of the coordinate system calculation template and each position in the image, where the calculation formula is as follows:wherein->Cross-correlation coefficients representing the template and the positions in the image, < >>Covariance representing coordinates of each position in the template and the image,/->Representing the variance of the coordinates of each position in the template, +.>Representation of the drawingsThe variance of each position coordinate in the image, and the position corresponding to the maximum cross correlation coefficient is the matching position.
In this embodiment, it should be specifically described that, the image preprocessing operation in the image processing module includes adjusting the image including the bolt and the marker line to a size of 80×80 pixels in proportion, and making a semantic segmentation data set for semantic segmentation of the bolt marker line and building a semantic segmentation network to train out a semantic segmentation model of the bolt marker line, which specifically includes the following steps: bolt mark line semantic segmentation data set manufacturing: under different illumination conditions, bolt images of all shapes and specifications on a train and with mark lines are acquired at various shooting angles, labelme software is used for manufacturing a semantic segmentation data set by taking the mark lines as the foreground and the rest as the background, and a semantic segmentation network is trained through the semantic segmentation data set to obtain a mark line semantic segmentation model.
In this embodiment, it should be specifically described that, in the data extraction module, the bolt offset parameter includes an offset of the bolt, a diameter of the bolt, and a pretightening force of the bolt, the bolt loosening parameter includes a current length of the bolt, an initial length of the bolt, and an elastic restoring force of the bolt, and the bolt deformation parameter includes a maximum bearing capacity of the bolt, a rigidity of the bolt, and a cross-sectional area of the bolt.
In this embodiment, it should be specifically described that, the calculating of the marker line profile index in the profile detection module includes the following steps:
step S01: marking the marking line images of all the bolts as 1, 2 and 3 … … n in sequence, calculating the contour indexes of the marking lines in sequence, directly applying force to the bolts by using a tension measuring instrument, measuring the applied force, and determining the pretightening force of the bolts, the elastic restoring force of the bolts and the rigidity of the bolts;
step S02: and calculating bolt offset coefficients of all bolts based on the bolt offset parameters, wherein the calculation formula is as follows:wherein->Representing the bolt offset coefficient of the respective bolts, +.>Indicating the offset of each bolt->Error values representing the respective bolt offsets +.>Representing the bolt diameter of the respective bolt, < > and->Representing the bolt pretightening force of each bolt;
step S03: bolt loosening coefficients of all bolts are calculated based on the bolt loosening parameters, and the calculation formula is as follows:wherein->Representing the bolt loosening coefficient of the respective bolts, +.>Indicating the current length of each bolt,representing the initial length of the respective bolt, < > and->Representing the elastic restoring force of the bolt;
step S04: and calculating the bolt deformation coefficient of each bolt based on the bolt deformation parameters, wherein the calculation formula is as follows:wherein->Representing the deformation coefficient of the individual bolts, +.>Indicating the maximum bearing capacity of the individual bolts, +.>Representing the bolt stiffness of the individual bolts,/>Representing the cross-sectional area of each bolt;
step S05: and calculating the outline index of each bolt mark line based on the bolt offset coefficient, the bolt loosening coefficient and the bolt deformation coefficient, wherein the calculation formula is as follows:wherein->Marking line profile index for each bolt, +.>Representing the bolt offset coefficient of the respective bolts, +.>Representing the bolt loosening coefficient of the respective bolts, +.>Representing the deformation coefficient of the individual bolts, +.>、/>And->Representing the corresponding weights of the coefficients.
In this embodiment, it should be specifically described that, in the loosening analysis module, the marking line profile index is compared with a preset threshold, and the number N of marking line profiles with marking line profile indexes smaller than the preset threshold is counted, if N is smaller than 2, it is described that only one marking line is provided or no marking line is detected, it is determined that the bolt is not loosened, and if N is greater than or equal to 2, it is determined whether the bolt is loosened according to the center line number M;
judging whether bolts are loosened by adopting a screening method, wherein the specific process of the screening method is as follows:
firstly, judging the number N of the outlines of the marking lines, if N is smaller than 2, indicating that only one marking line is needed or no marking line is detected, and considering that the bolt is not loosened under the condition that only one marking line is needed; in the case where the mark line is not detected, in order to reduce false alarms, it is also considered that the bolt is not loosened;
if N is equal to 2, two mark line contours are fitted and marked as K1 and K2 respectively, the minimum circumscribed rectangle of K1 and K2 is calculated, the aspect ratio of each minimum circumscribed rectangle is judged, if the aspect ratio is larger than an aspect ratio threshold value, the center line parallel to the longest side of the minimum circumscribed rectangle and passing through the center point of the center line is calculated, the number M of the center lines is recorded, in this embodiment, since the shooting camera is fixed in position and the image size is adjusted to 80 x 80, through experimental verification, the aspect ratio threshold value is set to 1.4, the area sizes of K1 and K2 are compared, wherein the contour with large area is marked as Kmax, the contour with small area is translated to the center point of the minimum circumscribed rectangle of Kmax according to the center point of the minimum circumscribed rectangle, the translated contour is set as K3, and the IOU values of Kmax and K3 are calculated, wherein the calculation formula of IOU is as follows:wherein kmax is a contour with large area, and K3 is a contour with smaller area which translates to a contour on a center point of a minimum circumscribed rectangle of kmax according to the center point of the minimum circumscribed rectangle;
judging the value of the center line number M, judging whether the IOU value is larger than an IOU threshold value or not if M is smaller than 2, judging whether the bolt looseness is caused if the IOU value is smaller than the IOU threshold value, setting the IOU threshold value to be 0.8 in the embodiment, calculating the shortest distance between K1 and K2 if the IOU value is larger than the threshold value, judging whether the shortest distance is smaller than the shortest distance threshold value or not, setting the shortest distance threshold value to be 12 pixels in the embodiment, judging whether the bolt looseness is caused if the shortest distance is larger than the shortest distance threshold value, otherwise, calculating the center distances between the contour K1 and the contour K2, judging whether the center distance is smaller than the center distance threshold value, setting the center distance threshold value to be 15 pixels in the embodiment, judging that the bolt is not loosened if the center distance is smaller than the center distance threshold value, and judging that the bolt looseness is caused otherwise;
if M is equal to 2, calculating an included angle of two central lines, judging whether the included angle is smaller than an angle threshold, setting the angle threshold to be 5 degrees in the embodiment, if the included angle is larger than the angle threshold, then the subsequent judging method is the same as the judging method of M being smaller than 2, if the included angle is smaller than the angle threshold, calculating the shortest distance between K1 and K2, wherein the shortest distance is the shortest distance between a point on the profile K1 and a point on the profile K2, further judging whether the shortest distance is smaller than the shortest distance threshold, if the shortest distance is larger than the shortest distance threshold, judging whether bolts are loosened, otherwise, calculating the center distances between the profile K1 and the profile K2, judging whether the center distance is smaller than the center distance threshold, if the center distance is smaller than the center distance threshold, judging that the bolts are not loosened, otherwise judging that the bolts are loosened;
it should be noted that, when the M values are different, the solutions of the center distances are different, if M is equal to 0, the center distance is equal to the euclidean distance between the minimum circumscribed rectangular centerlines of the profile K1 and the profile K2; if M is equal to 1, the center distance is equal to the Euclidean distance from the minimum circumscribed rectangular center point of the profile with the length-width ratio smaller than the threshold value to the center line of the other profile; if M is equal to 2, respectively obtaining the distance S1 from the center point of the minimum circumscribed rectangle of the profile K1 to the central line of the profile K2 and the Euclidean distance S2 from the center point of the minimum circumscribed rectangle of the profile K2 to the central line of the profile K1, wherein the center distance is equal to the maximum value in the S1 and the S2;
when the number N of the marked line outlines is greater than 2, firstly, calculating the first three outlines with the largest area, respectively calculating the smallest circumscribed rectangles of the first three outlines, marking the smallest circumscribed rectangles as r1, r2 and r3, then calculating the central point coordinates of r1, r2 and r3, marking the central point coordinates as d1, d2 and d3 and the average value of d1, d2 and d3 as d4, and obtaining the sitting position with the smallest distance from d4 in the d1, d2 and d3The corresponding contour is marked as K4, the other two contours are marked as K5 and K6 respectively, then the straight line where the line connecting the central line points of the contour K4 and the contour K5 and the contour K6 is positioned is marked as lr 1 And lr 2 The method comprises the steps of carrying out a first treatment on the surface of the Calculating the shortest distance between the contour K4 and the contour K5, and the shortest distance between the contour K4 and the contour K6 is marked as S3 and S4, and then judging the straight line lr 1 And straight line lr 2 If the included angle is smaller than the angle threshold, calculating the maximum value in S3 and S4, judging whether the maximum value in S3 and S4 is smaller than the shortest distance threshold, if so, judging that the bolt is not loosened, and if not, judging that the bolt is loosened; if the included angle is larger than the angle threshold, translating the outline K5 and the outline K6 to the center point of the smallest circumscribed rectangle of the outline K4 according to the center point of the smallest circumscribed rectangle, calculating the IOU value of the translated outline and the outline K4, and recording as the IOU 1 And IOU (input output Unit) 2 Next, the IOU is judged 1 And IOU (input output Unit) 2 If both are greater than the IOU threshold, if not, the bolt can be determined to be loose, if yes, the maximum of S3 and S4 is calculated, and if the maximum of S3 and S4 is less than the shortest distance threshold, if yes, the bolt can be determined to be not loose, and if not, the bolt can be determined to be loose.
As shown in fig. 2, a bolt loosening detection method for a railway vehicle includes the steps of:
step S11: the template diagram is manufactured through a marking unit and an acquisition unit: the marking unit is used for drawing a bolt loosening marking line on the bolt, the acquisition unit is used for acquiring all images of the part to be detected of the vehicle through the image acquisition equipment, and all bolts to be detected are marked;
step S12: performing template matching on the acquired image and a template map: acquiring an image containing a bolt and a mark line by using a camera and a light source device, performing template matching operation on the acquired image and the template map, and cutting out the image containing the bolt and the mark line;
step S13: the image which is equal to the image after the image pretreatment and only contains the mark line is obtained through the processing and the segmentation;
step S14: extracting feature data of an image containing only the marker lines: extracting feature data of an image only containing the mark line, which is obtained after semantic segmentation of the bolt mark line;
step S15: calculating a mark line profile index: receiving characteristic data of an image only containing the mark line, and calculating a mark line contour index;
step S16: judging whether the bolt is loosened according to the outline index of the mark line;
step S17: and outputting the image of the loose bolt as a judging result to the man-machine interaction end.
A non-transitory computer-readable storage medium storing computer instructions that cause the computer to execute a bolt looseness detection system for a rail vehicle of any of the above.
In this embodiment, it should be specifically explained that, the difference between the implementation and the prior art is mainly that the implementation is provided with an image acquisition module, a template diagram is manufactured by a marking unit and an acquisition unit, the template matching module performs template matching operation on the acquired image and the template diagram, an image including a bolt and a marking line is cut out, the image processing module obtains an image including only the marking line, the size and the number of channels are equal to those of the image after image preprocessing, the data extraction module extracts characteristic data of the image, the contour detection module builds a model to calculate a marking line contour index, the loosening analysis module compares the marking line contour index with a preset threshold, and counts the marking line contour index to be smaller than the marking line contour number N of the preset threshold, if N is smaller than 2, only one marking line is indicated or no marking line is detected, if N is greater than or equal to 2, whether the bolt is loosened is determined according to the number M of centerlines, the loosening bolt image detected by the loosening analysis module is output to the human-computer interaction end, the state of the bolt can be monitored in real time, the loosening condition of the bolt can be detected and the alarm can be detected in time, the detection performance is improved, and accidents are avoided.
Finally: the foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and principles of the invention are intended to be included within the scope of the invention.
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (8)
1. A bolt looseness detecting system for a railway vehicle is characterized in that: the device comprises an image acquisition module, a template matching module, an image processing module, a data extraction module, a contour detection module, a looseness analysis module and an output interaction module;
the image acquisition module comprises a marking unit and an acquisition unit, wherein the marking unit is used for drawing bolt loosening marking lines on bolts, the acquisition unit is used for acquiring all images of the part to be detected of the vehicle through image acquisition equipment, marking all bolts to be detected and manufacturing a template diagram;
the template matching module acquires an image containing a bolt and a mark line by using a camera and a light source device, performs template matching operation on the acquired image and the template map, and cuts out the image containing the bolt and the mark line;
the image processing module comprises an image preprocessing unit and an image semantic segmentation unit, wherein the image preprocessing unit performs image preprocessing on an image containing bolts and mark lines, and the image semantic segmentation unit performs bolt mark line semantic segmentation on the image subjected to the image preprocessing to obtain an image which is equal to the image subjected to the image preprocessing and contains only mark lines in size and channel number;
the data extraction module is used for extracting characteristic data of the image which only comprises the mark line and is obtained after the semantic segmentation of the bolt mark line in the image processing module, wherein the characteristic data comprise bolt offset parameters, bolt loosening parameters and bolt deformation parameters;
the contour detection module is used for receiving the characteristic data of the image which only contains the mark line and is extracted by the data extraction module, calculating the mark line contour index, and transmitting the calculated mark line contour index to the looseness analysis module;
the loosening analysis module is used for receiving the marking line profile index calculated by the profile detection module, comparing the marking line profile index with a preset threshold value, counting the number of marking line profiles with marking line profile indexes smaller than the preset threshold value, and judging whether the bolt is loosened;
the output interaction module is used for outputting the loosening bolt image detected by the loosening analysis module to the man-machine interaction end.
2. The bolt looseness detection system for a rail vehicle of claim 1, wherein: the template matching operation in the template matching module adopts a normalized cross-correlation method, and the matching degree is determined by establishing cross-correlation coefficients of a coordinate system calculation template and each position in the image, wherein the calculation formula is as follows:wherein->Cross-correlation coefficients representing the template and the positions in the image, < >>Covariance representing coordinates of each position in the template and the image,/->Representing the variance of the coordinates of each position in the template, +.>And (5) representing the variance of each position coordinate in the image, wherein the position corresponding to the maximum cross correlation coefficient is the matching position.
3. The bolt looseness detection system for a rail vehicle of claim 1, wherein: the image preprocessing operation in the image processing module comprises the steps of adjusting the image containing the bolt and the mark line to be 80 x 80 pixel size in proportion, and carrying out semantic segmentation on the bolt mark line by making a semantic segmentation data set and building a semantic segmentation network so as to train out a bolt mark line semantic segmentation model, wherein the concrete process is as follows: bolt mark line semantic segmentation data set manufacturing: under different illumination conditions, bolt images of all shapes and specifications on a train and with mark lines are acquired at various shooting angles, labelme software is used for manufacturing a semantic segmentation data set by taking the mark lines as the foreground and the rest as the background, and a semantic segmentation network is trained through the semantic segmentation data set to obtain a mark line semantic segmentation model.
4. The bolt looseness detection system for a rail vehicle of claim 1, wherein: the bolt deflection parameters in the data extraction module comprise deflection quantity, bolt diameter and bolt pretightening force of the bolt, the bolt loosening parameters comprise current length, initial length and elastic restoring force of the bolt, and the bolt deformation parameters comprise maximum bearing capacity of the bolt, bolt rigidity of the bolt and cross-section area of the bolt.
5. The bolt looseness detection system for a rail vehicle of claim 1, wherein: the calculation of the mark line contour index in the contour detection module comprises the following steps:
step S01: marking the marking line images of all the bolts as 1, 2 and 3 … … n in sequence, calculating the contour indexes of the marking lines in sequence, directly applying force to the bolts by using a tension measuring instrument, measuring the applied force, and determining the pretightening force of the bolts, the elastic restoring force of the bolts and the rigidity of the bolts;
step S02: and calculating bolt offset coefficients of all bolts based on the bolt offset parameters, wherein the calculation formula is as follows:wherein->Representing the bolt offset coefficient of the respective bolts, +.>Indicating the offset of each bolt->Error values representing the respective bolt offsets +.>Representing the bolt diameter of the respective bolt, < > and->Representing the bolt pretightening force of each bolt;
step S03: bolt loosening coefficients of all bolts are calculated based on the bolt loosening parameters, and the calculation formula is as follows:wherein->Representing the bolt loosening coefficient of the respective bolts, +.>Indicating the current length of each bolt,representing the initial length of the respective bolt, < > and->Representing the elastic restoring force of the bolt;
step S04: and calculating the bolt deformation coefficient of each bolt based on the bolt deformation parameters, wherein the calculation formula is as follows:wherein->Representing the deformation coefficient of the individual bolts, +.>Indicating the maximum bearing capacity of the individual bolts, +.>Representing the bolt stiffness of the individual bolts,/>Representing the cross-sectional area of each bolt;
step S05: and calculating the outline index of each bolt mark line based on the bolt offset coefficient, the bolt loosening coefficient and the bolt deformation coefficient, wherein the calculation formula is as follows:wherein->Indicating the index of the mark line profile of each bolt,representing the bolt offset coefficient of the respective bolts, +.>Representing the bolt loosening coefficient of the respective bolts, +.>Representing the deformation coefficient of the individual bolts, +.>、/>And->Representing the corresponding weights of the coefficients.
6. The bolt looseness detection system for a rail vehicle of claim 1, wherein: and comparing the mark line profile index with a preset threshold value in the loosening analysis module, counting the number N of mark line profiles with the mark line profile index smaller than the preset threshold value, if N is smaller than 2, indicating that only one mark line is needed or no mark line is detected, judging that the bolt is not loosened, and if N is larger than or equal to 2, judging whether the bolt is loosened according to the central line number M.
7. A bolt looseness detection method for a rail vehicle, for using the bolt looseness detection system for a rail vehicle described in any of claims 1 to 6, characterized in that: the method comprises the following steps:
step S11: the template diagram is manufactured through a marking unit and an acquisition unit: the marking unit is used for drawing a bolt loosening marking line on the bolt, the acquisition unit is used for acquiring all images of the part to be detected of the vehicle through the image acquisition equipment, and all bolts to be detected are marked;
step S12: performing template matching on the acquired image and a template map: acquiring an image containing a bolt and a mark line by using a camera and a light source device, performing template matching operation on the acquired image and the template map, and cutting out the image containing the bolt and the mark line;
step S13: the image which is equal to the image after the image pretreatment and only contains the mark line is obtained through the processing and the segmentation;
step S14: extracting feature data of an image containing only the marker lines: extracting feature data of an image only containing the mark line, which is obtained after semantic segmentation of the bolt mark line;
step S15: calculating a mark line profile index: receiving characteristic data of an image only containing the mark line, and calculating a mark line contour index;
step S16: judging whether the bolt is loosened according to the outline index of the mark line;
step S17: and outputting the image of the loose bolt as a judging result to the man-machine interaction end.
8. A non-transitory computer readable storage medium storing computer instructions, characterized by: the computer instructions cause the computer to perform a bolt looseness detection system for a rail vehicle of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410068789.8A CN117593515B (en) | 2024-01-17 | 2024-01-17 | Bolt loosening detection system and method for railway vehicle and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410068789.8A CN117593515B (en) | 2024-01-17 | 2024-01-17 | Bolt loosening detection system and method for railway vehicle and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117593515A true CN117593515A (en) | 2024-02-23 |
| CN117593515B CN117593515B (en) | 2024-03-29 |
Family
ID=89922262
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410068789.8A Active CN117593515B (en) | 2024-01-17 | 2024-01-17 | Bolt loosening detection system and method for railway vehicle and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117593515B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118411571A (en) * | 2024-06-28 | 2024-07-30 | 中数智科(杭州)科技有限公司 | Bolt abnormality detection method for railway vehicle |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110246132A (en) * | 2019-06-23 | 2019-09-17 | 中车青岛四方车辆研究所有限公司 | Rail vehicle bolt looseness detection method and system |
| CN110390668A (en) * | 2019-06-26 | 2019-10-29 | 石家庄铁道大学 | Bolt looseness detection method, terminal device and storage medium |
| CN110599506A (en) * | 2019-10-16 | 2019-12-20 | 湖南大学 | Point cloud segmentation method for three-dimensional measurement of complex special-shaped curved surface robot |
| US20210027475A1 (en) * | 2018-03-23 | 2021-01-28 | University Of Kansas | Vision-based fastener loosening detection |
| CN114445435A (en) * | 2021-12-23 | 2022-05-06 | 中数智科(杭州)科技有限公司 | Vehicle body bolt looseness judging method based on mark line displacement deviation |
| CN114994060A (en) * | 2022-07-15 | 2022-09-02 | 君享科技(深圳)有限公司 | Intelligent detection system and method for magnetic ring appearance defects under machine vision |
| CN115482207A (en) * | 2022-09-01 | 2022-12-16 | 上汽大众汽车有限公司 | Bolt looseness detection method and system |
| CN115511828A (en) * | 2022-09-27 | 2022-12-23 | 华电电力科学研究院有限公司 | Bolt detection method, device, equipment and medium |
| CN115930879A (en) * | 2022-12-30 | 2023-04-07 | 中国铁建重工集团股份有限公司 | Contour detection device and method for workpiece, server and storage medium |
| CN116067555A (en) * | 2023-04-06 | 2023-05-05 | 西南交通大学 | Bolt looseness detection system and method for urban rail transit and storage medium |
| KR20230065014A (en) * | 2021-11-04 | 2023-05-11 | 한국전력공사 | Automated analysis method and apparatus of fittings in power equipment using keyframe |
| CN116109613A (en) * | 2023-02-24 | 2023-05-12 | 华中科技大学 | Defect detection method and system based on distribution characterization |
| US20230306761A1 (en) * | 2020-09-02 | 2023-09-28 | The General Hospital Corporation | Methods for identifying cross-modal features from spatially resolved data sets |
| CN117011212A (en) * | 2022-06-21 | 2023-11-07 | 南通市科睿轨道科技有限公司 | Track engineering section fastener connection state monitoring analysis system |
| CN117029733A (en) * | 2023-10-08 | 2023-11-10 | 中冶建筑研究总院有限公司 | Bolt loosening detection method, system and device based on computer vision |
| CN117369460A (en) * | 2023-10-30 | 2024-01-09 | 西安理工大学 | Intelligent inspection method and system for loosening faults of vehicle bolts |
-
2024
- 2024-01-17 CN CN202410068789.8A patent/CN117593515B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210027475A1 (en) * | 2018-03-23 | 2021-01-28 | University Of Kansas | Vision-based fastener loosening detection |
| CN110246132A (en) * | 2019-06-23 | 2019-09-17 | 中车青岛四方车辆研究所有限公司 | Rail vehicle bolt looseness detection method and system |
| CN110390668A (en) * | 2019-06-26 | 2019-10-29 | 石家庄铁道大学 | Bolt looseness detection method, terminal device and storage medium |
| CN110599506A (en) * | 2019-10-16 | 2019-12-20 | 湖南大学 | Point cloud segmentation method for three-dimensional measurement of complex special-shaped curved surface robot |
| US20230306761A1 (en) * | 2020-09-02 | 2023-09-28 | The General Hospital Corporation | Methods for identifying cross-modal features from spatially resolved data sets |
| KR20230065014A (en) * | 2021-11-04 | 2023-05-11 | 한국전력공사 | Automated analysis method and apparatus of fittings in power equipment using keyframe |
| CN114445435A (en) * | 2021-12-23 | 2022-05-06 | 中数智科(杭州)科技有限公司 | Vehicle body bolt looseness judging method based on mark line displacement deviation |
| CN117011212A (en) * | 2022-06-21 | 2023-11-07 | 南通市科睿轨道科技有限公司 | Track engineering section fastener connection state monitoring analysis system |
| CN114994060A (en) * | 2022-07-15 | 2022-09-02 | 君享科技(深圳)有限公司 | Intelligent detection system and method for magnetic ring appearance defects under machine vision |
| CN115482207A (en) * | 2022-09-01 | 2022-12-16 | 上汽大众汽车有限公司 | Bolt looseness detection method and system |
| CN115511828A (en) * | 2022-09-27 | 2022-12-23 | 华电电力科学研究院有限公司 | Bolt detection method, device, equipment and medium |
| CN115930879A (en) * | 2022-12-30 | 2023-04-07 | 中国铁建重工集团股份有限公司 | Contour detection device and method for workpiece, server and storage medium |
| CN116109613A (en) * | 2023-02-24 | 2023-05-12 | 华中科技大学 | Defect detection method and system based on distribution characterization |
| CN116067555A (en) * | 2023-04-06 | 2023-05-05 | 西南交通大学 | Bolt looseness detection system and method for urban rail transit and storage medium |
| CN117029733A (en) * | 2023-10-08 | 2023-11-10 | 中冶建筑研究总院有限公司 | Bolt loosening detection method, system and device based on computer vision |
| CN117369460A (en) * | 2023-10-30 | 2024-01-09 | 西安理工大学 | Intelligent inspection method and system for loosening faults of vehicle bolts |
Non-Patent Citations (1)
| Title |
|---|
| 沈浩;江臣;陈宇文;王国香;李枝军;: "基于深度学习的钢桁架桥螺栓病害智能识别方法", 南京工业大学学报(自然科学版), no. 05, 30 September 2020 (2020-09-30) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118411571A (en) * | 2024-06-28 | 2024-07-30 | 中数智科(杭州)科技有限公司 | Bolt abnormality detection method for railway vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117593515B (en) | 2024-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN117593515B (en) | Bolt loosening detection system and method for railway vehicle and storage medium | |
| KR101748121B1 (en) | System and method for detecting image in real-time based on object recognition | |
| CN110334750B (en) | Power transmission line iron tower bolt corrosion degree image classification and identification method | |
| CN111626204B (en) | Railway foreign matter invasion monitoring method and system | |
| CN113516629A (en) | Intelligent detection system for TFDS passing operation | |
| CN104240239A (en) | Method for detecting local road segment hazy weather based on road image | |
| CN116168356B (en) | Vehicle damage judging method based on computer vision | |
| CN109934135B (en) | Rail foreign matter detection method based on low-rank matrix decomposition | |
| CN113487561B (en) | Pantograph foreign matter detection method and device based on gray gradient abnormal voting | |
| CN102622885A (en) | Method and device for detecting traffic incidents | |
| CN113639685A (en) | Displacement detection method, device, equipment and storage medium | |
| CN112734702A (en) | Bridge safety monitoring method based on big data analysis and machine vision and cloud platform | |
| CN113447255A (en) | Bolt node looseness detection method and system based on unmanned aerial vehicle image space positioning | |
| CN110793501A (en) | Subway tunnel clearance detection method | |
| CN113807220A (en) | Traffic event detection method and device, electronic equipment and readable storage medium | |
| CN103473925B (en) | A kind of verification method of road vehicle detection system | |
| CN113032907A (en) | Method and system for correcting vehicle shaking disease data deviation based on waveform correlation | |
| CN116631187B (en) | Intelligent acquisition and analysis system for case on-site investigation information | |
| CN113295253A (en) | Method and system for monitoring weighing data abnormity of non-sensing wagon balance vehicle | |
| CN114882708B (en) | Vehicle identification method based on monitoring video | |
| CN113888866B (en) | Road vehicle management system with multistage early warning function | |
| CN115690660A (en) | Aerial cableway cable-off detection method and system | |
| CN114429295A (en) | Building engineering structure crack state safety monitoring and analyzing method based on BIM three-dimensional model technology | |
| CN113239832A (en) | Hidden danger intelligent identification method and system based on image identification | |
| CN118411571B (en) | Bolt abnormality detection method for railway vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |