CN116363603A - Deep learning-based car ladder handrail detection method, storage medium and equipment - Google Patents

Deep learning-based car ladder handrail detection method, storage medium and equipment Download PDF

Info

Publication number
CN116363603A
CN116363603A CN202310271676.3A CN202310271676A CN116363603A CN 116363603 A CN116363603 A CN 116363603A CN 202310271676 A CN202310271676 A CN 202310271676A CN 116363603 A CN116363603 A CN 116363603A
Authority
CN
China
Prior art keywords
image
network
handrail
quality
car ladder
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310271676.3A
Other languages
Chinese (zh)
Inventor
张光伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Kejia General Mechanical and Electrical Co Ltd
Original Assignee
Harbin Kejia General Mechanical and Electrical Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Kejia General Mechanical and Electrical Co Ltd filed Critical Harbin Kejia General Mechanical and Electrical Co Ltd
Priority to CN202310271676.3A priority Critical patent/CN116363603A/en
Publication of CN116363603A publication Critical patent/CN116363603A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

A car ladder handrail detection method, storage medium and equipment based on deep learning belong to the technical field of railway wagon detection. The method aims at solving the problem that the detection effect is not ideal due to poor image quality of the escalator handrail when the escalator handrail is detected by the existing method. Firstly, building a training set of the ladder handrail images of the vehicle, and carrying out degradation and quality improvement to build image pairs; and then adding an adaptively adjusted convolutional network before the characteristic extraction network of the Faster-RCNN to obtain an improved Faster-RCNN network, and training the network by using the image constructed by the degradation and the quality improvement based on the loss function of the two parts of sub-networks to obtain the parameter weight. And finally detecting the handrail part of the car ladder according to the trained network.

Description

Deep learning-based car ladder handrail detection method, storage medium and equipment
Technical Field
The invention belongs to the technical field of railway wagon detection, and particularly relates to a car ladder handrail detection method, a storage medium and equipment.
Background
In order to ensure safe operation of the railway wagon, periodic inspection of the railway wagon parts is required. The early-stage train fault detection mode mainly comprises manual image checking or field checking, and is low in efficiency, so that conditions such as missing of faults and false detection are easily caused, and the running safety of the train is influenced. Therefore, the fault automatic identification technology is significant in motor car detection, and the method uses a deep learning technology to take the high-definition image obtained by shooting as data, so that automatic fault alarm is realized, and the motor car operation efficiency is improved.
With the development of deep learning technology, the existing deep learning technology can realize accurate detection of most railway wagon components. In general, the closer the data quality is to the data quality in the prediction and training, the higher the accuracy of the recognition. Therefore, in general, training data with different qualities is added to train the recognition network model, that is, the recognition network can recognize the data with different qualities by including as much image data with different qualities as possible on the training data layer, so as to ensure the accuracy of detection. However, in some image data with poor quality, the effect of improving the accuracy in this way is very limited, and the recognition network still cannot well judge whether the recognition area has a fault or not. Especially, the detection of the car ladder handrail has very many conditions of poor image quality, so that the situation of poor recognition effect is often caused by poor image quality when the detection is performed by utilizing the existing deep learning technology.
Disclosure of Invention
The invention aims to solve the problem that the detection effect is not ideal due to the poor image quality of the escalator handrail when the escalator handrail is detected by using the existing mode.
The car ladder handrail detection method based on deep learning comprises the following steps:
positioning a car ladder handrail part area on the side of the car body through the acquired side part image; loading an improved Faster-RCNN model and weight, carrying out convolutional network image quality improvement processing of self-adaptive adjustment on an image to be identified, and then predicting, wherein the prediction results comprise handrails, handrail breakage, rivets and rivet loss; according to the detected handrails and the detected handrail breaking classes in the images to be identified, carrying out logic judgment by combining the detected rivets and the detected rivet breaking classes, so as to realize fault prediction after improving the image quality;
the improved Faster-RCNN is obtained by the following steps:
s1, acquiring an image of a handrail of a car ladder on the side part of a car body, and constructing an image data set;
carrying out data pair construction on each image in the image data set, namely carrying out quality reduction treatment on the images, and then carrying out quality improvement treatment on the images before quality reduction to obtain an image pair of a low-quality image and a high-quality image;
s2, building an improved Faster-RCNN identification network: adding an adaptive adjustment convolution network before a Faster-RCNN feature extraction network, and inputting data to be identified into the Faster-RCNN feature extraction network for processing after obtaining a result through the adaptive adjustment convolution network;
the self-adaptive adjustment convolution network structure is as follows:
firstly, a convolution module of 3*3 is connected with a Swin-Conv module after the convolution module of 3*3, and then is connected with an improved cascade structure network, wherein the improved cascade structure network adopts cascade structure units of SwinT+BN+ReLU; setting a Swin-Conv network after the cascade structure network, and finally connecting a convolution module of 3*3;
s3, training the improved Faster-RCNN by using the low-quality image and the high-quality image:
sending the reduced quality image into a self-adaptive convolutional network, and performing Loss with a high quality image correspondingly generated when a data set is established, and calculating to obtain Loss deblur ；
The self-adaptive convolutional network generation result is sent to the recognition network Faster-RCNN to generate the Loss of the corresponding detection part dec The method comprises the steps of carrying out a first treatment on the surface of the And then the Loss of the whole network is obtained:
Loss actually ＝αLoss deblur +(1-α)Loss dec
wherein, loss actually For the actual total Loss, α is the Loss weight;
thereby obtaining the trained improved Faster-RCNN.
Further, in the process of constructing the image dataset in step S1, it is necessary to perform fault marking on the car ladder handrail image.
Further, when the image is subjected to the quality degradation process, the image quality degradation formula is as follows:
Figure BDA0004134911800000021
where y is the reduced quality image, x is the original image, k is the blur kernel,
Figure BDA0004134911800000022
representing convolution ∈ s For downsampling, s is the downsampling multiple and n is noise.
Further, when the image is subjected to quality reduction processing, different parameters are manually set based on an image quality reduction formula to perform quality reduction.
Further, in the process of manually setting different parameters to degrade based on an image degradation formula, the degradation process is performed by setting different noise types, and the noise n comprises Gaussian noise and photon noise.
Further, in the self-adaptive convolutional network, the cascade structure network comprises 17 cascade structure units, and the Swin-Conv network arranged behind the cascade structure network comprises 4 Swin-Conv modules.
Further, the structure of the Swin-Conv module is as follows:
first, X is obtained by convolution of 1*1 and processing of split layer 1 ，X 2 Two features:
X 1 ，X 2 ＝Split(Conv1*1(X))
wherein X represents the input of the 1*1 convolution;
X 1 treatment by SwinT, X 2 Processing by RConv to obtain two processed results Y 1 ,Y 2 ；
Y 1 ,Y 2 ＝SwinT(X 1 ),RConv(X 2 )
Then for the obtained Y 1 ,Y 2 After feature fusion and convolution of 1*1, add to the original input, namely:
Z＝Conv1*1(Concat(Y 1 ,Y 2 ))+X。
further, the process of logically determining, based on the detected handrail and handrail break class in the image to be identified, in combination with the detected rivet and rivet break class, includes the steps of:
when the handrails are detected, if the detected result also contains rivets or the rivets are broken, calculating the length-width ratio of a detection frame where the handrails are positioned, and judging that faults exist when the length-width ratio is larger than a threshold value 1;
when the handrail breakage is detected but the rivet breakage or the rivet loss is not contained, calculating the length-width ratio of a detection frame where the handrail breakage is located, and judging that a fault exists when the length-width ratio is larger than a threshold value 2.
A computer storage medium having stored therein at least one instruction loaded and executed by a processor to implement a deep learning based car ladder handrail detection method.
A deep learning based car ladder handrail detection apparatus, the apparatus comprising a processor and a memory having stored therein at least one instruction loaded and executed by the processor to implement a deep learning based car ladder handrail detection method.
The beneficial effects are that:
1. the automatic detection replaces manual detection, the detection efficiency and accuracy are improved, the influence of the physiology and the psychology of personnel is avoided, and the operation quality is greatly improved.
2. The present invention provides a method for improving a network so that network detection is more accurate. The invention adopts the mode of combining training by utilizing the low-quality image and the high-quality image, can still ensure the detection effect under the condition of more poor image quality of the car ladder handrail, i.e. improves the overall detection accuracy. Meanwhile, the invention modifies the Faster-RCNN identification network to a certain extent, so that the improved network can better express the image characteristics of one of the image pairs on the basis of better detection capability, thereby better promoting the matching treatment of the image pairs in the follow-up process and further ensuring the detection effect of the invention.
Drawings
Fig. 1 is a flow chart of a car ladder handrail detection.
Fig. 2 is a block diagram of a constructed neural network.
Fig. 3 is a low quality image obtained by degradation.
Fig. 4 is a high quality image obtained by upgrading.
Detailed Description
The first embodiment is as follows:
the embodiment is a vehicle ladder handrail detection method based on deep learning, which classifies collected vehicle ladder handrails by adopting a convolutional neural network, wherein a detection flow chart is shown in fig. 1, and specifically comprises the following steps:
1. building training data sets
The line images are acquired through the motor car shooting equipment, the car ladder handrail images on the side of the car body are collected, the diversity and the complexity of the images are guaranteed, the brightness of various images are changed, the images on rainy and snowy days and sunny days are distinguished, the quality of the images is good, and the like, and the images are contained in the data set.
The fault marking is carried out on the car ladder handrail image, and the car ladder handrail image comprises four types of handrails, namely handrails broken, rivets and lost rivets. When the images are marked, because the failed images are few, manual manufacturing failure can be selected, namely PS can be carried out on the images according to the actual loss condition, so that the effect of sample equalization is achieved, and a low-quality and high-quality data pair needs to be constructed no matter the failed images of the PS or the original normal images.
The image is subjected to data pair construction, and these images are first subjected to noise addition and image processing with a blur kernel, so that the image quality is reduced as low-quality image data, as shown in fig. 3. Then, noise reduction and deblurring are carried out on the images before the quality is reduced, so that another group of relatively clear high-quality image data is obtained, as shown in fig. 4; an image pair of a low-quality image and a high-quality image is obtained by degrading the image and enhancing the image.
The formula for image degradation is as follows:
Figure BDA0004134911800000041
where y is the degraded image, x is the original image, k is the blur kernel,
Figure BDA0004134911800000042
representing convolution ∈ s For downsampling, s is a downsampling multiple, and n is noise;
the image degradation can be performed by manually setting parameters by using the formula, wherein n mainly uses Gaussian noise and photon noise, and besides, some system tools can be used for degradation or promotion to construct a data pair.
Different parameters are used as much as possible when constructing a low-quality image and a high-quality image pair, so that the diversity of a degradation mode and a quality improvement mode can be ensured, and the network effect is better, wherein the low-quality image obtained by the degradation mode is simulated data needing to be improved, and the improved image is a desired image suitable for identification.
2. Building an identification network and obtaining a weight file
The built identification network is mainly used for detecting faults, and the noise level, the brightness and darkness degree and the blurring condition of the image can seriously influence the identification condition of the faults. In some image data with poor quality, the mode of containing image data with different quality as much as possible on the aspect of training data and enabling an identification network to recognize the data with different quality has limited effect of improving accuracy, and the problem that whether an identification area has faults or not still cannot be well judged by the identification network is solved. On the basis of using a low-quality image and high-quality image pair, the invention also modifies the Faster-RCNN identification network to a certain extent, so that the improved network can better express the image characteristics of one of the image pairs on the basis of better detection capability, thereby better promoting the matching processing of the image pair in the follow-up process; the self-adaptive adjustment convolution network is added before the characteristic extraction network of the fast-RCNN, so that the data to be identified is input into the characteristic extraction network of the fast-RCNN after the result is obtained through the self-adaptive adjustment convolution network.
The added convolutional neural network plays a role in self-adaptive adjustment of the image, so that the image to be identified can reach a high-quality state more suitable for identification after self-adaptive adjustment, and the image is improved in noise level and blur level.
In the self-adaptive convolutional network, a Swin-Conv module is used, and the specific structure of the Swin-Conv module is as follows:
firstly, a convolution module of 3*3 is connected with a Swin-Conv module after the convolution module of 3*3, and then a modified cascade structure network is connected, wherein the modified cascade structure network adopts SwinT (Swin Transformer) cascade structure units, namely, 17 cascade structure units of SwinT+BN+ReLU; the cascade structure network is followed by a Swin-Conv network comprising 4 Swin-Conv modules, and finally a convolution module of 3*3 is connected.
The specific structure of the Swin-Conv module is as follows:
first, X is obtained by convolution of 1*1 and processing of split layer 1 ，X 2 Two features:
X 1 ，X 2 ＝Split(Conv1*1(X))
wherein X represents the input of the 1*1 convolution;
the two features are then treated separately as follows:
operation X 1 Treatment by SwinT, X 2 Processing by RConv to obtain two processed results Y 1 ,Y 2 ；
Y 1 ,Y 2 ＝SwinT(X 1 ),RConv(X 2 )
Then for the obtained Y 1 ,Y 2 After feature fusion and convolution of 1*1, add to the original input:
Z＝Conv1*1(Concat(Y 1 ,Y 2 ))+X
training the improved Faster-RCNN with images of low quality images and high quality images: sending the degraded image into a Faster-RCNN (convolutional network) added with self-adaptive adjustment, obtaining a result, and carrying out Loss with a high-quality image correspondingly generated when a data set is established, wherein the Loss of the part is selected to be a conventional L1Loss, and the Loss is obtained by calculation deblur The result of the generation after passing through the part of the network is sent to the recognition network Faster-RCNN, and the Loss of the corresponding detection part is generated according to the mmdetection framework dec The two Loss add up to the Loss of the entire network:
Loss actually ＝αLoss deblur +(1-α)Loss dec
wherein, loss actually For the actual total Loss, it is sent to the optimizer for optimization, and α is the weight occupied by Loss, where a fixed value of 0.3 is used.
Compared with the conventional Faster-RCNN detection algorithm, the network can adaptively adjust the image to a certain extent before detection, so that the image is more suitable for detection, the accuracy of the detection algorithm is further improved, and the program can more accurately identify faults by improving the accuracy of the identification algorithm.
3. Prediction
Positioning a car ladder handrail part area on the side part of the car body through the acquired side part image;
loading improved Faster-RCNN model weight, carrying out Faster-RCNN processing added with a self-adaptive convolution network on an image to be identified (an image of a car ladder handrail part area positioned to the side part of a car body and a car ladder handrail part area intercepted), carrying out logic judgment according to detected handrails and handrail breaking classes in the image to be identified and combined with detected rivets and rivet breaking classes, so as to realize fault prediction after improving the image quality, and uploading alarm information to a platform if the image is considered to need alarm through logic judgment.
In some embodiments, when the handrail class is detected, if the detected image also contains rivets or rivet broken classes, calculating the length-width ratio of a detection frame where the handrail is positioned, alarming when the length-width ratio is larger than a threshold value 1, and not alarming when the length-width ratio is smaller than the threshold value 1. When the handrail breakage is detected but the two types of rivet breakage or rivet loss are not contained, calculating the length-width ratio of a detection frame where the handrail breakage is located, if the length-width ratio is lower than a threshold value 2, not alarming, and if the length-width ratio is higher than the threshold value 2, alarming.
Through research and analysis, the two types of rivet breaking and rivet losing are found to occur along with breaking, and in order to avoid misjudgment aiming at the condition of detecting the handrail, the rivet breaking or rivet losing is used as the misjudgment of the handrail to be further judged by the misjudgment of the handrail through further research, and the occurrence of misjudgment can be greatly reduced; as for the invention, through research and analysis, the fact that no rivet is lost or the rivet is broken is used as a further judgment for misjudging the handrail as the handrail is broken.
The second embodiment is as follows:
the embodiment is a computer storage medium, in which at least one instruction is stored, where the at least one instruction is loaded and executed by a processor to implement the vehicle ladder handrail detection method based on deep learning.
It should be understood that the instructions comprise a computer program product, software, or computerized method corresponding to any of the methods described herein; the instructions may be used to program a computer system, or other electronic device. Computer storage media may include readable media having instructions stored thereon and may include, but is not limited to, magnetic storage media, optical storage media; magneto-optical storage media include read-only memory ROM, random-access memory RAM, erasable programmable memory (e.g., EPROM and EEPROM), and flash memory layers, or other types of media suitable for storing electronic instructions.
And a third specific embodiment:
the embodiment is a car ladder handrail detection device based on deep learning, the device comprises a processor and a memory, and it should be understood that the device comprising any device comprising the processor and the memory described in the invention can also comprise other units and modules for displaying, interacting, processing, controlling and other functions through signals or instructions;
the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to realize the car ladder handrail detection method based on deep learning.
The above examples of the present invention are only for describing the calculation model and calculation flow of the present invention in detail, and are not limiting of the embodiments of the present invention. Other variations and modifications of the above description will be apparent to those of ordinary skill in the art, and it is not intended to be exhaustive of all embodiments, all of which are within the scope of the invention.

Claims (10)

1. The car ladder handrail detection method based on deep learning is characterized by comprising the following steps of:
positioning a car ladder handrail part area on the side of the car body through the acquired side part image; loading an improved Faster-RCNN model and weight, carrying out convolutional network image quality improvement processing of self-adaptive adjustment on an image to be identified, and then predicting, wherein the prediction results comprise handrails, handrail breakage, rivets and rivet loss; according to the detected handrails and the detected handrail breaking classes in the images to be identified, carrying out logic judgment by combining the detected rivets and the detected rivet breaking classes, so as to realize fault prediction after improving the image quality;
the improved Faster-RCNN is obtained by the following steps:
s1, acquiring an image of a handrail of a car ladder on the side part of a car body, and constructing an image data set;
carrying out data pair construction on each image in the image data set, namely carrying out quality reduction treatment on the images, and then carrying out quality improvement treatment on the images before quality reduction to obtain an image pair of a low-quality image and a high-quality image;
s2, building an improved Faster-RCNN identification network: adding an adaptive adjustment convolution network before a Faster-RCNN feature extraction network, and inputting data to be identified into the Faster-RCNN feature extraction network for processing after obtaining a result through the adaptive adjustment convolution network;
the self-adaptive adjustment convolution network structure is as follows:
firstly, a convolution module of 3*3 is connected with a Swin-Conv module after the convolution module of 3*3, and then is connected with an improved cascade structure network, wherein the improved cascade structure network adopts cascade structure units of SwinT+BN+ReLU; setting a Swin-Conv network after the cascade structure network, and finally connecting a convolution module of 3*3;
s3, training the improved Faster-RCNN by using the low-quality image and the high-quality image:
sending the reduced quality image into a self-adaptive convolutional network, and performing Loss with a high quality image correspondingly generated when a data set is established, and calculating to obtain Loss deblur ；
The self-adaptive convolutional network generation result is sent to the recognition network Faster-RCNN to generate the Loss of the corresponding detection part dec The method comprises the steps of carrying out a first treatment on the surface of the And then the Loss of the whole network is obtained:
Lossa ctually ＝αLoss deblur +(1-α)Loss dec
wherein, loss actually For the actual total Loss, α is the Loss weight;
thereby obtaining the trained improved Faster-RCNN.
2. The deep learning-based car ladder handrail detection method according to claim 1, wherein in the process of constructing the image dataset in step S1, fault marking is required for car ladder handrail images.
3. The method for detecting the handrail of the car ladder based on the deep learning according to claim 2, wherein when the image is subjected to the quality reduction processing, an image quality reduction formula is as follows:
Figure FDA0004134911780000011
where y is the reduced quality image, x is the original image, k is the blur kernel,
Figure FDA0004134911780000012
representing convolution ∈ s For downsampling, s is the downsampling multiple and n is noise.
4. The method for detecting a car ladder handrail based on deep learning according to claim 3, wherein when the image is subjected to the quality degradation process, different parameters are manually set based on an image quality degradation formula to degrade.
5. The method for detecting the car ladder handrail based on deep learning according to claim 4, wherein in the process of manually setting different parameters to degrade based on an image degradation quality formula, the degradation processing is performed by setting different noise types, and the noise n comprises Gaussian noise and photon noise.
6. The method for detecting the car ladder handrail based on deep learning according to claim 5, wherein in the self-adaptive adjustment convolutional network, the cascade structure network comprises 17 cascade structure units, and the Swin-Conv network arranged behind the cascade structure network comprises 4 Swin-Conv modules.
7. The deep learning-based car ladder handrail detection method as claimed in claim 6, wherein the Swin-Conv module has the following structure:
first, X is obtained by convolution of 1*1 and processing of split layer 1 ，X 2 Two features:
X 1 ，X 2 ＝Split(Conv1*1(X))
wherein X represents the input of the 1*1 convolution;
X 1 treatment by SwinT, X 2 Processing by RConv to obtain two processed results Y 1 ,Y 2 ；
Y 1 ,Y 2 ＝SwinT(X 1 ),RConv(X 2 )
Then for the obtained Y 1 ,Y 2 After feature fusion and convolution of 1*1, add to the original input, namely:
Z＝Conv1*1(Concat(Y 1 ,Y 2 ))+X。
8. the deep learning based car ladder handrail detection method according to any one of claims 1 to 7, wherein the process of performing logic judgment in combination with the detected rivets and rivet break classes according to the detected handrail and handrail break classes in the image to be identified comprises the steps of:
when the handrails are detected, if the detected result also contains rivets or the rivets are broken, calculating the length-width ratio of a detection frame where the handrails are positioned, and judging that faults exist when the length-width ratio is larger than a threshold value 1;
when the handrail breakage is detected but the rivet breakage or the rivet loss is not contained, calculating the length-width ratio of a detection frame where the handrail breakage is located, and judging that a fault exists when the length-width ratio is larger than a threshold value 2.
9. A computer storage medium having stored therein at least one instruction that is loaded and executed by a processor to implement the deep learning based car ladder handrail detection method of any one of claims 1 to 8.
10. A deep learning based car ladder handrail detection apparatus, the apparatus comprising a processor and a memory having stored therein at least one instruction loaded and executed by the processor to implement the deep learning based car ladder handrail detection method of any one of claims 1 to 8.
CN202310271676.3A 2023-03-20 2023-03-20 Deep learning-based car ladder handrail detection method, storage medium and equipment Pending CN116363603A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310271676.3A CN116363603A (en) 2023-03-20 2023-03-20 Deep learning-based car ladder handrail detection method, storage medium and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310271676.3A CN116363603A (en) 2023-03-20 2023-03-20 Deep learning-based car ladder handrail detection method, storage medium and equipment

Publications (1)

Publication Number Publication Date
CN116363603A true CN116363603A (en) 2023-06-30

Family

ID=86935714

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310271676.3A Pending CN116363603A (en) 2023-03-20 2023-03-20 Deep learning-based car ladder handrail detection method, storage medium and equipment

Country Status (1)

Country Link
CN (1) CN116363603A (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114140353A (en) * 2021-11-25 2022-03-04 苏州大学 Swin-Transformer image denoising method and system based on channel attention
CN115170882A (en) * 2022-07-19 2022-10-11 哈尔滨市科佳通用机电股份有限公司 Optimization method of rail wagon part detection network and guardrail breaking fault identification method
CN115795683A (en) * 2022-12-08 2023-03-14 四川大学 Wing profile optimization method fusing CNN and Swin transform network

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114140353A (en) * 2021-11-25 2022-03-04 苏州大学 Swin-Transformer image denoising method and system based on channel attention
CN115170882A (en) * 2022-07-19 2022-10-11 哈尔滨市科佳通用机电股份有限公司 Optimization method of rail wagon part detection network and guardrail breaking fault identification method
CN115795683A (en) * 2022-12-08 2023-03-14 四川大学 Wing profile optimization method fusing CNN and Swin transform network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JINGYUN LIANG 等: "《SwinIR: Image Restoration Using Swin Transformer》", 《2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS(ICCVW)》, pages 1833 - 1844 *
KAI ZHANG 等: "《Practical Blind Denoising via Swin-Conv-UNet and Data Synthesis》", 《ARXIV》, pages 1 - 12 *

Similar Documents

Publication Publication Date Title
US11887064B2 (en) Deep learning-based system and method for automatically determining degree of damage to each area of vehicle
CN107437245B (en) High-speed railway contact net fault diagnosis method based on deep convolutional neural network
US20210089895A1 (en) Device and method for generating a counterfactual data sample for a neural network
CN111079747B (en) Railway wagon bogie side frame fracture fault image identification method
CN111932489A (en) Weld defect detection method, system, storage medium, computer device and terminal
JP6844563B2 (en) Inspection equipment, image identification equipment, identification equipment, inspection method, and inspection program
KR20200039049A (en) Inspection method for appearance badness and inspection system for appearance badness
CN111652295B (en) Railway wagon coupler yoke key joist falling fault identification method
CN111091543B (en) Railway wagon swing bolster spring loss fault target detection method
CN111738342A (en) Pantograph foreign matter detection method, storage medium and computer equipment
CN113159061B (en) Actual tunnel surrounding rock fragment identification method based on example segmentation
CN111079628A (en) Floor damage fault image identification method
CN112417766A (en) Fault diagnosis method mainly based on fault-free data
CN114387230A (en) PCB defect detection method based on re-verification detection
CN111091554B (en) Railway wagon swing bolster fracture fault image identification method
CN114119551A (en) Quantitative analysis method for human face image quality
EP4064183A1 (en) Model generation apparatus, estimation apparatus, model generation method, and model generation program
CN116363603A (en) Deep learning-based car ladder handrail detection method, storage medium and equipment
CN116385953B (en) Railway wagon door hinge breaking fault image identification method
CN111415326A (en) Method and system for detecting abnormal state of railway contact net bolt
CN114873425B (en) Escalator driving chain fault diagnosis method based on vibration characteristic enhancement
CN116484513A (en) Rolling bearing fault diagnosis method based on multi-level abstract time feature fusion
CN113239865A (en) Deep learning-based lane line detection method
CN112766174A (en) Railway train carriage group bottom plate loss fault detection method
CN111539277A (en) Detection method and system for construction machinery in power transmission line area

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination