CN114494161A - Pantograph foreign matter detection method and device based on image contrast and storage medium - Google Patents

Pantograph foreign matter detection method and device based on image contrast and storage medium Download PDF

Info

Publication number
CN114494161A
CN114494161A CN202210031068.0A CN202210031068A CN114494161A CN 114494161 A CN114494161 A CN 114494161A CN 202210031068 A CN202210031068 A CN 202210031068A CN 114494161 A CN114494161 A CN 114494161A
Authority
CN
China
Prior art keywords
image
pantograph
similarity
sliding window
standard
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210031068.0A
Other languages
Chinese (zh)
Inventor
占栋
喻杨洋
张金鑫
李想
赵杰超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chengdu Tangyuan Electric Co Ltd
Original Assignee
Chengdu Tangyuan Electric Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chengdu Tangyuan Electric Co Ltd filed Critical Chengdu Tangyuan Electric Co Ltd
Priority to CN202210031068.0A priority Critical patent/CN114494161A/en
Publication of CN114494161A publication Critical patent/CN114494161A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a pantograph foreign matter detection method based on image comparison, equipment and a storage medium, and relates to the technical field of image processing and image recognition. In the invention, a non-defective standard historical image of the same train is introduced, and a local area comparison mode is carried out to detect the pantograph foreign matter; based on the similarity of the image blocks, the image blocks are considered as defects as long as the similarity is low, and the defect is irrelevant to the type of foreign matters; the detection size can be dynamically set, and small foreign matters can be detected by setting a small sliding window; the differences of the positions and angles of the pantograph in the standard image and the current image are solved in an image registration mode, so that the accuracy of subsequent similarity judgment is improved; foreign object detection of different foreign object sizes is solved through setting of the sliding window and the similarity threshold.

Description

Pantograph foreign matter detection method and device based on image contrast and storage medium
Technical Field
The present invention relates to the field of image processing and image recognition technologies, and in particular, to a pantograph foreign object detection method and apparatus based on image contrast, and a storage medium.
Background
All rail transit trains acquire electric energy from the outside through sliding contact between a train pantograph and a contact network, and the contact relation between the pantograph and the contact network is important for safe and normal operation of the trains. Due to the special contact relation and the structural characteristics of the pantograph and the contact network, the pantograph or the contact network of the operation train is not convenient to install test equipment, and once the operation train is in serious abnormal contact with the pantograph network or is attached with foreign matters, the operation train is also not convenient to check the train roof in a line section, and the train roof needs to be checked by means of special attachment equipment or the train is maintained to operate by changing the pantograph, so that the train is directly influenced to operate at a normal point.
The contact network and pantograph slide plate monitoring system (5C) is mainly used for image acquisition of a pantograph slide plate area, acquired images can be transmitted to a station duty room, a motor train section and a power supply management department, the operation condition of a pantograph can be timely mastered through checking a pantograph picture, and a defective pantograph can be timely maintained or replaced.
The existing pantograph foreign matter analysis method for a pantograph slide plate monitoring device (5C) in an image mainly directly searches for the position of foreign matters in the image in a target detection mode, as shown in figure 1. Such algorithms have certain limitations. First, due to the variety of foreign objects, the target detection algorithm can generally only identify a limited class of foreign objects, and such methods cannot detect foreign objects that are not among the detectable types of foreign objects. Secondly, such algorithms have a high detection rate for large-volume foreign matter, but are not sensitive to high-rise small-volume foreign matter (such as iron wires, feathers, plastic bag fragments) in the pantograph of the overhead line system.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention aims to provide a pantograph foreign matter detection method, device and storage medium based on image comparison, which take the current image and the standard image comparison of the same train as ideas, take image registration and deep learning twin network as tools, realize detection of pantograph foreign matters in a 5C image, and solve the defects that the foreign matter identification type is limited and the foreign matter is insensitive to small-size foreign matters.
In order to achieve the above purpose, the invention adopts the technical scheme that:
a pantograph foreign matter detection method based on image contrast comprises an image acquisition step, an image registration step, a sliding window and similarity threshold setting step, a sliding window twin network similarity calculation step, a foreign matter judging step and a pantograph foreign matter region output step;
the image acquisition step, acquiring a current image of the detected train pantograph and a historical standard image without foreign matters of the detected train pantograph by using an image acquisition device;
in this step, image acquisition equipment is located the top or the side of train, shoots through the mode of bowing or side shooting current image and standard image, preferably adopts the mode of bowing to shoot current image and standard image because for the side shooting, the pantograph angle correction that the bowing got is more favorable to carrying out the pantograph foreign matter detection based on image contrast mode, compares the position one by one of current image with the position one by one of standard image to confirm whether there is the foreign matter in each position.
Typically, at least one image of the acquired data to be analyzed contains a complete pantograph. The pre-prepared standard image is also an image containing the complete pantograph. The invention mainly compares the one-by-one position of at least one image containing the complete pantograph with the one-by-one position of a standard image through 1:1 comparison, thereby confirming whether foreign matters exist in each position.
The standard image of the invention is an image of the same train pantograph history without foreign matters. The selection of the standard image comprises an image exposure correctness judging step, an image focusing correctness judging step, a pantograph image integrity confirming step, a pantograph image definition confirming step and a pantograph image foreign matter-free confirming step, and specifically comprises the following steps:
image exposure correctness judgment: establishing histogram distribution of the image to count the number of pixels with different gray values, and judging the correctness of image exposure by using a statistical result;
image focusing correctness judgment: calculating an image frequency spectrum through two-dimensional Fourier transform, and judging whether the image is correctly focused or not according to the low-frequency component ratio in the image frequency spectrum;
confirming the integrity of the pantograph image: confirming whether the image contains a complete pantograph or not;
confirming the image definition of the pantograph: confirming whether each part of the pantograph in the image is clearly visible;
confirmation that the pantograph image contains no foreign matter: it is confirmed whether or not the image contains no foreign matter.
The foreign bodies include plastic bags, kites, branches and bird carcasses.
The image registration step is to perform image registration on the current image and the standard image acquired in the image acquisition step to obtain the current image and the standard image with the same pantograph position and size;
in the image registration step, the image registration of the current image and the standard image is performed by adopting a registration method based on SIFT feature point detection, which specifically comprises the following steps:
firstly, detecting feature points, describing the feature points and acquiring the feature points of a target by using a registration algorithm for detecting SIFT feature points of a current image, and detecting the feature points, describing the feature points and acquiring the feature points of the target by using a registration algorithm for detecting SIFT feature points of a standard image; and matching the characteristic points of the current image with the characteristic points of the standard image, and finally correcting the matched points.
Preferentially, the specific steps of SIFT feature extraction and matching are as follows:
generating a Gaussian difference pyramid (DOG) and constructing a scale space: carrying out scale transformation on original images of a current image and a standard image to obtain a scale space representation sequence under the multi-scale of the image, extracting a main contour of the scale space of the obtained sequence, and carrying out edge and corner detection and extraction of key points on different resolutions by taking the main contour as a feature vector after extraction;
and (3) detecting a spatial extreme point: comparing each pixel point with all adjacent points thereof, comparing the size of each pixel point with the adjacent points of each pixel point image domain and scale space domain, and searching for an extreme point of the DOG function;
precise positioning of the stabilization key points: further screening the searched extreme points, and removing unstable and erroneously detected extreme points; simultaneously, enabling an extreme point extracted from the down-sampled image in the Gaussian difference pyramid to correspond to the exact position in the original image;
in the above steps, the DOG value is sensitive to noise and edges, so local extreme points detected in the scale space of step 2 need to be further screened to remove unstable and erroneously detected extreme points, and another point is that a down-sampled image is adopted in the process of constructing the gaussian pyramid, and the extreme points extracted from the down-sampled image correspond to the exact position in the original image, which is also the problem to be solved in the step.
And (3) stable key point direction information distribution: distributing key point directions by utilizing the gradient of each extreme point;
in the above steps, stable extreme points are extracted under different scale spaces, which ensures the scale invariance of the key points. The problem to be solved by assigning direction information to the keypoints is to make the keypoints invariant to image angle and rotation. The allocation of the direction is achieved by graduating each extreme point.
For any key point, the gradient magnitude is expressed as:
Figure DEST_PATH_IMAGE001
，
the gradient direction is as follows:
Figure 978399DEST_PATH_IMAGE002
，
description of key points: partitioning a pixel region around the key point, calculating an intra-block gradient histogram, and generating a unique vector;
matching the characteristic points: and calculating Euclidean distances of 128-dimensional key points of the two groups of feature points, comparing the Euclidean distances with a threshold value, wherein the smaller the Euclidean distances are, the higher the similarity is, and when the Euclidean distances are smaller than the set threshold value, the matching is judged to be successful.
Setting the sliding window and a similarity threshold, namely setting the sizes of the sliding windows of the current image and the standard image after the image registration and the similarity threshold of the images in the sliding windows;
in the step, the registered standard image and the current image have the same size, and the size of the sliding window is set, so that the local area of the image with the size of the sliding window is intercepted and the local area on the image is traversed in the subsequent sliding window twin network similarity calculation step; by setting the similarity threshold, in the subsequent step of judging whether foreign matters exist, the similarity of the image local area calculated in the twin network is compared with the similarity threshold to judge whether the image local area is similar to the similarity threshold, and then whether foreign matters exist is judged.
The window size determines the minimum detected foreign object size detected. The similarity threshold determines the tolerance level of the foreign object. The higher the similarity threshold, the more likely the paired sliding window images are deemed to be dissimilar, i.e., the current image is deemed to contain foreign objects within the current sliding window.
In the invention, after the image registration process is completed, the current image and the standard image have the same size, and according to statistics, the average resolution of common foreign matter types in the image acquired by the 5C equipment is 100 multiplied by 100 pixels, so that under the default condition, the size of a sliding window is 100 multiplied by 100 pixels, and the similarity threshold value is 0.8 to carry out subsequent similarity judgment.
The 100 pixels × 100 pixels mentioned above refers to the sliding window size in the default case. If the finer foreign matters need to be detected, the size of the sliding window needs to be reduced, otherwise, the size of the sliding window is increased. Similarly, a similarity threshold of 0.8 is also the default case, and decreasing or increasing (not more than 1) also changes the sensitivity of the algorithm.
The sliding window twin network similarity calculation step is that image local regions with consistent positions and the size of the sliding window are intercepted from the current image and the standard image after the image registration in an overlapped sliding window mode, and the intercepted image local regions are input into the twin network in pairs to calculate the image local region similarity;
manner of overlapping sliding windows: the window is a part of a rectangular area in the image, the sliding window is another part of the rectangular area in the selected image, and the overlapped sliding window is that the current rectangular area is overlapped with the rectangular area before sliding.
Preferably, the twin network comprises a first neural network and a second neural network which have the same structure and weight and are independent of each other, the first neural network and the second neural network respectively receive the input of the image local regions intercepted by the current image and the standard image, respectively map the input to a new space, and finally obtain the similarity of the two inputs in the new space as the similarity of the image local regions through Loss calculation.
Preferably, in the twin network, the cosine similarity is used to calculate the similarity between the two images, as follows:
Figure DEST_PATH_IMAGE003
，
wherein, X1、X2A sliding window for respectively representing the current image and a sliding window for the standard image, and f (X) a characteristic vector obtained after the image passes through the twin network.
Judging whether foreign matters exist or not, namely judging whether the similarity of the local area of the image obtained in the sliding window twin network similarity calculation step is smaller than the similarity threshold set in the sliding window and similarity threshold setting step or not, judging whether the local areas of the current image and the standard image are similar or not according to the judgment result, and judging whether foreign matters exist in the local area of the current image or not according to the similarity result or not;
preferably, the determining method is that when the similarity is smaller than the similarity threshold, the local area of the current image is not similar to the local area of the standard image, and a foreign object exists in the local area of the current image; and when the similarity is greater than or equal to the similarity threshold, the local areas of the current image and the standard image are similar, and no foreign matter exists in the local area of the current image.
And the pantograph foreign matter region output step is to repeat the sliding window twin network similarity calculation step and the step of judging whether foreign matters exist in the rest local regions of the current image and the standard image, judge whether foreign matters exist in the rest local regions of the current image, merge the local regions with the judgment result that the foreign matters exist, and output and display the merged local regions.
In the invention, the registered standard image and the current image are divided into a plurality of local areas with the size of a sliding window, each local area is traversed to carry out a sliding window twin network similarity calculation step and a step of judging whether foreign matters exist, whether foreign matters exist in each local area is judged, and the corresponding current image is returned and output and displayed in the current image for the local area with the judging result of the existence of the foreign matters.
And for the foreign matters which have large areas and occupy a plurality of local areas, the local areas are connected and combined in the current image, and the image after the connection and combination is output and displayed in the current image. And for the foreign matters with small areas and occupying only one local area, directly outputting and displaying the foreign matters in the current image.
A computer apparatus comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the computer program to perform steps in a pantograph foreign object detection method.
A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps in a pantograph foreign object detection method.
The invention has the beneficial effects that:
1. in the prior art, a comparison mode is not adopted, and an image in a standard normal state of a train pantograph can be used as a reference; in the invention, the pantograph foreign bodies are detected by introducing the same standard historical image of the train without defects and carrying out local area comparison.
2. In the prior art, only limited kinds of foreign matters can be detected due to the principle limitation of the target detection method; the invention is based on the similarity of image blocks, and the image blocks are considered as defects as long as the similarity is low, and are irrelevant to the types of foreign matters.
3. In the prior art, the principle of the target detection method is limited, so that the detection size of the foreign matter is required; the invention can dynamically set the foreign matter detection size, and can detect small foreign matters by setting the small sliding window.
4. According to the method, the difference between the positions and the angles of the pantograph in the standard image and the current image is solved in an image registration mode, so that the accuracy of subsequent similarity judgment is improved.
5. According to the invention, through setting of the sliding window and the similarity threshold, foreign object detection with different sizes of foreign objects, different tolerance degrees and no object type distinction is solved.
6. In the invention, the similarity judgment is realized by introducing the twin network.
Drawings
FIG. 1 is a flow chart of the prior art;
FIG. 2 is a flow chart of the present invention;
FIG. 3 is a schematic view of a portion of the present invention;
FIG. 4 is a schematic diagram illustrating the determination of similarity according to the present invention;
FIG. 5 is a diagram of the determination results of all local regions of the current image according to the present invention;
FIG. 6 is a combined view of the adjacent local areas in FIG. 5 where foreign matter is present;
FIG. 7 is a flowchart of SIFT feature point detection based image registration according to the present invention;
FIG. 8 is a schematic diagram of the basic mechanism of the twin network of the present invention.
Detailed Description
The conception, the specific structure and the technical effects of the present invention will be clearly and completely described below in conjunction with the embodiments and the accompanying drawings to fully understand the objects, the features and the effects of the present invention.
Example 1
A pantograph foreign matter detection method based on image comparison is shown in fig. 2 and comprises an image acquisition step, an image registration step, a sliding window and similarity threshold setting step, a sliding window twin network similarity calculation step, a foreign matter judging step and a pantograph foreign matter region output step.
S1, image acquisition
And in the image acquisition step, the current image of the pantograph of the detected train is acquired by using an image acquisition device, and a pre-collected historical foreign-matter-free standard image of the pantograph, which corresponds to the detected train, is retrieved from a database according to the basic data such as the number information and the current position of the coming train.
In order to identify foreign objects of indefinite size and indefinite type, 5C pantograph images to be analyzed need to be collected and prepared in advance for the same train history 5C pantograph images (hereinafter referred to as standard images). The history image is an image without any foreign matter and defect. The historical 5C image may be compared with the current image to be analyzed (hereinafter referred to as the current image) position by position as a standard defect-free image, so as to determine whether a foreign object exists at each position.
S2, image registration
And the image registration step is to perform image registration on the current image and the standard image acquired in the image acquisition step to obtain the current image and the standard image with the same pantograph position and size.
Since the image capturing devices for capturing the standard image and the current image are not necessarily the same image capturing device, the installation positions of different image capturing devices may cause imaging differences, and in order to eliminate the imaging differences caused by the above differences, the positions and sizes of the pantographs of the standard image and the current image need to be unified in an image registration manner.
S3 setting of sliding window and similarity threshold
And the step of setting the sliding window and the similarity threshold value, namely setting the sizes of the sliding windows of the current image and the standard image after the image registration and the similarity threshold value of the images in the sliding window.
The registered standard image and the current image have the same size, and the size of the sliding window and the similarity threshold of the images in the sliding window are set so as to facilitate the subsequent calculation of the similarity between the two windows compared by the standard image and the current image, and a schematic diagram of the sliding window is shown in fig. 3.
S4, sliding window twin network similarity calculation
And the sliding window twin network similarity calculation step is to intercept image local regions with consistent positions and the size of the sliding window from the current image and the standard image after the image registration in an overlapped sliding window mode, and input the intercepted image local regions into the twin network in pairs to calculate the image local region similarity.
S5, judging whether foreign matter exists or not
The step of determining whether foreign matters exist, the step of determining whether the similarity of the local area of the image obtained in the sliding window twin network similarity calculation step is smaller than the similarity threshold set in the sliding window and similarity threshold setting step, the step of determining whether the local areas of the current image and the standard image are similar according to the determination result, and the step of determining whether foreign matters exist in the local area of the current image according to the result of whether the local areas of the current image are similar, as shown in fig. 4.
S6 pantograph foreign matter region output
And the pantograph foreign matter region output step is to repeat the sliding window twin network similarity calculation step and the step of judging whether foreign matters exist in the rest local regions of the current image and the standard image, judge whether foreign matters exist in the rest local regions of the current image, merge the local regions with the judgment result that the foreign matters exist, and output and display the merged local regions.
In this embodiment, the standard image and the current image after registration are divided into local areas with the size of a plurality of sliding windows, each local area is traversed to perform a sliding window twin network similarity calculation step and a step of determining whether a foreign object exists, whether a foreign object exists in each local area is determined, and for the local area in which the foreign object exists as a determination result, the current image corresponding to the local area is returned and output and displayed in the current image.
And for the foreign matters which have large areas and occupy a plurality of local areas, the local areas are connected and combined in the current image, and the image after the connection and combination is output and displayed in the current image. Local region similarity determination as shown in fig. 5, each box represents a local region, the thin line box represents that the similarity is above the threshold, and the thick line box represents that the similarity is below the threshold. The combined output is shown in fig. 6.
And for the foreign matters with small areas and occupying only one local area, directly outputting and displaying the foreign matters in the current image.
Example 2
This embodiment further illustrates the image capturing step based on embodiment 1. In the image acquisition step, image acquisition equipment is located the top or the side of train, shoots through the mode of bowing or side shoot current image and standard image, preferably adopt the mode of bowing to shoot current image and standard image because for the side is clapped, the pantograph angle correction that the bowing obtained more does benefit to and carries out the pantograph foreign matter detection based on image contrast mode, compares the position one by one of current image and standard image's position one by one to confirm whether there is the foreign matter in each position.
Typically, at least one image of the acquired data to be analyzed contains a complete pantograph. The pre-prepared standard image is also an image containing the complete pantograph. In this embodiment, the positions of at least one image including a complete pantograph are compared with the positions of a standard image one by a 1:1 comparison, so as to determine whether a foreign object exists at each position.
The standard image of the present embodiment is an image of the same train pantograph history without a foreign object. The selection of the standard image comprises an image exposure correctness judging step, an image focusing correctness judging step, a pantograph image integrity confirming step, a pantograph image definition confirming step and a pantograph image foreign matter-free confirming step, and specifically comprises the following steps:
1. image exposure correctness judgment: establishing histogram distribution of the image to count the number of pixels with different gray values, and judging the correctness of image exposure by using a statistical result;
2. image focusing correctness judgment: calculating an image frequency spectrum through two-dimensional Fourier transform, and judging whether the image is correctly focused or not according to the low-frequency component ratio in the image frequency spectrum;
3. confirming the integrity of the pantograph image: confirming whether the image contains a complete pantograph or not;
4. confirming the image definition of the pantograph: confirming whether each part of the pantograph in the image is clearly visible;
5. confirmation that the pantograph image contains no foreign matter: it is confirmed whether or not the image contains no foreign matter. The foreign bodies include plastic bags, kites, branches and bird carcasses.
Example 3
This embodiment further explains the image registration step on the basis of embodiment 2. In the image registration step, the image registration of the current image and the standard image is performed by using a registration method based on SIFT feature point detection, as shown in fig. 7, specifically as follows:
firstly, detecting feature points, describing the feature points and acquiring the feature points of a target by using a registration algorithm for detecting SIFT feature points of a current image, and detecting the feature points, describing the feature points and acquiring the feature points of the target by using a registration algorithm for detecting SIFT feature points of a standard image; and matching the characteristic points of the current image with the characteristic points of the standard image, and finally correcting the matched points.
The specific steps of SIFT feature extraction and matching are as follows:
1. generating a Gaussian difference pyramid (DOG) and constructing a scale space: carrying out scale transformation on the original images of the current image and the standard image to obtain a scale space representation sequence under the multi-scale of the image, extracting a main contour of the scale space of the obtained sequence, and carrying out edge and corner detection and extraction of key points on different resolutions by taking the main contour as a feature vector after extraction.
2. And (3) detecting a spatial extreme point: and comparing each pixel point with all adjacent points of the pixel point, comparing the size of each pixel point with the adjacent points of each pixel point in the image domain and the scale space domain, and searching for the extreme point of the DOG function.
3. Precise positioning of the stabilization key points: further screening the searched extreme points, and removing unstable and erroneously detected extreme points; and simultaneously, corresponding the extreme point extracted from the down-sampled image in the Gaussian difference pyramid to the exact position in the original image.
In the above steps, the DOG value is sensitive to noise and edges, so local extreme points detected in the scale space of step 2 need to be further screened to remove unstable and erroneously detected extreme points, and another point is that a down-sampled image is adopted in the process of constructing the gaussian pyramid, and the extreme points extracted from the down-sampled image correspond to the exact position in the original image, which is also the problem to be solved in the step.
4. And (3) stable key point direction information distribution: the keypoint direction is assigned using the gradient of each extreme point.
In the above steps, stable extreme points are extracted under different scale spaces, which ensures the scale invariance of the key points. The problem to be solved by assigning direction information to the keypoints is to make the keypoints invariant to image angle and rotation. The allocation of the direction is achieved by graduating each extreme point.
For any key point, the gradient magnitude is expressed as:
Figure 186657DEST_PATH_IMAGE001
，
the gradient direction is as follows:
Figure 797767DEST_PATH_IMAGE002
，
5. description of key points: and partitioning a pixel region around the key point, calculating an intra-block gradient histogram, and generating a unique vector which is an abstract expression of the image information of the region.
6. Matching the characteristic points: and calculating Euclidean distances of 128-dimensional key points of the two groups of feature points, comparing the Euclidean distances with a threshold value, wherein the smaller the Euclidean distances are, the higher the similarity is, and when the Euclidean distances are smaller than the set threshold value, the matching is judged to be successful.
Example 4
In this embodiment, on the basis of embodiment 2, the sliding window twin network similarity calculation step is further described. The twin network is used as a neural network structure for calculating similarity, and the basic structure is shown in fig. 8.
In the sliding window twin network similarity calculation step, the twin network comprises a first neural network and a second neural network which have the same structure and weight and are independent of each other, the first neural network and the second neural network respectively receive the input of the image local regions intercepted by the current image and the standard image, respectively map the input to a new space, and finally, the similarity of the two inputs in the new space is obtained through Loss calculation and is used as the similarity of the image local regions.
In the twin network, the cosine similarity is adopted to calculate the similarity of the two images, as follows:
Figure 89071DEST_PATH_IMAGE004
，
wherein, X1、X2A sliding window for respectively representing the current image and a sliding window for the standard image, and f (X) a characteristic vector obtained after the image passes through the twin network.
Example 5
This embodiment further explains the step of determining whether or not a foreign object is present on the basis of embodiment 4. In the step of judging whether foreign matters exist, the judging method is that when the similarity is smaller than the similarity threshold, the local areas of the current image and the standard image are not similar, and foreign matters exist in the local area of the current image; and when the similarity is greater than or equal to the similarity threshold, the local areas of the current image and the standard image are similar, and no foreign matter exists in the local area of the current image.
Example 6
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to perform the steps of any one of the pantograph foreign object detection methods of the embodiments 1-5.
The processor may be a Central Processing Unit (CPU) in this embodiment. The Processor may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or a combination thereof.
The memory, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and units. The processor executes various functional applications of the processor and the processing of the work data by executing the non-transitory software programs, instructions and modules stored in the memory, that is, implementing the method in the above embodiments.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor, and the like. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory may optionally include memory located remotely from the processor, and such remote memory may be coupled to the processor via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
One or more units are stored in the memory and when executed by the processor perform the method of any of embodiments 1-5 above.
Example 7
A computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the steps in the pantograph foreign object detection method of any one of embodiments 1 to 5 above.
The embodiments of the present invention have been described in detail, but the present invention is not limited to the embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention, and the equivalents or substitutions are included in the scope of the present invention defined by the claims.

Claims (9)

1. A pantograph foreign matter detection method based on image contrast is characterized by comprising the following steps: the method comprises the steps of image acquisition, image registration, sliding window and similarity threshold setting, sliding window twin network similarity calculation, foreign matter judgment and pantograph foreign matter area output;
the image acquisition step, acquiring a current image of the detected train pantograph and a historical standard image without foreign matters of the detected train pantograph by using an image acquisition device;
the image registration step is to perform image registration on the current image and the standard image acquired in the image acquisition step to obtain the current image and the standard image with the same pantograph position and size;
setting the sliding window and a similarity threshold, namely setting the sizes of the sliding windows of the current image and the standard image after the image registration and the similarity threshold of the images in the sliding windows;
the sliding window twin network similarity calculation step is that image local regions with consistent positions and the size of the sliding window are intercepted from the current image and the standard image after the image registration in an overlapped sliding window mode, and the intercepted image local regions are input into the twin network in pairs to calculate the image local region similarity;
judging whether foreign matters exist or not, namely judging whether the similarity of the local area of the image obtained in the sliding window twin network similarity calculation step is smaller than the similarity threshold set in the sliding window and similarity threshold setting step or not, judging whether the local areas of the current image and the standard image are similar or not according to the judgment result, and judging whether foreign matters exist in the local area of the current image or not according to the similarity result or not;
and the pantograph foreign matter region output step is to repeat the sliding window twin network similarity calculation step and the step of judging whether foreign matters exist in the rest local regions of the current image and the standard image, judge whether foreign matters exist in the rest local regions of the current image, merge the local regions with the judgment result that the foreign matters exist, and output and display the merged local regions.
2. The pantograph foreign object detection method according to claim 1, wherein: in the image acquisition step, the image acquisition equipment is positioned above the train, and the current image and the standard image are shot in a way of overhead shooting.
3. The pantograph foreign object detection method according to claim 1, wherein: in the image acquisition step, the selection of the standard image comprises an image exposure correctness judgment step, an image focusing correctness judgment step, a pantograph image integrity confirmation step, a pantograph image definition confirmation step and a pantograph image foreign matter-free confirmation step, and specifically comprises the following steps:
image exposure correctness judgment: establishing histogram distribution of the image to count the number of pixels with different gray values, and judging the correctness of image exposure by using a statistical result;
image focusing correctness judgment: calculating an image frequency spectrum through two-dimensional Fourier transform, and judging whether the image is correctly focused or not according to the low-frequency component ratio in the image frequency spectrum;
confirming the integrity of the pantograph image: confirming whether the image contains a complete pantograph or not;
confirming the image definition of the pantograph: confirming whether each part of the pantograph in the image is clearly visible;
confirmation that the pantograph image contains no foreign matter: it is confirmed whether or not the image contains no foreign matter.
4. The pantograph foreign object detection method according to claim 1, wherein: in the image registration step, the image registration of the current image and the standard image is performed by adopting a registration method based on SIFT feature point detection, which specifically comprises the following steps:
firstly, detecting feature points, describing the feature points and acquiring the feature points of a target by using a registration algorithm for detecting SIFT feature points of a current image, and detecting the feature points, describing the feature points and acquiring the feature points of the target by using a registration algorithm for detecting SIFT feature points of a standard image; and matching the characteristic points of the current image with the characteristic points of the standard image, and finally correcting the matched points.
5. The pantograph foreign object detection method according to claim 1, wherein: in the sliding window twin network similarity calculation step, the twin network comprises a first neural network and a second neural network which have the same structure and weight and are independent of each other, the first neural network and the second neural network respectively receive the input of the image local regions intercepted by the current image and the standard image, respectively map the input to a new space, and finally, the similarity of the two inputs in the new space is obtained through Loss calculation and is used as the similarity of the image local regions.
6. The pantograph foreign object detection method according to claim 5, wherein: in the twin network of the sliding window twin network similarity calculation step, the cosine similarity is adopted to calculate the similarity of the two images, as follows:
Figure 909054DEST_PATH_IMAGE001
，
wherein, X1、X2A sliding window for respectively representing the current image and a sliding window for the standard image, and f (X) a characteristic vector obtained after the image passes through the twin network.
7. The pantograph foreign object detection method according to claim 1, wherein: in the step of judging whether foreign matters exist, the judging method is that when the similarity is smaller than the similarity threshold, the local areas of the current image and the standard image are not similar, and foreign matters exist in the local area of the current image; and when the similarity is greater than or equal to the similarity threshold, the local areas of the current image and the standard image are similar, and no foreign matter exists in the local area of the current image.
8. A computer device, characterized by: comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the steps of the pantograph foreign object detection method according to any one of the preceding claims 1 to 7.
9. A computer-readable storage medium characterized by: stored thereon, a computer program which, when being executed by a processor, carries out the steps of the pantograph foreign object detection method according to any one of the preceding claims 1 to 7.
CN202210031068.0A 2022-01-12 2022-01-12 Pantograph foreign matter detection method and device based on image contrast and storage medium Pending CN114494161A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210031068.0A CN114494161A (en) 2022-01-12 2022-01-12 Pantograph foreign matter detection method and device based on image contrast and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210031068.0A CN114494161A (en) 2022-01-12 2022-01-12 Pantograph foreign matter detection method and device based on image contrast and storage medium

Publications (1)

Publication Number Publication Date
CN114494161A true CN114494161A (en) 2022-05-13

Family

ID=81511510

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210031068.0A Pending CN114494161A (en) 2022-01-12 2022-01-12 Pantograph foreign matter detection method and device based on image contrast and storage medium

Country Status (1)

Country Link
CN (1) CN114494161A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115601719A (en) * 2022-12-13 2023-01-13 中铁十二局集团有限公司（Cn） Climbing robot and method for detecting invasion of foreign objects in subway tunnel
CN116228698A (en) * 2023-02-20 2023-06-06 北京鹰之眼智能健康科技有限公司 Filler state detection method based on image processing
CN116523852A (en) * 2023-04-13 2023-08-01 成都飞机工业（集团）有限责任公司 Foreign matter detection method of carbon fiber composite material based on feature matching

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115601719A (en) * 2022-12-13 2023-01-13 中铁十二局集团有限公司（Cn） Climbing robot and method for detecting invasion of foreign objects in subway tunnel
CN116228698A (en) * 2023-02-20 2023-06-06 北京鹰之眼智能健康科技有限公司 Filler state detection method based on image processing
CN116228698B (en) * 2023-02-20 2023-10-27 北京鹰之眼智能健康科技有限公司 Filler state detection method based on image processing
CN116523852A (en) * 2023-04-13 2023-08-01 成都飞机工业（集团）有限责任公司 Foreign matter detection method of carbon fiber composite material based on feature matching

Similar Documents

Publication Publication Date Title
CN110414507B (en) License plate recognition method and device, computer equipment and storage medium
CN114494161A (en) Pantograph foreign matter detection method and device based on image contrast and storage medium
CN112419299B (en) Bolt missing detection method, device, equipment and storage medium
US10643332B2 (en) Method of vehicle image comparison and system thereof
KR102505545B1 (en) Vehicle image comparison method and system therefor
US9208172B2 (en) Method and system for vehicle identification
WO2018049704A1 (en) Vehicle detection, tracking and localization based on enhanced anti-perspective transformation
CN111666848B (en) Method, device and equipment for detecting arrival of transport vehicle and storage medium
CN104573680A (en) Image detection method, image detection device and traffic violation detection system
Sun et al. A robust lane detection method for autonomous car-like robot
US10599946B2 (en) System and method for detecting change using ontology based saliency
CN104463238B (en) A kind of automobile logo identification method and system
US20230342937A1 (en) Vehicle image analysis
CN114494373A (en) High-precision rail alignment method and system based on target detection and image registration
CN109523570A (en) Beginning parameter transform model method and device
CN117671299A (en) Loop detection method, device, equipment and storage medium
CN112287905A (en) Vehicle damage identification method, device, equipment and storage medium
CN112308061B (en) License plate character recognition method and device
CN115984759A (en) Substation switch state identification method and device, computer equipment and storage medium
CN113255405B (en) Parking space line identification method and system, parking space line identification equipment and storage medium
KR101934317B1 (en) System for automatic registration of images using association analysis of linear features
CN114581365B (en) Bolt detection model construction and bolt loosening detection method
CN110781758A (en) Dynamic video monitoring method and device for abnormal pantograph structure
Kim et al. Character segmentation method for a License plate with topological transform
CN117351439B (en) Dynamic monitoring management system for intelligent expressway overrun vehicle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination