CN114310935B - Track state detection system based on inspection robot, robot and method - Google Patents
Track state detection system based on inspection robot, robot and method Download PDFInfo
- Publication number
- CN114310935B CN114310935B CN202111547549.9A CN202111547549A CN114310935B CN 114310935 B CN114310935 B CN 114310935B CN 202111547549 A CN202111547549 A CN 202111547549A CN 114310935 B CN114310935 B CN 114310935B
- Authority
- CN
- China
- Prior art keywords
- convolution
- convolution block
- return wave
- layer
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Investigating Or Analyzing Materials By The Use Of Ultrasonic Waves (AREA)
Abstract
The invention relates to a track state detection system based on a patrol robot, the robot and a method, wherein the system comprises: inspection robot, inspection robot includes: the multi-path acquisition subsystem is used for acquiring track images and flaw detection return wave signals, and the ranging subsystem is used for measuring the relative distance information between each station; the positioning subsystem is used for acquiring interval positioning information and combining the relative distance information acquired by the ranging subsystem to acquire accurate positioning information; and the primary judging subsystem is used for processing and analyzing through the track image and flaw detection return wave signals and outputting a primary state judging result by combining with the accurate positioning information. The invention provides a scheme for rapidly judging the track state at the robot side, ensures that the inspection robot can rapidly find track defects and judge defect types under the condition of poor communication, and simultaneously provides a more perfect defect judging scheme at the system side so as to carry out recheck on the judgment of the robot side.
Description
Technical Field
The invention relates to the technical field of track state detection, in particular to a track state detection system based on a patrol robot, a robot and a method.
Background
With rapid development of rail transit, maintenance work of a rail line is increasingly important. In many inspection scenes, not only is the communication environment bad, but also the safety and health problems of inspection personnel cannot be guaranteed, and the intelligent detection requirements of the track line cannot be met gradually by means of a manual detection traditional method, so that the intelligent inspection robot system is applied to the application scenes of the transformer substation, the underground cable and the overhead line, the danger of the inspection personnel is reduced, the labor cost is reduced, and the automation degree of a power grid is greatly improved.
However, in the case of poor communication conditions, it is difficult to accurately judge the track state by using the remote control system, and even if the control analysis device is considered to be mounted on the robot side, the electric quantity of the inspection robot is not only greatly stressed, the inspection progress is seriously affected, but also the originally tense inspection time is longer than the fly-catching elbow due to overlong calculation and judgment time.
Disclosure of Invention
First, the technical problem to be solved
In view of the above-mentioned shortcomings and disadvantages of the prior art, the invention provides a track state detection system, a robot and a method based on a patrol robot, which solve the technical problem that the conventional patrol robot is difficult to rapidly judge faults when communication conditions are bad.
(II) technical scheme
In order to achieve the above purpose, the main technical scheme adopted by the invention comprises the following steps:
in a first aspect, an embodiment of the present invention provides a track state detection system based on a patrol robot, where a plurality of stations providing position information are provided on a track, the track state detection system includes: the inspection robot runs on the track;
the inspection robot includes:
the multi-path acquisition subsystem is used for acquiring track images and flaw detection return wave signals;
the distance measuring subsystem is used for measuring the relative distance information between each station;
the positioning subsystem is used for interactively acquiring interval positioning information with the station and combining the relative distance information acquired by the ranging subsystem to acquire accurate positioning information;
the primary judging subsystem is used for processing and analyzing the track image through a preset lightweight SSD model and processing and analyzing the flaw detection return wave signal through a preset ultrasonic rapid identification model; based on the image classification result and the return wave signal classification result, and outputting a preliminary state judgment result by combining the accurate positioning information.
Optionally, the multi-path acquisition subsystem includes:
the ultrasonic wave acquisition module is used for acquiring a return ultrasonic wave signal of the track;
the image acquisition module is used for acquiring images of the track surface, the welding or the connecting position;
optionally, the ultrasonic acquisition module is an ultrasonic combined probe comprising a plurality of ultrasonic sensors; the image acquisition module is a CCD camera component.
Optionally, the lightweight SSD model includes: a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a sixth convolution block, a seventh convolution block, an eighth convolution block, and a category output module;
the first convolution block includes: a convolution layer with a convolution kernel of 3×3, a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the second convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the third convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the fourth convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the fifth convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the sixth convolution block includes: the convolution kernel is a convolution layer of 3×3 and a maximum pooling layer;
the seventh convolution block includes: the convolution kernel is a 1×1 convolution layer;
the eighth convolution block includes: the convolution kernel is a 1×1 convolution layer and a 3×3 convolution layer;
the class output module is used for carrying out feature fusion on the received output feature graphs of the fourth convolution block, the seventh convolution block and the eighth convolution block and outputting a classification result.
Optionally, the light SSD model is provided with an attention mechanism module at the output ends of the fourth convolution block, the seventh convolution block and the eighth convolution block;
the attention mechanism module satisfies the following formula:
y＝f(x,w i )+sub(x)
wherein the output characteristic diagram of each convolution block is x, f (x, w i ) In order to convolve the output characteristic diagram, the convolution times are i and less than or equal to 5, and sub (x) is to downsample the output characteristic diagram.
Optionally, the ultrasonic rapid identification model includes:
the preprocessing module is used for preprocessing the flaw detection return wave signals;
the window level dividing module is used for dividing the preprocessed flaw detection return wave signal into a plurality of window levels according to time phases;
the category judging module is used for calculating the similarity between a plurality of window levels and a preset category time sequence and outputting a return wave signal classification result based on the similarity;
the calculating of the similarity between the window levels and the preset category time curve is performed by the following formula:
X[i]＝k×x[i]+c
wherein x [ i ] is the ith window level, Y [ i ] is a preset category time sequence, and k is not equal to 0.
Optionally, the track state detection system further includes: the control analysis platform is connected with the inspection robot and is used for receiving the collected track images, flaw detection return wave signals and preliminary state judgment results;
an SSD model and a long and short time memory neural network are arranged on the control analysis platform;
the SSD model is based on the light SSD model, and further comprises: a ninth convolution block, a tenth convolution block, and an eleventh convolution block;
the SSD model outputs an image classification result through feature fusion based on the received output feature graphs of the fourth convolution block, the seventh convolution block, the eighth convolution block, the ninth convolution block, the tenth convolution block and the eleventh convolution block;
the long-short-term memory neural network carries out return wave classification results on the flaw detection return wave signals;
and the control analysis platform compares the image classification result and the return wave classification result with the preliminary state judgment result after normalization, and if the difference value is larger than a preset threshold value, the control analysis platform takes the image classification result and the return wave classification result output by the control analysis platform as the final state judgment result and prompts the inspection robot.
Optionally, the return wave classification result of the flaw detection return wave signal by the long-short-term memory neural network includes:
performing wavelet noise reduction treatment on the flaw detection return wave signal;
extracting a multidimensional feature vector of the flaw detection return wave signal subjected to wavelet noise reduction treatment through a short-time Fourier algorithm;
based on the multidimensional feature vector, a long-short-term memory neural network is adopted to carry out return wave classification results on the flaw detection return wave signals.
In a second aspect, the present invention provides a patrol robot for detecting a state of a track, the patrol robot comprising:
the multi-path acquisition subsystem is used for acquiring track images and flaw detection return wave signals;
the distance measuring subsystem is used for measuring the relative distance information between each station;
the positioning subsystem is used for interactively acquiring interval positioning information with the station and combining the relative distance information acquired by the ranging subsystem to acquire accurate positioning information;
the primary judging subsystem is used for processing and analyzing the track image through a preset lightweight SSD model and processing and analyzing the flaw detection return wave signal through a preset ultrasonic rapid identification model; based on the image classification result and the return wave signal classification result, and outputting a preliminary state judgment result by combining the accurate positioning information.
In a third aspect, an embodiment of the present invention provides a method for detecting a track state based on a patrol robot, where a plurality of stations for providing position information are provided on a track, the method includes:
acquiring a track image through an image acquisition module, acquiring flaw detection return wave signals through an ultrasonic acquisition module, acquiring interval positioning information through interaction between a near field communication reading module and the stations, and acquiring relative distance information between the stations through a distance measurement module;
processing and analyzing the track image through a preset lightweight SSD model, and processing and analyzing the flaw detection return wave signal through a preset ultrasonic rapid identification model; based on the image classification result and the return wave signal classification result, outputting a preliminary state judgment result by combining the accurate positioning information;
processing and analyzing the track image through a preset SSD model to obtain an image classification result, and processing and analyzing the flaw detection return wave signal through a preset long-short-time memory neural network to obtain a return wave classification result;
and comparing the image classification result and the return wave classification result with the preliminary state judgment result after normalization, and if the difference value is larger than a preset threshold value, taking the image classification result and the return wave classification result output by the control analysis platform as the final state judgment result and prompting the inspection robot.
(III) beneficial effects
The beneficial effects of the invention are as follows: the invention provides a scheme for rapidly judging the track state at the robot side, ensures that the inspection robot can rapidly find track defects and judge defect types under the condition of poor communication, and simultaneously provides a more perfect defect judgment scheme at the system side, and the judgment result at the robot side is rechecked under the condition of good communication or disconnection recovery communication, thereby improving the judgment accuracy.
Drawings
Fig. 1 is a schematic diagram of a track state detection system based on a patrol robot;
fig. 2 is a schematic diagram of a light SSD model of a track state detection system based on a patrol robot;
fig. 3 is a schematic flow chart of a track state detection method based on a patrol robot.
Detailed Description
The invention will be better explained for understanding by referring to the following detailed description of the embodiments in conjunction with the accompanying drawings.
As shown in fig. 1, a track state detection system based on a patrol robot according to an embodiment of the present invention is provided with a plurality of stations for providing position information on a track, where the track state detection system includes: and the inspection robot runs on the track. The inspection robot includes: the multi-path acquisition subsystem is used for acquiring track images and flaw detection return wave signals; the distance measuring subsystem is used for measuring the relative distance information between each station; the positioning subsystem is used for interactively acquiring interval positioning information with the station and combining the relative distance information acquired by the ranging subsystem to acquire accurate positioning information; the primary judging subsystem is used for processing and analyzing the track image through a preset lightweight SSD model and processing and analyzing the flaw detection return wave signal through a preset ultrasonic rapid identification model; based on the image classification result and the return wave signal classification result, and outputting a preliminary state judgment result by combining the accurate positioning information.
The invention provides a scheme for rapidly judging the track state at the robot side, ensures that the inspection robot can rapidly find track defects and judge defect types under the condition of poor communication, and simultaneously provides a more perfect defect judgment scheme at the system side, and the judgment result at the robot side is rechecked under the condition of good communication or disconnection recovery communication, thereby improving the judgment accuracy.
In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Second, the multiplex acquisition subsystem includes: the ultrasonic wave acquisition module is used for acquiring a return ultrasonic wave signal of the track; the image acquisition module is used for acquiring images of the track surface, the welding or the connecting position; the ultrasonic acquisition module is an ultrasonic combined probe comprising a plurality of ultrasonic sensors; the image acquisition module is a CCD camera component. A visible light photoelectric coupling element (CCD) acquires an infrared imaging image of a field captured picture and equipment, preprocessing is carried out by a correlation analysis method, the color and information of the image are corrected, and the image is enhanced and noise is reduced.
The ranging subsystem then comprises an encoder and an inertial measurement unit, the encoder calculating the displacement of the robot relative to the station by collecting the pulse variations over the period, but this method suffers from accumulated errors, is not suitable for positioning over long distances of the robot and therefore requires re-metering at each pass of a station. The inertial measurement unit internally comprises a triaxial accelerometer and a triaxial gyroscope, so that triaxial acceleration and triaxial angular velocity of the robot can be output, and position change and speed can be obtained after resolving. Although the positioning method based on the inertial measurement unit has better short-time precision, drift can be generated with the increase of time, and smaller constant errors can become extremely large after multiple integration.
Then, as shown in fig. 2, the lightweight SSD model includes: a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a sixth convolution block, a seventh convolution block, an eighth convolution block, and a category output module;
the first convolution block includes: a convolution layer with a convolution kernel of 3×3, a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the second convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the third convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the fourth convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the fifth convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the sixth convolution block includes: the convolution kernel is a convolution layer of 3×3 and a maximum pooling layer;
the seventh convolution block includes: the convolution kernel is a 1×1 convolution layer;
the eighth convolution block includes: the convolution kernel is a 1×1 convolution layer and a 3×3 convolution layer;
and the class output module is used for carrying out feature fusion on the received output feature graphs of the fourth convolution block, the seventh convolution block and the eighth convolution block and outputting a classification result.
Further, the light SSD model is provided with attention mechanism modules at the output ends of the fourth convolution block, the seventh convolution block and the eighth convolution block;
the attention mechanism module satisfies the following formula:
y＝f(x,w i )+sub(x)
wherein the output characteristic diagram of each convolution block is x, f (x, w i ) In order to convolve the output characteristic diagram, the convolution times are i and less than or equal to 5, and sub (x) is to downsample the output characteristic diagram.
Furthermore, the ultrasonic rapid recognition model includes:
the preprocessing module is used for preprocessing flaw detection return wave signals;
the window level dividing module is used for dividing the preprocessed flaw detection return wave signal into a plurality of window levels according to time phases;
the category judging module is used for calculating the similarity between a plurality of window levels and a preset category time sequence and outputting a return wave signal classification result based on the similarity;
the similarity between the window levels and the preset category time curve is calculated through the following formula:
X[i]＝k×x[i]+c
wherein x [ i ] is the ith window level, and Y [ i ] is a preset category time sequence. Since the time series has a time attribute, the similarity measurement cannot be directly performed on the window level, and the following transformation is required for the window level:
x [ i ] =k×x [ i ] +c, k+.0, c is any real number.
Further, the track state detection system further includes: and the control analysis platform is connected with the inspection robot and is used for receiving the collected track image, flaw detection return wave signals and preliminary state judgment results.
And an SSD model and a long and short time memory neural network are arranged on the control analysis platform.
The SSD model is based on the lightweight SSD model, and further comprises: ninth convolution block, tenth convolution block, and eleventh convolution block.
The SSD model outputs an image classification result through feature fusion based on the received output feature maps of the fourth, seventh, eighth, ninth, tenth, and eleventh convolution blocks.
And the long-time and short-time memory neural network performs return wave classification results on the flaw detection return wave signals.
And the control analysis platform compares the normalized image classification result and the normalized return wave classification result with the preliminary state judgment result, and takes the image classification result and the return wave classification result output by the control analysis platform as the final state judgment result and prompts the inspection robot if the difference value is larger than a preset threshold value.
Further, the return wave classification result of the flaw detection return wave signal by the long-short-term memory neural network comprises the following steps: firstly, carrying out wavelet noise reduction treatment on flaw detection return wave signals; secondly, extracting a multidimensional feature vector of the flaw detection return wave signal subjected to wavelet noise reduction treatment through a short-time Fourier algorithm; and then, based on the multidimensional feature vector, adopting a long-short-term memory neural network to carry out return wave classification results on the flaw detection return wave signals.
In addition, an embodiment of the present invention further provides a patrol robot for detecting a track state, including:
and the multipath acquisition subsystem is used for acquiring the track image and flaw detection return wave signals.
And the distance measuring subsystem is used for measuring the relative distance information between the distance measuring subsystem and each station.
And the positioning subsystem is used for interactively acquiring interval positioning information with the station and combining the relative distance information acquired by the ranging subsystem to acquire accurate positioning information.
The primary judging subsystem is used for processing and analyzing the track image through a preset lightweight SSD model and processing and analyzing the flaw detection return wave signal through a preset ultrasonic rapid identification model; based on the image classification result and the return wave signal classification result, and outputting a preliminary state judgment result by combining the accurate positioning information.
As shown in fig. 3, the present invention provides a track state detection method based on a patrol robot, wherein a plurality of stations for providing position information are arranged on a track, and the method comprises:
s1, acquiring a track image through an image acquisition module, acquiring flaw detection return wave signals through an ultrasonic acquisition module, acquiring interval positioning information through interaction between a near field communication reading module and stations, and acquiring relative distance information between the ultrasonic acquisition module and each station through a distance measurement module.
S2, processing and analyzing the track image through a preset lightweight SSD model, and processing and analyzing flaw detection return wave signals through a preset ultrasonic rapid identification model; based on the image classification result and the return wave signal classification result, and outputting a preliminary state judgment result by combining the accurate positioning information.
S3, processing and analyzing the track image through a preset SSD model to obtain an image classification result, and processing and analyzing the flaw detection return wave signal through a preset long-short-time memory neural network to obtain a return wave classification result.
And S4, normalizing the image classification result and the return wave classification result with the preliminary state judgment result, comparing, and if the difference value is larger than a preset threshold value, taking the image classification result and the return wave classification result output by the control analysis platform as the final state judgment result and prompting the inspection robot.
In summary, the invention provides a track state detection system, a robot and a method based on a patrol robot, wherein the patrol robot is provided with a lightweight SSD model and an ultrasonic rapid recognition model which are used for preferentially calculating the speed under the condition of ensuring certain calculation accuracy, and a preliminary state judgment result is obtained rapidly through the two models, so that the detection efficiency is greatly improved; in the control analysis platform, the invention provides a more perfect judgment scheme, which processes the track image and the flaw detection return wave signal respectively through an SSD model and a long and short time memory neural network, so as to ensure the accuracy of judgment.
In the description of the present invention, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium; may be a communication between two elements or an interaction between two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
In the present invention, unless expressly stated or limited otherwise, a first feature is "on" or "under" a second feature, which may be in direct contact with the first and second features, or in indirect contact with the first and second features via an intervening medium. Moreover, a first feature "above," "over" and "on" a second feature may be a first feature directly above or obliquely above the second feature, or simply indicate that the first feature is higher in level than the second feature. The first feature being "under", "below" and "beneath" the second feature may be the first feature being directly under or obliquely below the second feature, or simply indicating that the first feature is level lower than the second feature.
In the description of the present specification, the terms "one embodiment," "some embodiments," "examples," "particular examples," or "some examples," etc., refer to particular features, structures, materials, or characteristics described in connection with the embodiment or example as being included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
While embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that alterations, modifications, substitutions and variations may be made in the above embodiments by those skilled in the art within the scope of the invention.
Claims (4)
1. Track state detecting system based on inspection robot, characterized by that is equipped with a plurality of stations that provide positional information on the track, track state detecting system includes: the inspection robot runs on the track;
the inspection robot includes:
the multi-path acquisition subsystem is used for acquiring track images and flaw detection return wave signals; the multi-channel acquisition subsystem comprises: the ultrasonic wave acquisition module is used for acquiring a return ultrasonic wave signal of the track; the image acquisition module is used for acquiring images of the track surface, the welding or the connecting position;
the distance measuring subsystem is used for measuring the relative distance information between each station;
the positioning subsystem is used for interactively acquiring interval positioning information with the station and combining the relative distance information acquired by the ranging subsystem to acquire accurate positioning information;
the primary judging subsystem is used for processing and analyzing the track image through a preset lightweight SSD model and processing and analyzing the flaw detection return wave signal through a preset ultrasonic rapid identification model; based on the image classification result and the return wave signal classification result, outputting a preliminary state judgment result by combining the accurate positioning information;
the lightweight SSD model includes: a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a sixth convolution block, a seventh convolution block, an eighth convolution block, and a category output module;
the first convolution block includes: a convolution layer with a convolution kernel of 3×3, a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the second convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the third convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the fourth convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the fifth convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the sixth convolution block includes: the convolution kernel is a convolution layer of 3×3 and a maximum pooling layer;
the seventh convolution block includes: the convolution kernel is a 1×1 convolution layer;
the eighth convolution block includes: the convolution kernel is a 1×1 convolution layer and a 3×3 convolution layer;
the class output module is used for carrying out feature fusion on the received output feature graphs of the fourth convolution block, the seventh convolution block and the eighth convolution block and outputting a classification result;
the light SSD model is provided with attention mechanism modules at the output ends of the fourth convolution block, the seventh convolution block and the eighth convolution block;
the attention mechanism module satisfies the following formula:
y＝f(x,w i )+sub(x)
wherein the output characteristic diagram of each convolution block is x, f (x, w i ) In order to convolve the output characteristic diagram, the convolution times are i and less than or equal to 5, and sub (x) is to downsample the output characteristic diagram;
the ultrasonic rapid identification model comprises:
the preprocessing module is used for preprocessing the flaw detection return wave signals;
the window level dividing module is used for dividing the preprocessed flaw detection return wave signal into a plurality of window levels according to time phases;
the category judging module is used for calculating the similarity between a plurality of window levels and a preset category time sequence and outputting a return wave signal classification result based on the similarity;
the calculating of the similarity between the window levels and the preset category time curve is performed by the following formula:
X[i]＝k×x[i]+c
wherein x [ i ] is the ith window level, Y [ i ] is a preset category time sequence, k is not equal to 0, and c is any real number;
the track state detection system further includes: the control analysis platform is connected with the inspection robot and is used for receiving the collected track images, flaw detection return wave signals and preliminary state judgment results;
an SSD model and a long and short time memory neural network are arranged on the control analysis platform;
the SSD model is based on the light SSD model, and further comprises: a ninth convolution block, a tenth convolution block, and an eleventh convolution block;
the SSD model outputs an image classification result through feature fusion based on the received output feature graphs of the fourth convolution block, the seventh convolution block, the eighth convolution block, the ninth convolution block, the tenth convolution block and the eleventh convolution block;
the long-short-term memory neural network carries out return wave classification results on the flaw detection return wave signals;
the control analysis platform compares the image classification result and the return wave classification result with the preliminary state judgment result after normalization, and if the difference value is larger than a preset threshold value, the image classification result and the return wave classification result output by the control analysis platform are used as final state judgment results and prompt the inspection robot;
the long-short-term memory neural network performs return wave classification results on the flaw detection return wave signals, and the return wave classification results comprise:
performing wavelet noise reduction treatment on the flaw detection return wave signal;
extracting a multidimensional feature vector of the flaw detection return wave signal subjected to wavelet noise reduction treatment through a short-time Fourier algorithm;
based on the multidimensional feature vector, a long-short-term memory neural network is adopted to carry out return wave classification results on the flaw detection return wave signals.
2. The inspection robot-based track state detection system of claim 1, wherein the ultrasonic acquisition module is an ultrasonic combined probe comprising a plurality of ultrasonic sensors; the image acquisition module is a CCD camera component.
3. A robot for inspection for detecting a track condition, comprising:
the multi-path acquisition subsystem is used for acquiring track images and flaw detection return wave signals; the multi-channel acquisition subsystem comprises: the ultrasonic wave acquisition module is used for acquiring a return ultrasonic wave signal of the track; the image acquisition module is used for acquiring images of the track surface, the welding or the connecting position;
the distance measuring subsystem is used for measuring the relative distance information between each station;
the positioning subsystem is used for interactively acquiring interval positioning information with the station and combining the relative distance information acquired by the ranging subsystem to acquire accurate positioning information;
the primary judging subsystem is used for processing and analyzing the track image through a preset lightweight SSD model and processing and analyzing the flaw detection return wave signal through a preset ultrasonic rapid identification model; based on the image classification result and the return wave signal classification result, outputting a preliminary state judgment result by combining the accurate positioning information;
the lightweight SSD model includes: a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a sixth convolution block, a seventh convolution block, an eighth convolution block, and a category output module;
the first convolution block includes: a convolution layer with a convolution kernel of 3×3, a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the second convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the third convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the fourth convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the fifth convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the sixth convolution block includes: the convolution kernel is a convolution layer of 3×3 and a maximum pooling layer;
the seventh convolution block includes: the convolution kernel is a 1×1 convolution layer;
the eighth convolution block includes: the convolution kernel is a 1×1 convolution layer and a 3×3 convolution layer;
the class output module is used for carrying out feature fusion on the received output feature graphs of the fourth convolution block, the seventh convolution block and the eighth convolution block and outputting a classification result;
the light SSD model is provided with attention mechanism modules at the output ends of the fourth convolution block, the seventh convolution block and the eighth convolution block;
the attention mechanism module satisfies the following formula:
y＝f(x,w i )+sub(x)
wherein the output characteristic diagram of each convolution block is x, f (x, w i ) In order to convolve the output characteristic diagram, the convolution times are i and less than or equal to 5, and sub (x) is to downsample the output characteristic diagram;
the ultrasonic rapid identification model comprises:
the preprocessing module is used for preprocessing the flaw detection return wave signals;
the window level dividing module is used for dividing the preprocessed flaw detection return wave signal into a plurality of window levels according to time phases;
the category judging module is used for calculating the similarity between a plurality of window levels and a preset category time sequence and outputting a return wave signal classification result based on the similarity;
the calculating of the similarity between the window levels and the preset category time curve is performed by the following formula:
X[i]＝k×x[i]+c
wherein x [ i ] is the ith window level, Y [ i ] is a preset category time sequence, k is not equal to 0, and c is any real number;
the inspection robot is used for being connected with a preset control analysis platform, and the control analysis platform is used for receiving the collected track images, flaw detection return wave signals and preliminary state judgment results;
an SSD model and a long and short time memory neural network are arranged on the control analysis platform;
the SSD model is based on the light SSD model, and further comprises: a ninth convolution block, a tenth convolution block, and an eleventh convolution block;
the SSD model outputs an image classification result through feature fusion based on the received output feature graphs of the fourth convolution block, the seventh convolution block, the eighth convolution block, the ninth convolution block, the tenth convolution block and the eleventh convolution block;
the long-short-term memory neural network carries out return wave classification results on the flaw detection return wave signals;
the control analysis platform compares the image classification result and the return wave classification result with the preliminary state judgment result after normalization, and if the difference value is larger than a preset threshold value, the image classification result and the return wave classification result output by the control analysis platform are used as final state judgment results and prompt the inspection robot;
the long-short-term memory neural network performs return wave classification results on the flaw detection return wave signals, and the return wave classification results comprise:
performing wavelet noise reduction treatment on the flaw detection return wave signal;
extracting a multidimensional feature vector of the flaw detection return wave signal subjected to wavelet noise reduction treatment through a short-time Fourier algorithm;
based on the multidimensional feature vector, a long-short-term memory neural network is adopted to carry out return wave classification results on the flaw detection return wave signals.
4. The track state detection method based on the inspection robot is characterized in that a plurality of stations for providing position information are arranged on a track, and the method comprises the following steps:
the control analysis platform sequentially acquires a track image through the image acquisition module, acquires flaw detection return wave signals through the ultrasonic acquisition module, acquires interval positioning information through interaction of the near field communication reading module and the stations, and acquires relative distance information between the stations through the distance measurement module;
the control analysis platform processes and analyzes the track image through a preset lightweight SSD model, and processes and analyzes the flaw detection return wave signal through a preset ultrasonic rapid identification model; based on the image classification result and the return wave signal classification result, combining accurate positioning information obtained according to the interval positioning information and the relative distance information to output a preliminary state judgment result;
the control analysis platform processes and analyzes the track images through a preset SSD model to obtain image classification results, and processes and analyzes the flaw detection return wave signals through a preset long-short-time memory neural network to obtain return wave classification results;
the control analysis platform compares the image classification result and the return wave classification result with the preliminary state judgment result after normalization, and if the difference value is larger than a preset threshold value, the image classification result and the return wave classification result are used as final state judgment results and prompt the inspection robot;
the lightweight SSD model includes: a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a sixth convolution block, a seventh convolution block, an eighth convolution block, and a category output module;
the first convolution block includes: a convolution layer with a convolution kernel of 3×3, a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the second convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the third convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the fourth convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the fifth convolution block includes: a depth separable convolution layer with a convolution kernel of 3×3, a convolution layer with a convolution kernel of 1×1, and a maximum pooling layer;
the sixth convolution block includes: the convolution kernel is a convolution layer of 3×3 and a maximum pooling layer;
the seventh convolution block includes: the convolution kernel is a 1×1 convolution layer;
the eighth convolution block includes: the convolution kernel is a 1×1 convolution layer and a 3×3 convolution layer;
the class output module is used for carrying out feature fusion on the received output feature graphs of the fourth convolution block, the seventh convolution block and the eighth convolution block and outputting a classification result;
the light SSD model is provided with attention mechanism modules at the output ends of the fourth convolution block, the seventh convolution block and the eighth convolution block;
the attention mechanism module satisfies the following formula:
y＝f(x,w i )+sub(x)
wherein the output characteristic diagram of each convolution block is x, f (x, w i ) In order to convolve the output characteristic diagram, the convolution times are i and less than or equal to 5, and sub (x) is to downsample the output characteristic diagram;
the ultrasonic rapid identification model comprises:
the preprocessing module is used for preprocessing the flaw detection return wave signals;
the window level dividing module is used for dividing the preprocessed flaw detection return wave signal into a plurality of window levels according to time phases;
the category judging module is used for calculating the similarity between a plurality of window levels and a preset category time sequence and outputting a return wave signal classification result based on the similarity;
the calculating of the similarity between the window levels and the preset category time curve is performed by the following formula:
X[i]＝k×x[i]+c
wherein x [ i ] is the ith window level, Y [ i ] is a preset category time sequence, k is not equal to 0, and c is any real number;
the track state detection system further includes: the control analysis platform is connected with the inspection robot and is used for receiving the collected track images, flaw detection return wave signals and preliminary state judgment results;
an SSD model and a long and short time memory neural network are arranged on the control analysis platform;
the SSD model is based on the light SSD model, and further comprises: a ninth convolution block, a tenth convolution block, and an eleventh convolution block;
the SSD model outputs an image classification result through feature fusion based on the received output feature graphs of the fourth convolution block, the seventh convolution block, the eighth convolution block, the ninth convolution block, the tenth convolution block and the eleventh convolution block;
the long-short-term memory neural network carries out return wave classification results on the flaw detection return wave signals;
the control analysis platform compares the image classification result and the return wave classification result with the preliminary state judgment result after normalization, and if the difference value is larger than a preset threshold value, the image classification result and the return wave classification result output by the control analysis platform are used as final state judgment results and prompt the inspection robot;
the long-short-term memory neural network performs return wave classification results on the flaw detection return wave signals, and the return wave classification results comprise:
performing wavelet noise reduction treatment on the flaw detection return wave signal;
extracting a multidimensional feature vector of the flaw detection return wave signal subjected to wavelet noise reduction treatment through a short-time Fourier algorithm;
based on the multidimensional feature vector, a long-short-term memory neural network is adopted to carry out return wave classification results on the flaw detection return wave signals.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111547549.9A CN114310935B (en) | 2021-12-16 | 2021-12-16 | Track state detection system based on inspection robot, robot and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111547549.9A CN114310935B (en) | 2021-12-16 | 2021-12-16 | Track state detection system based on inspection robot, robot and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114310935A CN114310935A (en) | 2022-04-12 |
| CN114310935B true CN114310935B (en) | 2023-08-04 |
Family
ID=81052680
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111547549.9A Active CN114310935B (en) | 2021-12-16 | 2021-12-16 | Track state detection system based on inspection robot, robot and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114310935B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR3105148B1 (en) * | 2019-12-23 | 2023-10-06 | Commissariat Energie Atomique | SYSTEM AND METHOD FOR DETECTING A DEFECT IN A RAIL OF A RAILWAY |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108229580A (en) * | 2018-01-26 | 2018-06-29 | 浙江大学 | Sugared net ranking of features device in a kind of eyeground figure based on attention mechanism and Fusion Features |
| CN110796643A (en) * | 2019-10-18 | 2020-02-14 | 四川大学 | Rail fastener defect detection method and system |
| CN111024728A (en) * | 2019-12-26 | 2020-04-17 | 江西交通职业技术学院 | Railway detection method and system based on computer vision and ultrasonic flaw detection |
| CN112308855A (en) * | 2020-11-27 | 2021-02-02 | 株洲时代电子技术有限公司 | Rail damage recognition model generation device, damage detection device and system |
| CN112884753A (en) * | 2021-03-10 | 2021-06-01 | 杭州申昊科技股份有限公司 | Track fastener detection and classification method based on convolutional neural network |
| WO2021130036A1 (en) * | 2019-12-23 | 2021-07-01 | Commissariat A L'energie Atomique Et Aux Energies Alternatives | System and method for detecting a defect in a railway track rail |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106250850B (en) * | 2016-07-29 | 2020-02-21 | 深圳市优必选科技有限公司 | Face detection tracking method and device, and robot head rotation control method and system |
| CN110569846A (en) * | 2019-09-16 | 2019-12-13 | 北京百度网讯科技有限公司 | Image character recognition method, device, equipment and storage medium |
-
2021
- 2021-12-16 CN CN202111547549.9A patent/CN114310935B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108229580A (en) * | 2018-01-26 | 2018-06-29 | 浙江大学 | Sugared net ranking of features device in a kind of eyeground figure based on attention mechanism and Fusion Features |
| CN110796643A (en) * | 2019-10-18 | 2020-02-14 | 四川大学 | Rail fastener defect detection method and system |
| WO2021130036A1 (en) * | 2019-12-23 | 2021-07-01 | Commissariat A L'energie Atomique Et Aux Energies Alternatives | System and method for detecting a defect in a railway track rail |
| CN111024728A (en) * | 2019-12-26 | 2020-04-17 | 江西交通职业技术学院 | Railway detection method and system based on computer vision and ultrasonic flaw detection |
| CN112308855A (en) * | 2020-11-27 | 2021-02-02 | 株洲时代电子技术有限公司 | Rail damage recognition model generation device, damage detection device and system |
| CN112884753A (en) * | 2021-03-10 | 2021-06-01 | 杭州申昊科技股份有限公司 | Track fastener detection and classification method based on convolutional neural network |
Non-Patent Citations (3)
| Title |
|---|
| ASSD: attentive single shot multibox detector;Jingru Yi;《Computer Vision and Image Understanding》;第1-9页 * |
| 基于深度学习的图像目标检测算法综述;张婷婷;《电信科学》;第36卷(第7期);第92-106页 * |
| 高速轨道超声成像伤损检测及其参数学习方法;吴福培;《计算机集成制造系统》;第27卷(第3期);第747-756页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114310935A (en) | 2022-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110889455B (en) | Fault detection positioning and safety assessment method for chemical engineering garden inspection robot | |
| CN102565625B (en) | Method for intelligently diagnosing thermal defects of high-voltage transmission line based on infrared image | |
| CN103837087B (en) | Pantograph automatic testing method based on active shape model | |
| CN106679567A (en) | Contact net and strut geometric parameter detecting measuring system based on binocular stereoscopic vision | |
| CN206034212U (en) | Railway rails elastic tensioning clamp state automatic checkout device | |
| CN114310935B (en) | Track state detection system based on inspection robot, robot and method | |
| CN112686938A (en) | Electric transmission line clear distance calculation and safety warning method based on binocular image ranging | |
| CN110222648A (en) | A kind of aerial cable fault recognition method and device | |
| CN107727658A (en) | Pantograph crackle on-line measuring device and method based on image procossing | |
| CN112802004B (en) | Portable intelligent video detection device for health of power transmission line and pole tower | |
| CN108981565A (en) | A kind of municipal rail train pantograph disalignment monitoring device and method | |
| CN113884011A (en) | Non-contact concrete surface crack measuring equipment and method | |
| CN110435689B (en) | Detection method of intelligent obstacle detection vehicle based on modular control | |
| CN115331000A (en) | ORB algorithm-based bow net running state detection method | |
| CN117498225B (en) | Unmanned aerial vehicle intelligent power line inspection system | |
| CN116630267A (en) | Roadbed settlement monitoring method based on unmanned aerial vehicle and laser radar data fusion | |
| CN113567451A (en) | Cable defect detection and diameter measurement method | |
| Ahrary et al. | An autonomous sewer robots navigation based on stereo camera information | |
| CN112258398A (en) | Conveyor belt longitudinal tearing detection device and method based on TOF and binocular image fusion | |
| CN115409691A (en) | Bimodal learning slope risk detection method integrating laser ranging and monitoring image | |
| CN116543317A (en) | Noise reduction method for detecting bridge cable vibration frequency by unmanned aerial vehicle | |
| CN115857040A (en) | Dynamic visual detection device and method for foreign matters on locomotive roof | |
| CN107194923B (en) | Ultraviolet image diagnosis method for defect inspection of contact network power equipment | |
| CN106679630B (en) | A kind of contact net positioner slope detection system | |
| CN112270357A (en) | VIO vision system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |