CN106023076B - The method of the damage condition of the protective fence of the joining method and detection high-speed railway of panorama sketch - Google Patents
The method of the damage condition of the protective fence of the joining method and detection high-speed railway of panorama sketch Download PDFInfo
- Publication number
- CN106023076B CN106023076B CN201610312137.XA CN201610312137A CN106023076B CN 106023076 B CN106023076 B CN 106023076B CN 201610312137 A CN201610312137 A CN 201610312137A CN 106023076 B CN106023076 B CN 106023076B
- Authority
- CN
- China
- Prior art keywords
- panorama
- image
- railway
- video
- sampling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 230000001681 protective effect Effects 0.000 title claims abstract description 51
- 238000001514 detection method Methods 0.000 title abstract description 50
- 230000007547 defect Effects 0.000 claims abstract description 31
- 230000033001 locomotion Effects 0.000 claims description 57
- 238000005070 sampling Methods 0.000 claims description 51
- 230000011218 segmentation Effects 0.000 claims description 34
- 229910000831 Steel Inorganic materials 0.000 claims description 8
- 230000007613 environmental effect Effects 0.000 claims description 8
- 239000010959 steel Substances 0.000 claims description 8
- 238000013442 quality metrics Methods 0.000 abstract 1
- 238000004422 calculation algorithm Methods 0.000 description 15
- 238000004364 calculation method Methods 0.000 description 8
- 238000000605 extraction Methods 0.000 description 8
- 230000003287 optical effect Effects 0.000 description 8
- 238000012544 monitoring process Methods 0.000 description 6
- 238000007689 inspection Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000004888 barrier function Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000002829 reductive effect Effects 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 239000004568 cement Substances 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000009545 invasion Effects 0.000 description 3
- 230000036961 partial effect Effects 0.000 description 3
- XEEYBQQBJWHFJM-UHFFFAOYSA-N Iron Chemical compound [Fe] XEEYBQQBJWHFJM-UHFFFAOYSA-N 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000002950 deficient Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000001788 irregular Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000001808 coupling effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 229910052742 iron Inorganic materials 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000007789 sealing Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000011410 subtraction method Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30221—Sports video; Sports image
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention provides a kind of methods of Panoramagram montage method based on virtual sampling channel model and the damage condition of the protective fence of the detection high-speed railway of Panoramagram montage method.The detection method, comprising: obtain the propulsion video of record high-speed railway ambient condition；It is the left and right panorama sketch of railway operation environment by the propulsion Video Quality Metric, as protective fence panorama sketch based on the Panoramagram montage method of virtual sampling channel model；Divide principle according to entropy is maximized, the position of guardrail and background in the protective fence panorama sketch is positioned, obtains the two-value code of 0 and 1 composition；Run-length encoding is carried out to the two-value code；According to the run-length encoding, the pixel wide D of the background area between adjacent two guardrail is calculatedcur；According to the pixel wide D of the background area between adjacent two guardrailcurWith the normal pixel spacing d between adjacent two guardrail, judge adjacent two guardrail with the presence or absence of defect.
Description
Technical Field
The invention relates to the technical field of computer application, in particular to a panorama splicing method and a method for detecting the defect state of a protective guard of a high-speed railway.
Background
Safety is the basis and precondition for the development of high speed railways. Factors affecting the safety of high-speed railways are many, and people, trains, tracks and running environments and coupling effects among the people, the trains, the tracks and the running environments are involved. In order to ensure the driving safety of the high-speed railway, the operating environment of the high-speed railway needs to be closed in the whole process. A protective guard is additionally arranged along a high-speed railway line, which is a means for realizing a closed environment.
In order to monitor the human damage or the climbing invasion into the guard railing and the sound condition of the infrastructures such as the signal equipment, the power supply system and the like in the closed environment, at present, the fixed video equipment is mainly adopted for monitoring. For example, on the jinghu high-speed railway, more than 400 video monitoring devices are installed at the positions of the jinghu high-speed railway, such as roadbed, intersection, bridge, highway bridge, throat area and the like, so as to ensure the safe operation of vehicles.
In a typical video monitoring system, an image pickup apparatus is in a stationary state, and a monitored object may be in a moving state. A commonly used method for detecting a moving object by using the relative motion between the object and the background in a video sequence is a method for detecting a moving object, which mainly includes: background subtraction, time difference, Optical flow (Optical flow), and the like. The method has the advantages that training is not needed, and online detection can be carried out; the disadvantage is that only the corresponding target can be found, and the specific target in the detection area can not be known, and further judgment at a later stage is needed.
The background subtraction method mainly depends on the accuracy of modeling the background image, but in an actual environment, due to the influence of factors such as environmental changes, the difficulty of extracting and updating the background is increased, and the difficulty is increased for accurately extracting the target. Therefore, in the current methods, a statistical learning method is mostly adopted to analyze and model the background of continuous video frames, the background model is updated on line along with the time, and then the motion region is detected through the difference between the current frame and the background model.
The time difference method is a relatively simple method with small calculation amount, and is characterized in that the pixel difference of corresponding positions of two frames before and after a video sequence is calculated, if the pixel difference is greater than a set threshold value, the pixel is regarded as a target pixel, and otherwise, the pixel is regarded as a background pixel. The optical flow method detects a moving object by using a characteristic that a vector of the moving object changes with time. The advantage of this method is that moving objects can be detected even in the case of a moving camera. Its main disadvantages are: the method is sensitive to noise, large in calculation amount and not suitable for occasions with high real-time requirements.
However, the fixed point monitoring mode is limited by the acquisition field of vision, and cannot control all conditions along the line and the line, so that the method for monitoring the front line environment and the states of the guard railings on two sides along the line by using the vehicle end environment monitoring equipment installed on the high-speed comprehensive detection train is an effective method.
The detection task influencing the train operation safety is completed internationally by adopting a special high-speed comprehensive detection train. Detecting content generally includes: contact line geometry, contact line wear, pantograph-catenary effects, electrical parameters, gauge, track geometry, rail section and wave wear, car body and axle box acceleration, wheel-rail forces, tracks and bogies, communication detection and positioning, and the like. In addition, video equipment serving as a driving recorder is arranged at the front part and the rear part of the high-speed comprehensive detection train at home and abroad, and is used for acquiring the environment condition information along the way in a forward motion video mode, so that a basis is provided for manually detecting whether the environment along the railway has an abnormal state or not at a later stage. How to rapidly acquire information which influences the environmental sealing and the abnormal limit invasion of foreign matters or equipment in a line from a large number of acquired video images and perform correct early warning is a problem to be solved urgently by a high-speed railway.
Disclosure of Invention
The embodiment of the invention provides a panorama splicing method and a method for detecting the defect state of a protective guard of a high-speed railway, which are used for carrying out lossless information extraction on massive video data in a lightweight panorama format and reducing the storage and access expenses of a video data form.
In order to achieve the purpose, the invention adopts the following technical scheme:
a panorama splicing method based on a virtual sampling channel model comprises the following steps:
acquiring a forward motion video for recording the environmental state of the high-speed railway; extracting a video image sequence with the number of frames N from the forward motion video;
setting an external sampling rectangle OR in each frame image according to the railway scene structure determined by the vanishing pointm；
An internal sampling rectangle IR is set in each frame image according to the speed of the trainm；
The external sampling rectangle OR to be composed of each frame imagemAnd the internal sampling rectangle IRmA rectangular ring-shaped sampling ring band formed by dividingSplicing the areas S for four stripst,Sb,Sl,Sr；
Splicing the four strips of each frame image into a region St,Sb,Sl,SrCorrected to regular rectangular strips S by image wrappingt',Sb',Sl',Sr'；
4 XN corrected rectangular strips St',Sb',Sl',Sr' the panoramic images of 4 planes of the railway scene are generated by respectively splicing the sampling planes.
A method for detecting the defect state of a protective guard of a high-speed railway based on the panorama splicing method comprises the following steps:
acquiring a forward motion video for recording the environmental state of the high-speed railway;
secondly, converting the forward motion video into a left panorama and a right panorama of a railway running environment based on a panorama splicing method of a virtual sampling channel model to serve as a protective fence panorama;
thirdly, positioning the positions of the guard rails and the background in the guard rail panoramic image according to a maximum entropy segmentation principle, wherein the position where F (j) is 1 represents the position of the guard rails, and the position where F (j) is 0 represents the position of the background, so as to obtain a binary code consisting of 0 and 1; j is the column sequence number of the protective fence panorama;
step four, the run length coding is carried out on the binary code;
step five, calculating the pixel width D of the background area between two adjacent guardrails according to the stroke codescur；
Step six, according to the pixel width D of the background area between the two adjacent guardrailscurJudging whether the two adjacent guardrails have defects or not by the normal pixel distance d between the two adjacent guardrails, and generating a judgment result;
and step seven, outputting the judgment result.
According to the technical scheme provided by the embodiment of the invention, massive video data is subjected to lossless information extraction in a lightweight panoramic image format, so that the storage and access expenses of a video data form are reduced.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.
FIG. 1 is a schematic flow chart of a panorama stitching method based on a virtual sampling channel model according to the present invention;
FIG. 2 is a method for detecting a defect state of a guard rail of a high-speed railway based on a panorama stitching method according to the present invention;
FIG. 3 is a high speed railway scene sampling ring geometry of the present invention;
FIG. 4 illustrates image registration between two adjacent frames according to the present invention;
FIG. 5 is a histogram distribution of three-dimensional features F (j) of pixel columns in a panorama according to the present invention;
FIG. 6 is a mean-variance-gradient based thresholding in accordance with the present invention;
FIG. 7 is a run-length encoded representation of a guardrail panorama according to the present invention;
FIG. 8 is a perspective view of the present invention generated from a closed railroad environment: (a) a left panorama; (b) a right panorama; (c) a bottom panorama; (d) upper panorama
Fig. 9 is a video image acquired by the comprehensive detection train in the present invention: (a) the guard bar has no defect; (b) the guardrail has defects
FIG. 10 is a partial panoramic view of a railway according to the present invention: (a) a panorama of the left side of the railway; (b) a rail guard panorama taken from (a)
Fig. 11 is a binarization segmentation result of the guardrail panorama in the invention: (a) dividing a maximum entropy threshold of a gray one-dimensional histogram; (b) gray-gradient (GLGM) two-dimensional histogram maximum entropy threshold segmentation; (c) the invention provides a three-dimensional histogram maximum entropy threshold segmentation combining gray mean-variance-gradient
Fig. 12 is a binarization segmentation result of a high-speed rail guardrail panorama according to the invention: (a) a guardrail panorama of a high-speed railway; (b) dividing a maximum entropy threshold of a gray one-dimensional histogram; (c) gray-gradient (GLGM) two-dimensional histogram maximum entropy threshold segmentation; (d) the invention provides a three-dimensional histogram maximum entropy threshold segmentation combining gray mean-variance-gradient
FIG. 13 is a flow chart of steps of a rapid detection method for defects of a protective fence of a high-speed railway according to the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
As shown in fig. 1, a method for stitching a panorama based on a virtual sampling channel model according to the present invention includes:
step 11, acquiring a forward motion video for recording the environmental state of the high-speed railway; extracting a video image sequence with the number of frames N from the forward motion video;
step 12, according to the railway scene structure determined by the vanishing point, setting an external sampling rectangle OR in each frame imagem；
Step 13, setting an internal sampling rectangle IR in each frame of image according to the speed of the trainm；
Step 14, OR the rectangles sampled from the outside of each frame imagemAnd the internal sampling rectangle IRmThe formed rectangular annular sampling ring band is divided into four band splicing areas St,Sb,Sl,Sr；
Step 15, splicing the four strip splicing regions S of each frame of imaget,Sb,Sl,SrCorrected to regular rectangular strips S by image wrappingt',Sb',Sl',Sr'；
Step 16, correcting 4 XN corrected rectangular strips St',Sb',Sl',Sr' the panoramic images of 4 planes of the railway scene are generated by respectively splicing the sampling planes.
The step of determining the vanishing point specifically comprises:
the coordinate of the vanishing point in the image coordinate system is (x)0,y0)T(ii) a The analytic expression of the ith line segment is x + kiy+bi＝0，kiIs the slope of the ith line segment, biIs the intercept; weight wiIs the length of the ith line segment; n is the total column number of pixels in the protective guard panoramic image;
wherein,
in the embodiment of the invention, the lossless information extraction is carried out on the massive video data in the light-weight panoramic image format, so that the storage and access expenses of the video data form are reduced, and the method can be used for subsequent processing.
As shown in fig. 2, the method for detecting a defect state of a guard rail of a high-speed railway based on a panorama stitching method according to the present invention includes:
step 21, acquiring a forward motion video for recording the environmental state of the high-speed railway;
step 22, converting the forward motion video into a left panorama and a right panorama of a railway running environment based on a panorama splicing method of a virtual sampling channel model to be used as a protective fence panorama;
step 23, according to the maximum entropy division principle, positioning the positions of the guard rail and the background in the guard rail panoramic image, wherein the position where f (j) is 1 represents the position of the guard rail, and the position where f (j) is 0 represents the position of the background, so as to obtain a binary code consisting of 0 and 1; j is the column sequence number of the protective fence panorama;
step 24, performing run length coding on the binary code;
step 25, calculating the pixel width D of the background area between two adjacent guardrails according to the stroke codescur；
26, according to the pixel width D of the background area between the two adjacent guardrailscurAnd between the two adjacent guardrailsJudging whether the two adjacent guardrails have defects or not according to the normal pixel interval d, and generating a judgment result; wherein the step 26 comprises: judgment of DcurAnd whether k is greater than k.d, wherein k is an empirical tuning parameter.
And 27, outputting the judgment result.
In the embodiment of the invention, massive video data is subjected to lossless information extraction in a lightweight panorama format, so that the storage and access expenses of the video data form are reduced, and the video is converted into a form more suitable for manual inspection and computer analysis processing, so that the distortion-free rapid conversion from the forward motion video to the panorama is solved.
The step 22 includes:
extracting a video image sequence with the number of frames N from the forward motion video;
setting an external sampling rectangle OR in each frame image according to the railway scene structure determined by the vanishing pointm；
An internal sampling rectangle IR is set in each frame image according to the speed of the trainm；
Rectangular OR of each frame image from the external samplesmAnd the internal sampling rectangle IRmThe formed rectangular annular sampling ring band is divided into four band splicing areas St,Sb,Sl,Sr(ii) a Wherein S ist,Sb,Sl,SrSky, steel rail, left guardrail and right guardrail areas;
splicing the four strips of each frame image into a region St,Sb,Sl,SrCorrected to regular rectangular strips S by image wrappingt',Sb',Sl',Sr'；
2 XN corrected rectangular strips Sl',Sr' are respectively spliced according to respective sampling planesAnd generating a left panorama and a right panorama of the railway scene.
The step of determining the vanishing point specifically comprises:
the coordinate of the vanishing point in the image coordinate system is (x)0,y0)T(ii) a The analytic expression of the ith line segment is x + kiy+bi＝0，kiIs the slope of the ith line segment, biIs the intercept; weight wiIs the length of the ith line segment; n is the total column number of pixels in the protective guard panoramic image;
wherein,
the step 23 includes:
calculating a gray mean M (1, j), a standard deviation V (1, j) and a gradient mean G (1, j) for each column j of the protective fence panorama; j is 1, …, N; n is the total column number of pixels in the protective guard panoramic image;
calculating a segmentation threshold (. epsilon.) using F (j) maximum entropy*,ξ*,η*) (ii) a F (j) is the characteristic distribution of each column of pixels in the vertical direction extracted from the protective fence panoramic image;
according to the segmentation threshold value (epsilon)*,ξ*,η*) Carrying out binarization segmentation on F (j), setting a protective guard area as 1 and setting a background area as 0;
wherein, the first and second connecting parts are connected with each other; v (1, j) is the mean value of the gray levels of each column of pixels in the panoramic image of the protective fence;
m (1, j) is the gray variance of each column of pixels in the protective fence panoramic image; g (1, j) is the mean value of the gradient of each column of pixels in the panoramic image of the protective fence.
Wherein p (i, j) is the gray value of the pixel point (i, j) in the protective fence panorama, th,bhRespectively the coordinate positions of the top and bottom of the guard rail in the panorama of the guard rail.
Another embodiment of the present invention is described below.
The invention aims at detecting the closure of a closed operation environment of a high-speed railway, and provides a method for realizing dynamic detection of the defects of protective guards of the high-speed railway by splicing a panoramic image of a forward motion video and detecting the characteristics of the panoramic image.
The technical scheme of the invention is as follows: converting the forward motion video recording the high-speed railway environment state into a railway environment panoramic image through a virtual channel sampling splicing technology; extracting features of the panoramic picture; and judging and evaluating the defect state of the railway protective guard according to the characteristic travel code obtained after the characteristic extraction. That is, the video image is converted into a static panorama format, i.e., a forward motion video panorama is generated under the condition that the optical axis of the camera is consistent with the motion direction; generating a virtual sampling channel model of the forward motion video based on the geometric structure of the detection area; and (3) carrying out maximum entropy threshold segmentation on the basis of a mean-variance-gradient (MVG) three-dimensional histogram of a guard rail panoramic image and guard rail detection based on run-length coding.
The method specifically comprises the following steps:
step 1, positioning forward motion video based on railway line prior, and virtualizing a rectangular sampling channel;
step 2, segmenting the forward motion video frame into four parts of images, namely, a sky image, a ground image, a left image and a right image;
step 3, carrying out panorama stitching calculation based on the forward motion video frame;
step 4, calculating the characteristics of the panoramic image of the protective fence;
and 5, detecting the guard bar based on the stroke codes.
Wherein, step 1 specifically includes:
extracting a set of zonal sequences S from a video sequence1,S2,S3,…，SNAnd the adjacent panoramic annular zones have no overlap and no interval between the sampling of the physical space, namely 'complete sampling'. And carrying out geometric correction on the ring zone sequences and splicing the ring zone sequences together to generate a panoramic image.
Based on the railway line prior, a forward motion video virtual channel is positioned, and a 'fully sampled' virtual rectangular sampling channel is constructed by an object of a concerned depth layer. The construction of the virtual rectangular sampling channel is based on the following objective approximation:
(1) the railway facility scenario includes: guard rails or sound barriers on two sides, steel rails on the ground, overhead contact networks hung above and the like, wherein similar facilities are regarded as being approximately located on the same plane, and the distance between the plane and the camera is known;
(2) each plane of the railway facility scene is a depth layer closest to the camera, and the generated panoramic image does not have railway facility information loss caused by undersampling.
The step 2 specifically comprises the following steps:
(1) determining vanishing points
Using LSD (line segment detector) line segment detection algorithm to check the video image of the video sequence, dividing all detected line segments into two groups, wherein the first group is a line segment which has an included angle of [ -60 degrees, 60 degrees ] with a horizontal axis and is used for estimating vanishing points formed by the line segments parallel to the steel rail by adopting a least square method; the second group is line segments in the range of-5, 5 from the vertical axis to estimate the position of the line rod to demarcate the railway scene.
Let the coordinate of the vanishing point in the image coordinate system be (x)0,y0)TWhich is connected to the ith line segment x + kiy+bi＝0(kiIs a slope, biIntercept) of
WhereinMinimizing the point (x) according to the least squares method0,y0)TThe sum of the squares of the distances to all line segments, i.e.
Wherein, the weight value wiIs the length of the ith line segment.
Separately solving for x for the formula (2)0And y0And let it be zero, the following system of linear equations is obtained,
here, the
N is the total column number of pixels in the guardrail panorama;
the solution of the formula (3) is,
(x0,y0) Is an optimal estimate of the vanishing point. In order to improve the accuracy of estimating the vanishing point, a cross iteration method is adopted to remove some 'wrong' line segments.
(2) Dividing the four parts into a day part, a ground part, a left part and a right part.
The cuboid structure of the virtual rectangular sampling channel consisting of the steel rail, the contact net and the guard rail or the sound barrier has a fixed geometrical structure. The position of the selected sampling ring is determined according to the shape of the virtual rectangular sampling channel of the scene (fig. 3 is the scene sampling ring of the high-speed railway).
The vanishing point Q is connected to four vertexes of the sampling ring rectangle respectively, and four parts divided by the four connecting lines are four parts, namely a day part, a ground part, a left part and a right part.
And 3, carrying out panorama stitching calculation based on the forward motion video frame.
As shown in fig. 4, the image registration process based on vanishing point detection and camera motion information is as follows:
(1) a virtual rectangular sampling channel is formed by the positions of concerned railway scenes, namely guardrails, contact networks, sound barriers and steel rails, and complete sampling is carried out in the virtual rectangular sampling channel.
(2) Given a sequence of video images with a number of frames N I1,I2，…，IN-1，INSelecting a splicing area in each frame of image, namely determining an external rectangle ORm:AmBmCmDmAnd an internal rectangular IRm:A′mB′mC′mD′m。
(3) According to the scene structure determined by the vanishing point, an external rectangle OR is pre-designatedm:AmBmCmDmThe position, and the position in each frame is the same. Four rays which respectively pass through the top and the bottom of the electric poles on the left side and the right side of the image are led out through the vanishing point Q, and the scene is divided into four parts (namely, four parts including the sky part, the ground part, the left part and the right part) of the sky, the steel rail, the left guardrail and the right guardrail by the four rays. Four vertices A of the outer rectanglem、Bm、Cm、DmThe generated panoramic images of the upper scene, the lower scene, the left scene and the right scene are not overlapped, so that wrong sampling is avoided, and the process of drawing the virtual scene is simplified. Furthermore, since the moving direction of the camera is parallel to the rails, the Expansion center (FOE) of the optical flow of the pixel points in the image coincides with the vanishing point Q formed at a far position of the two rails in the image. The direction of movement of the four vertices of such an outer rectangle, i.e. the outer rectangle ORmThe scaling direction with time frame is uniquely determined by vanishing point Q, written as:
(4) and if the train speed is V and the frame rate of the camera is R, the distance of space sampling between two adjacent frames is V/R. If the internal parameters of the camera are known, the moving speed of the pixels in the image, namely the image speed v, can be obtained through corresponding geometric calculation.
(5) Obtaining an external rectangle OR in the mth framem:AmBmCmDmZoom direction ofAnd the image velocity v, the corresponding coordinate position of the outer rectangle in the m-1 frame can be determined fromAnd (6) directly obtaining.
(6) Outer rectangle OR in the m-1 th framem-1And an internal rectangular IRm-1Form a rectangular ring-shaped splicing region SmWill SmScaling direction along outer rectangle verticesStrip S divided into 4 trapezoidst、Sb、Sl、SrSky, rail, left guardrail and right guardrail area respectively.
(7) The structured tape S is wound with an Image Wrapping (Image Wrapping)t、Sb、Sl、SrPerforming a geometric transformation to map irregular trapezoidal stripes to regular rectangular stripes St'、Sb'、Sl'、Sr'. Repeating the above process for each frame of image to obtain four groups of strip sequences, and respectively splicing the sequences in sequence to obtain the panoramic image. The invention relates to the detection of left and right guard rails, for which only rectangular strips S are usedl'、SrThe left and right panoramic views of the railway running environment, namely the panoramic views of the guard rails, are obtained by splicing.
And 4, calculating the characteristics of the panoramic image of the protective fence.
Let p (i, j) be the gray value of pixel point (i, j) in the panoramic image of the protective fence, th,bhThe coordinate positions of the top and the bottom of the protective fence in the panoramic image are respectively recorded, and the gray mean, the gray variance and the gradient mean of each column of pixels are respectively recorded as
Extracting gray scale and gradient features along the vertical direction of the protective fence panorama to obtain
{F(j)}＝{(x,y)}＝{(M(1,j),V(1,j),G(1,j))} (9)
J is more than or equal to 1 and less than or equal to N, N is the total column number of pixels in the protective guard panorama, and F (j) represents the three-dimensional characteristic distribution of each column of pixels.
The statistical characteristics of the gray scale at the guard rail satisfy M (1, j) < epsilon, V (1, j) < ξ, G (1, j) < η where epsilon, ξ and η are segmentation thresholds for segmenting foreground regions of the guard rail, i.e., locating guard rail positions.
The feature distribution f (j) of each column of pixels in the vertical direction extracted from the guard rail panorama is shown in fig. 5.
If any threshold value (epsilon, ξ) is given to segment the feature distribution f (j) of the guardrail guard rail, the spatial distribution of the three-dimensional feature f (j) is divided into 8 feature sub-regions (as shown in fig. 6) by (epsilon, ξ), and the three feature sub-regions are recorded as:
each guard rail has an approximate brightness value and a uniform brightness distribution in the vertical direction, and the background area is a scene with irregular brightness distribution. This results in the position of the guard rail being characterized mostly by R1Region, and background features are mostly concentrated on R8And (4) a region.Although the other 6 regions also contain part of the guard bar and background information, the experimental results of ignoring these secondary feature regions show no significant effect on the positioning results in order to simplify the calculation process.
F (j) any one of the feature vectors is positioned in the guardrail region R1Or a background region R8Respectively has a probability of PF(. epsilon., ξ) and PB(ε,ξ,η)；
Here, pxyzThe probability that a pixel point in the image is located in the foreground or the background is obtained.
And satisfy
PF(ε,ξ,η)+PB(ε,ξ,η)≈1 (13)
According to the definition of the entropy, the three-dimensional entropies of the guardrail area and the background area are respectively
Overall three-dimensional entropy of F (j) of
H(ε,ξ,η)＝HF(ε,ξ,η)+HB(ε,ξ,η) (16)
The triple (. epsilon.) in which the maximum value can be obtained by the above formula H (. epsilon., ξ) based on the principle of maximum entropy division*,ξ*,η*) Is the optimal segmentation threshold sought, i.e.
According to the threshold value (epsilon)*,ξ*,η*) F (j) is subjected to binarization segmentation, the guard rail area is set to be 1, and the background area is set to be 0, namely
Step 5, guardrail detection based on stroke coding
According to the maximum entropy division principle, the positions of the guard rails and the background in the railway scene can be located, wherein the position of the guard rail is represented at F (j) of 1, and the position of the background is represented at F (j) of 0. As shown in fig. 5, a white area (value of 1) represents a guard rail, and a black area (value of 0) represents a background. Run-length coding is performed on a binary code composed of a series of 0 s and 1 s obtained according to a threshold segmentation method.
If the guard rail is defective, the pixel width D of the background area between two adjacent guard railscurIs necessarily larger than the normal pixel pitch D between the guard rails, so if Dcur>k · d (k is an empirical adjustment parameter), it can be determined that there is a defect in the region.
Another embodiment of the present invention is described below.
Example 1: a novel panorama stitching method based on a forward motion video virtual rectangular sampling channel model is disclosed.
The acquisition of the panoramic image of the railway comprises the following three steps: forward video acquisition, construction of a splicing area and strip splicing.
An algorithm for generating a panoramic image of a railway scene according to a virtual rectangular sampling channel model is shown as the following algorithm 1.
As shown in fig. 6, panoramagram results were generated for low quality video (720 x 576) captured under high speed conditions (150 km/h). The method provided by the invention can generate a satisfactory panoramic image. Such as near guard rails and poles, have no lost information and less distortion. The distant poles are significantly distorted by stretching due to "oversampling", but this is not a concern in practical inspection.
Example 2: guardrail detection based on panorama concatenation.
As shown in fig. 9, a video image of forward motion collected for integrated inspection of a train. The train keeps a relatively constant speed, and the acquisition frame rate of the camera is 25 frames/second. Fig. 9(a) shows a railway scenario in which the guard rail is not defective, and fig. 9(b) shows a railway scenario in which a defect exists.
The generated partial panorama (left side) of the railway is shown in fig. 10(a), and fig. 10(b) is a partial panorama of the guard rail extracted from fig. 10(a) (i e 405,490, j e 2500,3000, where i is the row coordinate of the pixel and j is the column coordinate).
The guard rail and guard rail positioning algorithm realized according to the maximum entropy division principle is as follows:
fig. 11 is a comparison result between the three-dimensional histogram maximization entropy segmentation method and the one-dimensional gray-scale histogram segmentation method and the two-dimensional gray-scale-gradient histogram segmentation method proposed by the present invention. Fig. 11(a) is a result of maximum entropy threshold segmentation based on only gray one-dimensional histograms, and it can be observed that many background regions are wrongly segmented into guard rails, which will result in false detection. Fig. 11(b) is a result of maximum entropy threshold segmentation of a two-dimensional histogram based on gray-gradient (GLGM), it can be observed that the guard rail at the location of column coordinates 2770 is erroneously segmented as a background, which would result in a detection false alarm. Fig. 11(c) shows the result of the three-dimensional histogram segmentation proposed according to the present invention, which is most accurate by comparing with the actual position of the fence shown in fig. 10 (b).
After the position of the guardrail is extracted, the binary code is subjected to stroke coding so as to achieve the purposes of lightweight storage and access of guardrail information. And then, quickly identifying the damaged position of the protective guard from the travel code of the panoramic image of the protective guard by using an algorithm 3. As shown in fig. 11(c), two regions with significant guardrail defects exist in the pixel range of [2500,3500], which are located in the regions [3000,3138] and [3290,3462], respectively.
The experimental video is in an avi format, the duration is about 1 hour, the data volume is 1.35GB, and the total number of pixels is (1024 × 768)/frame × 85500 frames. An image in jpg format with a data amount of 84MB is generated by panorama stitching, and the total number of pixels is 267500 × 600 (the total number of pixels in the horizontal direction × the total number of pixels in the vertical direction). The foreground and background areas of the guardrail in the panorama are only indicated by single-byte numbers, and finally 8917-bit digital codes (373413332423 … …) are obtained, and the data size is only 17.4 KB. Therefore, the invention realizes the multi-stage compression and extraction of the data volume of the railway guardrail from GB level, MB level and KB level, and overcomes the application bottleneck of the detection algorithm caused by large video data volume and difficult information storage and retrieval.
The invention provides an algorithm for detecting the defect position of a protective guard based on panoramic stroke coding.
Because the train has a long travel and the illumination of the whole guard rail panoramic image changes due to the influence of factors such as weather, environment and orientation, it is necessary to divide the guard rail panoramic image into K sub-images and calculate a suitable local threshold value (epsilon) for each sub-imagei,ξi,ηi) Where i is 1,2,3, …, K.
And determining that 38 defects actually exist in the guardrail in the section of the line through multiple manual visual detections on site, and recording the defects as GT (total mark) 38. Based on the detection results of different threshold segmentation methods, for example, as shown in table 1, the method provided by the present invention correctly detects 36 defects (TP-36) by comparing with the real condition of the guardrail defect; 3 normal guard rail positions are detected as defects (FP is 3) in error; two missed breaks were not detected (FN ═ 2). The detection accuracy and the recall rate are used for verifying the effectiveness of the detection method, and the threshold segmentation method provided by the invention achieves the accuracy of 92.3% and the recall rate of 94.7% in comparison of detection results, and is superior to the detection results of other segmentation algorithms.
TABLE 1 comparison of test results of different threshold segmentation methods
In order to further verify the applicability of the maximum entropy threshold segmentation method combined with the multi-feature MVG three-dimensional histogram, the same experiment is carried out on a white cement guard fence which is another type of guard fence of the high-speed railway. The white cement guard rail is different from the iron guard rail having a dull color as shown in fig. 10(b), and thus the brightness of the guard rail area is generally higher than that of the background area. As shown in fig. 12(d), as compared with the panorama of the guard rail in fig. 12(a), it can be seen that the segmentation algorithm proposed in the present invention achieves a satisfactory segmentation result for both the metal guard rail and the cement guard rail.
Fig. 13 is a flow chart of steps of a rapid detection method for defects of a protective fence of a high-speed railway.
The high-speed railway protective fence detection method based on the forward motion video panorama comprises the following steps:
according to the railway line prior, after the position of a virtual rectangular sampling channel of the forward motion video is positioned, the forward motion video frame is divided into four parts, namely a sky part, a ground part, a left part and a right part, and the four parts are respectively spliced by a panorama. That is to say, the video data is subjected to lossless information extraction in the panorama format, so that the lightweight detection image information is obtained, the storage and access expenses are reduced compared with the video data, and the video is converted into a form more suitable for manual visual inspection or computer analysis and processing.
Based on the panorama stitching calculation of forward motion video frames, the positions of sampling rings are quickly obtained by depending on the geometric structure prior of a scene and the motion speed information of a camera, the quick alignment between stitched images is realized, the left and right guard rail panoramas are generated, and the overall display effect of the panorama stitching calculation is obviously superior to that of an L-K optical flow method with characteristic matching. Namely, the forward motion video virtual sampling channel model is utilized to overcome the static blurring of narrow-band splicing of the forward motion video panorama, and the prior of the area geometric structure is utilized to construct a rapid panoramic splicing algorithm of the forward motion video.
And (3) segmenting the panoramic pictures of the left and right guard rails by adopting a maximum entropy threshold segmentation method of an MVG three-dimensional histogram, segmenting foreground and background areas of the guard rails in the panoramic pictures, and realizing automatic positioning of the positions of the guard rails. And stroke coding is carried out on the extracted guardrail position binary code, so that light-weight storage and access of guardrail information are realized, and the position of the damaged guardrail is quickly identified from the stroke coding of the guardrail panoramic image according to the algorithm 3.
That is to say, the automatic detection method of guardrail defects based on a guardrail panorama (Fence panorama) automatically extracts the position of a vertical guardrail in a guardrail by using a maximum entropy threshold segmentation method based on an MVG three-dimensional histogram to separate the vertical guardrail from a background image; and the binary code after the separation of the guardrail panoramic image is compressed by using the stroke code, the compressed coding format contains all position information of the guardrail, the corresponding decoding algorithm recovers the position of the guardrail from the coding, and the missing detection of the guardrail is realized.
The invention has the following beneficial effects:
(1) the invention provides a novel panorama splicing method, which is a panorama splicing method based on a forward motion video virtual sampling channel model and fills a blank generated by a forward motion video panorama under the condition that the optical axis of a camera is consistent with the motion direction; the method can be widely applied to automatic driving, safety inspection, high-compression-ratio driving record of vehicles such as rail transit, roads and the like, and detection of straightness of high-speed railway steel rails, detection of foreign matter invasion limit, detection of mountain landslides along the railway and the like. Since the sound barrier used in some areas is used to reduce noise interference to the surrounding environment and has a guard rail function, but is physically different from the guard rail, different defect state detection algorithms can be designed according to different types of guard rails.
(2) The automatic visual detection method of the panorama based on the forward motion video extracts massive video data in a lightweight panorama format in a lossless information extraction mode, reduces storage and access expenses of a video data form, converts the video into a form more suitable for manual inspection and computer analysis processing, and solves distortion-free rapid conversion from the forward motion video to the panorama. In addition, a large amount of storage space is saved by storing the video information in the form of the panoramic image, and the panoramic image is used for replacing a forward motion video image as a detection object, so that the rapid detection of the closed environment of the high-speed railway can be realized, and the actual requirement is met.
(3) Forward motion video shooting is used as a mobile scene acquisition mode, and has been widely applied to mobile scene recording and monitoring tasks due to wide visual field, deep and distant scene and wide space coverage. The invention provides a method for automatically and quickly detecting the defect state of a protective fence of a high-speed railway by using a computer vision technology based on a forward motion video aiming at the detection of the state of a closed running environment of the high-speed railway, and provides a new high-efficiency means for completing the detection of the state of the closed running environment of the high-speed railway by using a high-speed comprehensive detection train.
(4) The invention discloses a forward motion video virtual sampling channel model based on a detection region geometric structure, overcomes the static blurring of narrow-band splicing of forward motion videos, constructs a rapid panoramic splicing algorithm of the forward motion videos by utilizing the prior of the region geometric structure, and provides a corresponding automatic visual detection algorithm based on a panoramic image by taking railway protective fence defect detection as an application research background. The high-speed railway environment detection is a regular and normal work, and an environment video image is periodically and repeatedly acquired. And obtaining the information of the abnormal state through feature detection or scene image comparison. The invention provides a novel panorama stitching technology based on high-speed forward motion video aiming at high-speed railway environment detection.
(5) The method for splicing the panoramic images of the virtual rectangular sampling channels based on the forward motion video firstly realizes the generation of the panoramic images of the forward motion video under the condition that the optical axis of a camera is consistent with the motion direction; the problem of static distortion of a forward motion video panoramic image is solved by adopting a virtual rectangular sampling channel mode; panoramic images are generated based on a forward motion video virtual rectangular sampling channel, special requirements on equipment do not exist, and videos recorded by a common automobile data recorder can be spliced into panoramic images with different purposes.
Claims (6)
1. A panorama splicing method based on a virtual sampling channel model is characterized by comprising the following steps:
acquiring a forward motion video for recording the environmental state of the high-speed railway; extracting a video image sequence with the number of frames N from the forward motion video;
setting an external sampling rectangle OR in each frame of video image according to the railway scene structure determined by the vanishing pointm(ii) a The coordinate of the vanishing point in the image coordinate system is (x)0,y0)T(ii) a The analytic expression of the ith line segment is x + kiy+bi0, wherein kiIs the slope of the ith line segment, biIs the intercept; weight wiIs the length of the ith line segment; n is the total column number of pixels in the protective guard panoramic image;
wherein,
setting an internal sampling rectangle IR in each frame of video image according to the speed of the trainm；
The external sampling rectangle OR to be composed of each frame of video imagemAnd the internal sampling rectangle IRmThe formed rectangular annular sampling ring band is divided into four band splicing areas St,Sb,Sl,Sr；
Splicing the four strips of each frame of video image into a region St,Sb,Sl,SrCorrected to regular rectangular strips S by image wrappingt',Sb',Sl',Sr'；
4 XN corrected rectangular strips St',Sb',Sl',Sr' the panoramic images of 4 planes of the railway scene are generated by respectively splicing the sampling planes.
2. A method for detecting the defect state of a protective guard of a high-speed railway based on the panorama splicing method of claim 1 is characterized by comprising the following steps:
acquiring a forward motion video for recording the environmental state of the high-speed railway;
secondly, converting the forward motion video into a left panorama and a right panorama of a railway running environment based on a panorama splicing method of a virtual sampling channel model to serve as a protective fence panorama;
thirdly, positioning the positions of the guard rails and the background in the guard rail panoramic image according to a maximum entropy segmentation principle, wherein the position where F (j) is 1 represents the position of the guard rails, and the position where F (j) is 0 represents the position of the background, so as to obtain a binary code consisting of 0 and 1; j is the column sequence number of the protective fence panorama;
step four, the run length coding is carried out on the binary code;
step five, calculating the pixel width D of the background area between two adjacent guardrails according to the stroke codescur；
Step six, according to the pixel width D of the background area between the two adjacent guardrailscurJudging whether the two adjacent guardrails have defects or not by the normal pixel distance d between the two adjacent guardrails, and generating a judgment result;
and step seven, outputting the judgment result.
3. The method of claim 2, wherein the sixth step comprises:
when D is presentcurIf the value is greater than k · d, the judgment result is as follows: the two adjacent guardrails have defects; wherein k is an empirical tuning parameter.
4. The method according to claim 2, wherein the second step comprises:
extracting a video image sequence with the number of frames N from the forward motion video;
setting an external sampling rectangle OR in each frame of video image according to the railway scene structure determined by the vanishing pointm；
Setting an internal sampling rectangle IR in each frame of video image according to the speed of the trainm；
Rectangular OR of each frame of video image from said external samplesmAnd the internal sampling rectangle IRmThe formed rectangular annular sampling ring band is divided into four band splicing areas St,Sb,Sl,Sr(ii) a Wherein S ist,Sb,Sl,SrSky, steel rail, left guardrail and right guardrail areas;
splicing the four strips of each frame of video image into a region St,Sb,Sl,SrCorrected to regular rectangular strips S by image wrappingt',Sb',Sl',Sr'；
2 XN corrected rectangular strips Sl',Sr' respectively splicing according to respective sampling planes to generate a left panorama and a right panorama of the railway scene.
5. The method of claim 2, wherein step three comprises:
calculating a gray mean M (1, j), a standard deviation V (1, j) and a gradient mean G (1, j) for each column j of the protective fence panorama; j is 1, …, N; n is the total column number of pixels in the protective guard panoramic image;
calculating a segmentation threshold (. epsilon.) using F (j) maximum entropy*,ξ*,η*) (ii) a F (j) is the characteristic distribution of each column of pixels in the vertical direction extracted from the protective fence panoramic image;
according to the segmentation threshold value (epsilon)*,ξ*,η*) Carrying out binarization segmentation on F (j), setting a protective guard area as 1 and setting a background area as 0;
wherein M (1, j) is the gray average value of each column of pixels in the protective guard panorama;
v (1, j) is the gray variance of each column of pixels in the protective fence panoramic image; g (1, j) is the mean value of the gradient of each column of pixels in the panoramic image of the protective fence.
6. The method of claim 5,
wherein p (i, j) is the gray value of the pixel point (i, j) in the protective fence panorama, th,bhRespectively the coordinate positions of the top and bottom of the guard rail in the panorama of the guard rail.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610312137.XA CN106023076B (en) | 2016-05-11 | 2016-05-11 | The method of the damage condition of the protective fence of the joining method and detection high-speed railway of panorama sketch |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610312137.XA CN106023076B (en) | 2016-05-11 | 2016-05-11 | The method of the damage condition of the protective fence of the joining method and detection high-speed railway of panorama sketch |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106023076A CN106023076A (en) | 2016-10-12 |
| CN106023076B true CN106023076B (en) | 2019-04-23 |
Family
ID=57099345
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610312137.XA Expired - Fee Related CN106023076B (en) | 2016-05-11 | 2016-05-11 | The method of the damage condition of the protective fence of the joining method and detection high-speed railway of panorama sketch |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106023076B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106709892A (en) * | 2016-11-15 | 2017-05-24 | 昂纳自动化技术（深圳）有限公司 | Rapid region expansion algorithm and device of any structure element based on stroke coding |
| CN106851398A (en) * | 2017-03-10 | 2017-06-13 | 北京小鸟看看科技有限公司 | The storage method of panoramic video file, player method, storage device and player |
| CN107103578B (en) * | 2017-03-28 | 2020-12-11 | 北京交通大学 | Method for carrying out panorama stitching by utilizing forward vehicle-mounted video of high-speed railway |
| JP6903477B2 (en) | 2017-04-21 | 2021-07-14 | 株式会社東芝 | Orbit identification device and program |
| CN109842761B (en) * | 2019-02-01 | 2021-03-19 | 中国社会科学院大学 | Rapid high-definition panoramic imaging method and device for forward motion video |
| CN111241900B (en) * | 2019-04-12 | 2021-02-05 | 宁夏爱特云翔信息技术有限公司 | Traffic environment field maintenance method |
| CN112837341B (en) * | 2021-01-26 | 2022-05-03 | 石家庄铁道大学 | Self-adaptive time-space domain pedestrian appearance restoration method |
| CN117532624B (en) * | 2024-01-10 | 2024-03-26 | 南京东奇智能制造研究院有限公司 | Automatic positioning and aligning method and system for guardrail plate installation |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101621634A (en) * | 2009-07-24 | 2010-01-06 | 北京工业大学 | Method for splicing large-scale video with separated dynamic foreground |
| CN103236160A (en) * | 2013-04-07 | 2013-08-07 | 水木路拓科技(北京)有限公司 | Road network traffic condition monitoring system based on video image processing technology |
| CN103985254A (en) * | 2014-05-29 | 2014-08-13 | 四川川大智胜软件股份有限公司 | Multi-view video fusion and traffic parameter collecting method for large-scale scene traffic monitoring |
| CN105046649A (en) * | 2015-06-30 | 2015-11-11 | 硅革科技（北京）有限公司 | Panorama stitching method for removing moving object in moving video |
-
2016
- 2016-05-11 CN CN201610312137.XA patent/CN106023076B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101621634A (en) * | 2009-07-24 | 2010-01-06 | 北京工业大学 | Method for splicing large-scale video with separated dynamic foreground |
| CN103236160A (en) * | 2013-04-07 | 2013-08-07 | 水木路拓科技(北京)有限公司 | Road network traffic condition monitoring system based on video image processing technology |
| CN103985254A (en) * | 2014-05-29 | 2014-08-13 | 四川川大智胜软件股份有限公司 | Multi-view video fusion and traffic parameter collecting method for large-scale scene traffic monitoring |
| CN105046649A (en) * | 2015-06-30 | 2015-11-11 | 硅革科技（北京）有限公司 | Panorama stitching method for removing moving object in moving video |
Non-Patent Citations (5)
| Title |
|---|
| Railroad online: acquiring and visualizing route panoramas of rail scenes;Shengchun Wang 等;《VISUAL COMPUTER》;20140930;第30卷(第9期);1045-1057 |
| Rendering Railway Scenes in Cyberspace Based on Route Panoramas;Shengchun Wang 等;《2013 International Conference on Cyberworlds》;20131023;12-19 |
| ROUTE PANORAMA ACQUISITION AND RENDERING FOR HIGH-SPEED RAILWAY MONITORING;Shengchun Wang 等;《ICME 2013》;20130731;1-6 |
| 基于列车前向运动视频的全景图拼接算法;高大龙 等;《山东大学学报(工学版)》;20131231;第43卷(第6期);1-6 |
| 移动车载视频的立体全景图生成方法;王胜春 等;《光学学报》;20141231;第34卷(第12期);1215005-1至1215005-9 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106023076A (en) | 2016-10-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106023076B (en) | The method of the damage condition of the protective fence of the joining method and detection high-speed railway of panorama sketch | |
| Banić et al. | Intelligent machine vision based railway infrastructure inspection and monitoring using UAV | |
| Liu et al. | A review of applications of visual inspection technology based on image processing in the railway industry | |
| CN111899288B (en) | Tunnel leakage water area detection and identification method based on infrared and visible light image fusion | |
| JP6904614B2 (en) | Object detection device, prediction model creation device, object detection method and program | |
| CN102759347A (en) | Online in-process quality control device and method for high-speed rail contact networks and composed high-speed rail contact network detection system thereof | |
| CN104463903A (en) | Pedestrian image real-time detection method based on target behavior analysis | |
| CN109489724A (en) | A kind of tunnel safe train operation environment comprehensive detection device and detection method | |
| CN104866827A (en) | Method for detecting people crossing behavior based on video monitoring platform | |
| CN104881645A (en) | Vehicle front target detection method based on characteristic-point mutual information content and optical flow method | |
| CN107038411A (en) | A kind of Roadside Parking behavior precise recognition method based on vehicle movement track in video | |
| CN112800974A (en) | Subway rail obstacle detection system and method based on machine vision | |
| CN113011252B (en) | Rail foreign matter intrusion detection system and method | |
| CN112766046B (en) | Target detection method and related device | |
| Ketcham et al. | The intruder detection system for rapid transit using CCTV surveillance based on histogram shapes | |
| CN108898842A (en) | A kind of high efficiency encoding method and its system of multi-source monitor video | |
| CN113298059A (en) | Pantograph foreign matter intrusion detection method, device, computer equipment, system and storage medium | |
| CN111561967A (en) | Real-time online detection method and system for pantograph-catenary operation state | |
| Espino et al. | Rail extraction technique using gradient information and a priori shape model | |
| CN107506753B (en) | Multi-vehicle tracking method for dynamic video monitoring | |
| CN112907597A (en) | Railway track line detection method based on deep convolutional neural network | |
| CN115600124A (en) | Subway tunnel inspection system and inspection method | |
| CN116626706A (en) | Rail transit tunnel intrusion detection method and system | |
| CN114359860A (en) | Track identification method and system based on environment perception instance segmentation model | |
| Di Stefano et al. | Automatic 2D-3D vision based assessment of the attitude of a train pantograph |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20190423 Termination date: 20200511 |