CN104331910A - Track obstacle detection system based on machine vision - Google Patents

Abstract

A machine vision-based track obstacle detection system belongs to the field of railway safety. The black-and-white video image is obtained by the on-board camera installed on the train head, and the track is judged as straight or curved according to whether the camera is overtraveled. The curve uses a fixed high-definition camera and a wireless transmitter to receive wireless images to realize manual detection of the curve. Quasi-straight track will analyze the acquired real-time image sequence, adopt the track edge extraction algorithm based on top-hat transformation and Otsu threshold, combine the inter-frame difference method based on mathematical morphology improvement with the background difference method and the corner feature matching tracking algorithm, real-time The obstacles in front are divided into two categories: non-dangerous (stationary small obstacles, moving obstacles that quickly cross the track) and dangerous (stationary large obstacles, moving obstacles that affect the passage of trains). The invention can quickly process track images, can extract track edges more effectively and accurately, and has higher detection accuracy for track obstacles.

Description

A Track Obstacle Detection System Based on Machine Vision

technical field

The invention belongs to the field of railway safety, and in particular relates to a machine vision-based track obstacle detection system.

Background technique

In recent years, with the large-scale speed increase of trains, the change of train operation mode, and the obvious increase of passenger and freight volume, higher requirements have been put forward for the safety and reliability of railway transportation. At present, although there are relatively mature products abroad in the research of orbital obstacles, the design concept of most products is to send some form of signal (mainly including laser, radar, magnetic induction, ultrasonic, etc.) to the direction to be detected. And analyze the signal detected and reflected by the sensor as the basis for judging and identifying obstacles. Among these detection methods, for example, the use of ultrasonic waves for railway track detection can detect and identify the position of the target obstacle more accurately, but there is still a problem that the detection effect of the target obstacle with a large volume and a certain height is better, and there will be missed detection. The problem of flat and small obstacles; laser and radar detection have the disadvantages of limited spatial coverage and low resolution. At the same time, this kind of method belongs to aggressive detection, which inevitably increases the environmental noise, and also produces interference between sensors. All these shortcomings and deficiencies will affect the accurate and effective detection and identification of obstacles.

my country's invention patent, the publication number is CN201825066U "Locomotive Ground Signal and Obstacle Automatic Recognition System", it is proposed to install an electronic image recognition system on the car body to analyze the graphics on the line ahead of the operation. The system recognizes and judges the color light signal on the graph, the position of the turnout, the opening and closing status of the turnout, the soil stop at the end line and the parking car to remind the driver or control the automatic braking of the locomotive to prevent accidents. However, the above-mentioned system is only effective in the detection of individual specific substances, and it still cannot play its role in the field of automatic detection and identification of pedestrians, falling rocks and vehicles in front of the train.

Contents of the invention

Aiming at the deficiencies and shortcomings of the prior art, the present invention provides a machine vision-based track obstacle detection system, which can quickly process track images, extract track edges more accurately, and detect track obstacles with higher accuracy.

The technical solution proposed by the present invention is a track obstacle detection system based on machine vision, which obtains black and white video images through a vehicle-mounted camera installed on the train head, and judges whether the track is a straight track or a curve according to whether the vehicle's camera is overtraveled. The curve uses a fixed high-definition camera and a wireless transmitter to receive wireless images to realize manual detection of the curve. Quasi-straight track will analyze the acquired real-time image sequence, adopt the track edge extraction algorithm based on top-hat transformation and Otsu threshold, combine the inter-frame difference method based on mathematical morphology improvement with the background difference method and the corner feature matching tracking algorithm, real-time The obstacles in front are divided into two categories: non-dangerous (stationary small obstacles, moving obstacles that quickly cross the track) and dangerous (stationary large obstacles, moving obstacles that affect the passage of trains). The final results of detection, analysis and identification are notified to the drivers and passengers, so as to effectively avoid misjudgment or traffic accidents. Specific steps are as follows:

1. Collect real-time images

Monocular camera, optical anti-shake, ultra-telephoto black-and-white camera is used to collect image sequences in real time during the train's travel.

2. Determine the track type

When the on-board camera exceeds the distance, it is judged as a curve (large-angle turning track, right-angle track); then a fixed high-definition camera and a wireless transmitter are used to receive wireless images to realize manual detection of the curve track status;

If the on-board camera does not exceed the distance, it is judged as a straight track (straight track, small arc turning track), continue to use the on-board camera to obtain real-time images, and enter the next step to process the image.

3. Image preprocessing

First, the image is enhanced, and the "diamond" structural element with a diameter of 4 is used for the top-hat operation. The top-hat operation of the image X with respect to the structural element B is denoted as X°B, which is defined as Brighten the double-track part and suppress the background highlight part;

Then, the Otsu threshold is used to calculate the optimal threshold T for the enhanced image, and the image is binarized to obtain a binary image; the image after threshold segmentation contains multiple discontinuous regions;

Finally, the connectivity of each pixel in the discontinuous area with its adjacent pixels is checked, the extracted area is marked, and the double-track feature line is preserved through the connected domain mark.

4. Establish a detection window

Preserve the double-track characteristic straight line according to the connected domain mark, use the straight line fitting, and use the least square method to realize the edge positioning of the track extraction, linearize the two tracks, and obtain the straight line equation; determine the position, size and size of the detection window based on the specific position of the rail. shape, the detection window frames the portion of the track within its bounds.

5. Determine whether there are obstacles

Based on the invariance of straight-like orbital rules and the fact that the image in the detection window has excluded the background, the orbital motion can be ignored. When suspicious obstacles invade the detection window, it will inevitably cause changes in image features. This method selects the histogram feature.

Thresholds T ₁ and T ₂ are defined based on empirical values to determine the value change of the histogram variance variation Δσ _k ² and the histogram burr number BurrNum, through the binary black and white pixel ratio ratio, Δσ _k ² and BurrNum three parameters Monitoring to determine whether there is a possibility of obstacle intrusion. The algorithm is as follows:

Histogram mean: $\stackrel{\‾}{r_k}=\underset{r_k=0}{\overset{L-1}{\mathrm{\Σ}}}{r}_k{p}_k\left(r_k\right)$

(where r _k is the histogram feature data, p _k is the corresponding probability, and L is the number of feature data)

Histogram variance: ${\mathrm{\σ}}_k^2=\underset{r_k=0}{\overset{L-1}{\mathrm{\Σ}}}{(r_k-\stackrel{\‾}{r_k})}^2{p}_k\left(r_k\right)$

Binary black and white pixel ratio:

(n _b is the number of black pixels, n _k is the number of white pixels)

From the point of view of importance and reliability, the ratio n _b /n _k is more intuitive and can better reflect the actual situation, so here is the basis for judging:

(1) When ratio<4.5, regardless of the value of Δσ _k ² and BurrNum, it is judged that there is an intrusion obstacle;

(2) When ratio>4.5, observe how the values of Δσ _k ² and BurrNum change, and if the two exceed the threshold, it is judged that there is an intrusion obstacle;

(3) When ratio>4.5, and the values of Δσ _k ² and BurrNum do not exceed the threshold or one of them exceeds the threshold, it is judged that there is no obstacle, which may be affected by external disturbances such as lighting conditions.

If there are no obstacles, return to the beginning; if there are obstacles, proceed to the next step.

6. Obstacle detection

Stationary obstacles also have relative motion to the train, so it is judged whether it is a stationary or a moving obstacle by the motion track and motion characteristics.

The inter-frame difference method improved by mathematical morphology is used to perform differential operations on the images in the detection window to extract and segment moving objects; the inter-frame difference method is to extract two adjacent frames, three frames or multiple frames of images in the video image sequence, And the image operation of pixel subtraction is performed on the extracted frame image. If the difference between adjacent pixels is less than the set threshold, the pixel is considered to be a static background, otherwise, the pixel is extracted as a moving target. According to this principle, all pixels that meet the threshold of the moving target can be extracted by combining all the pixels that meet the threshold of the moving target. The moving target in the scene, and remove the interference background, so as to achieve the purpose of extracting and segmenting the moving target.

The principle of inter-frame difference method is as follows:

D _k (x,y)＝|F _k (x,y)-F _k-1 (x,y)|

F _k (x, y) represents continuous images in the video, D _k (x, y) represents the difference image obtained by subtracting two consecutive frames of images, and then D _k (x, y) is processed as follows:

$R_k(x,\mathrm{the\; y})=\left\{\begin{array}{cc}1& {\mathrm{D.}}_k(x,\mathrm{the\; y})>P\\ 0& {\mathrm{D.}}_k(x,\mathrm{the\; y})<P\end{array}\right.$ (P is the set threshold, which is obtained through repeated simulations of experiments)

R _k (x, y) is the basis for judging whether the target is moving: if it is 1, the target is determined to be moving; otherwise, it is static.

At the same time, using the background difference method, the detection image in the detection window is differentiated from the background image, and then a threshold is used to detect the moving area. Let b(x,y) be the background image, and define an image sequence f(x,y,i), where (x,y) is the coordinate of the image position, and i is the number of image frames. Subtract the gray value of the background from the gray value of each frame image to obtain the difference image:

id(x,y,i)=f(x,y,i)-b(x,y)

A binary difference image can be obtained by setting the threshold T:

(Threshold T is obtained through experimental simulation)

Finally, the pixel points of the images obtained by the two algorithms are intersected to extract the target more accurately, and the size and shape characteristics of the target are obtained.

7. Identification and classification of obstacles

The corner point feature matching tracking algorithm is used to identify the size, speed and direction of obstacles, and the feature extraction uses the Harris corner point detection algorithm, which is a signal-based point feature extraction operator. In this method, the brightness of adjacent points will be compared with ( The gray scale contrast of points in the pixel field) is defined as a corner point that is large enough.

The pixel correlation function used is as follows:

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; ]</span>\mathrm{<span\; class="notranslate">\; m</span>}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; v</span>}\end{array}\right]$

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>\left[\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}\\ {\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}& {\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]$

Among them, w is the smoothing window for noise reduction processing, (u, v) is the offset coordinate, I is the image pixel matrix, and I(x, y) is the pixel value of the point (x, y) in the image. I _x and I _y represent the first-order partial differentials of the image pixels in the horizontal and vertical directions, respectively, and I _x ² and I _y ² are the second-order gradient values in the two directions, respectively. Then the corners in the image can be detected by calculating the corner response function:

R＝det(M)-k*tr ² (M)

Among them, tr(M) and det(M) represent the trace and determinant values of the matrix M respectively, and the Harris corner detection algorithm recommends k to be 0.04.

Feature matching uses a feature point matching algorithm based on gray-scale correlation, taking three-frame extraction as an example:

First, for each feature point p _i ∈ _{E 1} , p _j ∈ E ₃ , respectively centering on the feature point, construct a window of N×N size, denoted as W _i , W _j , and then calculate the related functions

${\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; cor</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}}$

The correlation value of the corresponding feature point can be determined according to the value of C _cor (i,j), and the larger the value is, the closer the neighborhood gray level of the corresponding feature point is. In the process of finding the matching point, the mutual matching method is used for matching, that is, when the best matching point of the feature point pi in E1 is p _i in E3, and the best matching point of p _j is also p _i , It is considered that (p _i p _j ) is a pair of best matching feature points. Based on this, the final corner point matching result is obtained.

Based on this, the type of obstacle ahead can be judged as non-dangerous (stationary small obstacle, moving obstacle that quickly crosses the track) or dangerous (stationary large obstacle, moving obstacle affecting train passing).

Compared with the existing system, the present invention has the following beneficial effects:

Because the algorithm is relatively simple, the complexity is reduced, the processing time is shortened, and the track image can be processed quickly; the track edge extraction algorithm based on top-hat transformation and Otsu threshold can extract track edges more effectively and accurately; a hybrid method based on morphological transformation is used for Target detection effectively improves the shortcomings of the frame difference method accompanied by a large amount of noise and breakpoints due to the limitations in the selection of time intervals. It has the effect of denoising and smoothing the outline of the processing results of the frame difference method, and overcomes the background difference method in judging the target. When an object moves from static to dynamic, a "ghost" defect appears, which optimizes the effect of moving target detection; therefore, the detection accuracy of track obstacles is also high.

Description of drawings

In order to illustrate the embodiments of the present invention more clearly, the following will briefly introduce the accompanying drawings that are required in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. Those of ordinary skill in the art can also obtain other drawings based on these drawings without any creative effort.

Fig. 1 is the detection flow chart of straight road and class straight road system of the present invention;

Fig. 2 is the frame flow diagram of the curve system of the present invention;

Fig. 3 is a schematic diagram of a curve monitoring model;

Fig. 4 is a schematic diagram of a simulated trajectory of a straight road, a quasi-straight road at rest, and a moving obstacle;

Fig. 5 block diagram of inter-frame difference method and mathematical morphology fusion algorithm;

Figure 6 is a block diagram of the moving target detection algorithm.

Detailed ways

Referring to Figures 1-6, a track obstacle detection system based on machine vision, the system obtains black and white video images through the on-board camera installed on the train head, and judges whether the track is a straight track or a curve. The curve uses a fixed high-definition camera and a wireless transmitter to receive wireless images to realize manual detection of the curve. Quasi-straight track will analyze the acquired real-time image sequence, adopt the track edge extraction algorithm based on top-hat transformation and Otsu threshold, combine the inter-frame difference method based on mathematical morphology improvement with the background difference method and the corner feature matching tracking algorithm, real-time The obstacles in front are divided into two categories: non-dangerous (stationary small obstacles, moving obstacles that quickly cross the track) and dangerous (stationary large obstacles, moving obstacles that affect the passage of trains). The final results of detection, analysis and identification are notified to the drivers and passengers, so as to effectively avoid misjudgment or traffic accidents. Specific steps are as follows:

1. Collect real-time images

Monocular camera, optical anti-shake, ultra-telephoto black-and-white camera is used to collect image sequences in real time during the train's travel. In this embodiment, Hongdian H3225A-K-G MDVR vehicle-mounted video recorder is used, and black and white cameras with optical image stabilization and ultra-telephoto focal length can be applied. The collected image storage device is a vehicle-mounted industrial computer: Beijing Huakangtai double 19-inch tablet computer P1903.

2. Determine the track type

When the vehicle-mounted camera overruns, it is judged to be a curve (large-angle turning track, right-angle track); then a fixed high-definition camera and a wireless transmitter are used to receive wireless images to realize manual detection of the curve track status, referring to Figures 2 and 3; this implementation In the example, the fixed high-definition camera adopts a curved high-definition probe, brand Ansky model Ask-3510-67, and the wireless transmitter adopts a wireless transmission device: industrial-grade wireless analog signal transmission DTD110FC. The fixed position of the curved high-definition probe can be other positions than those shown in Figure 3.

If the on-board camera does not exceed the distance, it is judged as a straight track (straight track, small arc turning track), continue to use the on-board camera to obtain real-time images, and enter the next step, using the on-board industrial computer: Beijing Huakangtai dual 19-inch tablet PC P1903 for processing image.

3. Image preprocessing

First, the image is enhanced, and the "diamond" structural element with a diameter of 4 is used for the top-hat operation. The top-hat operation of the image X with respect to the structural element B is denoted as X°B, which is defined as Brighten the double-track part and suppress the background highlight part;

Then, the Otsu threshold is used to calculate the optimal threshold T for the enhanced image, and the image is binarized to obtain a binary image; the image after threshold segmentation contains multiple discontinuous regions;

Finally, the connectivity of each pixel in the discontinuous area with its adjacent pixels is checked, the extracted area is marked, and the double-track feature line is preserved through the connected domain mark.

4. Establish a detection window

Preserve the double-track characteristic straight line according to the connected domain mark, use the straight line fitting, and use the least square method to realize the edge positioning of the track extraction, linearize the two tracks, and obtain the straight line equation; determine the position, size and size of the detection window based on the specific position of the rail. shape, the detection window frames the portion of the track within its bounds.

5. Determine whether there are obstacles

Based on the invariance of the straight-like orbital rules and the fact that the background image in the detection window has been excluded, the orbital motion can be ignored. When suspicious obstacles invade the detection window, it will inevitably cause changes in image features. In this step, the histogram feature is selected.

Thresholds T ₁ and T ₂ are defined based on empirical values to determine the value change of the histogram variance variation Δσ _k ² and the histogram burr number BurrNum, through the binary black and white pixel ratio ratio, Δσ _k ² and BurrNum three parameters Monitoring to determine whether there is a possibility of obstacle intrusion. The algorithm is as follows:

Histogram mean: $\stackrel{\&OverBar;}{r_k}=\underset{r_k=0}{\overset{L-1}{\mathrm{\&Sigma;}}}{r}_k{p}_k\left(r_k\right)$

(where r _k is the histogram feature data, p _k is the corresponding probability, and L is the number of feature data)

Histogram variance: ${\mathrm{\&sigma;}}_k^2=\underset{r_k=0}{\overset{L-1}{\mathrm{\&Sigma;}}}{(r_k-\stackrel{\&OverBar;}{r_k})}^2{p}_k\left(r_k\right)$

Binary black and white pixel ratio:

(n _b is the number of black pixels, n _k is the number of white pixels)

From the point of view of importance and reliability, the ratio n _b /n _k is more intuitive and can better reflect the actual situation, so here is the basis for judging:

(1) When ratio<4.5, regardless of the value of Δσ _k ² and BurrNum, it is judged that there is an intrusion obstacle;

(2) When ratio>4.5, observe how the values of Δσ _k ² and BurrNum change, and if the two exceed the threshold, it is judged that there is an intrusion obstacle;

(3) When ratio>4.5, and the values of Δσ _k ² and BurrNum do not exceed the threshold or one of them exceeds the threshold, it is judged that there is no obstacle, which may be affected by external disturbances such as lighting conditions.

If there are no obstacles, return to the beginning; if there are obstacles, proceed to the next step.

6. Obstacle detection

Stationary obstacles also have relative motion to the train, so it is judged whether it is a stationary or a moving obstacle by the motion track and motion characteristics. Refer to Figure 3.

Referring to Figure 5, the inter-frame difference method improved by mathematical morphology is used to perform differential operations on the images in the detection window to extract and segment moving objects; the inter-frame difference method is to extract two adjacent frames or three frames or Multi-frame images, and perform pixel subtraction image operations on the extracted frame images. If the difference between adjacent pixels is less than the set threshold, the pixel is considered to be a static background, otherwise, the pixel is extracted as a moving target. According to this principle, all pixels that meet the threshold of the moving target can be extracted by combining all the pixels that meet the threshold of the moving target. The moving target in the scene, and remove the interference background, so as to achieve the purpose of extracting and segmenting the moving target.

The principle of inter-frame difference method is as follows:

D _k (x,y)＝|F _k (x,y)-F _k-1 (x,y)|

F _k (x, y) represents continuous images in the video, D _k (x, y) represents the difference image obtained by subtracting two consecutive frames of images, and then D _k (x, y) is processed as follows:

$R_k(x,\mathrm{the\; y})=\left\{\begin{array}{cc}1& {\mathrm{D.}}_k(x,\mathrm{the\; y})>P\\ 0& {\mathrm{D.}}_k(x,\mathrm{the\; y})<P\end{array}\right.$ (P is the set threshold, which is obtained through repeated simulations of experiments)

R _k (x, y) is the basis for judging whether the target is moving: if it is 1, the target is determined to be moving; otherwise, it is static.

Meanwhile, referring to FIG. 6 , the detection image in the detection window is differentiated from the background image by using the background difference method, and then a threshold is used to detect the moving region. Let b(x,y) be the background image, and define an image sequence f(x,y,i), where (x,y) is the coordinate of the image position, and i is the number of image frames. Subtract the gray value of the background from the gray value of each frame image to obtain the difference image:

id(x,y,i)=f(x,y,i)-b(x,y)

A binary difference image can be obtained by setting the threshold T:

(Threshold T is obtained through experimental simulation)

Finally, the pixel points of the images obtained by the two algorithms are intersected to extract the target more accurately, and the size and shape characteristics of the target are obtained.

7. Identification and classification of obstacles

The corner point feature matching tracking algorithm is used to identify the size, speed and direction of obstacles, and the feature extraction uses the Harris corner point detection algorithm, which is a signal-based point feature extraction operator. In this method, the brightness of adjacent points will be compared with ( The gray scale contrast of points in the pixel field) is defined as a corner point with a large enough point, and the pixel correlation function used is as follows:

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; ]</span>\mathrm{<span\; class="notranslate">\; m</span>}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; v</span>}\end{array}\right]$

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>\left[\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}\\ {\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}& {\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]$

Among them, w is the smoothing window for noise reduction processing, (u, v) is the offset coordinate, I is the image pixel matrix, and I(x, y) is the pixel value of the point (x, y) in the image. I _x and I _y represent the first-order partial differentials of the image pixels in the horizontal and vertical directions, respectively, and I _x ² and I _y ² are the second-order gradient values in the two directions, respectively. Then the corners in the image can be detected by calculating the corner response function:

R＝det(M)-k*tr ² (M)

Among them, tr(M) and det(M) represent the trace and determinant values of the matrix M respectively, and the Harris corner detection algorithm recommends k to be 0.04.

Feature matching uses a feature point matching algorithm based on gray-scale correlation, taking three-frame extraction as an example:

First, for each feature point p _i ∈ _{E 1} , p _j ∈ E ₃ , respectively centering on the feature point, construct a window of N×N size, denoted as W _i , W _j , and then calculate the related functions

${\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; cor</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}}$

The correlation value of the corresponding feature point can be determined according to the value of C _cor (i,j), and the larger the value is, the closer the neighborhood gray level of the corresponding feature point is. In the process of finding the matching point, the mutual matching method is used for matching, that is, when the best matching point of the feature point pi in E1 is p _i in E3, and the best matching point of p _j is also p _i , It is considered that (p _i p _j ) is a pair of best matching feature points. Based on this, the final corner point matching result is obtained.

Based on this, the type of obstacle ahead can be judged as non-dangerous (stationary small obstacle, moving obstacle that quickly crosses the track) or dangerous (stationary large obstacle, moving obstacle affecting train passing). Adjust the train operation according to the detection and identification results.

Claims (6)

1., based on a track obstacle detection system for machine vision, it is characterized in that: concrete steps are as follows:

Step 1: gather realtime graphic: obtain black and white video image by the vehicle-mounted vidicon being arranged on train head, adopt the B/W camera of monocular-camera mode, optical anti-vibration, super focal length far away;

Step 2: judge classification of track: whether the excess of stroke judges that track is class straight way or bend according to vehicle-mounted vidicon; Bend track adopts fixing high-definition camera and wireless launcher to receive wireless image and realizes manual detection bend situation; Class straight way track enters step 3;

Step 3: Image semantic classification: the real-time image sequences got is analyzed by class straight way, adopts the rail flanges extraction algorithm based on top cap conversion and Otsu threshold value, obtains track double track characteristic straight line;

Step 4: set up detection window: utilize fitting a straight line, adopts least square method to extract edge local to track, by rail linear, obtains straight-line equation; With rail concrete till determine detection window, rail is confined within the scope of it by detection window;

Step 5: judged whether barrier: based on histogram feature change in detection window, define threshold value T based on experience value ₁and T ₂for differentiating histogram variances variation delta σ _k ²with the value changing condition of histogram burr number BurrNum, by two-value monochrome pixels ratio r atio, Δ σ _k ²with the monitoring of BurrNum tri-parameters, determining whether barrier invasion may;

Step 6: detection of obstacles: the frame differential method improved based on mathematical morphology combines with background subtraction, extracts, obtain the size of barrier, shape facility to barrier feature;

Step 7: the identification of barrier and classification: adopt Corner Feature Matching pursuitalgorithm to carry out barrier speed and direction discernment, is divided into preceding object thing in real time without dangerous (static the small-scale obstacle thing, fast across the moving obstacle of track) and danger (static large obstacle, affect the moving obstacle that train passes through) two classes.

2. a kind of track obstacle detection system based on machine vision according to claim 1, it is characterized in that, the detailed process of step 3 is as follows:

First, image is done and strengthens process, adopt diameter be 4 " diamond " structural element do top cap computing, image X is designated as X ° of B about the top cap computing of structural element B, is defined as highlight double track part, the highlighted part of Background suppression;

Then, Otsu threshold calculations optimal threshold T is adopted to the image after strengthening, by image binaryzation, obtains bianry image; Image after Threshold segmentation comprises multiple discontinuity zone;

Finally, check that in discontinuity zone, each pixel is adjacent the connectedness of pixel, marker extraction region, retains double track characteristic straight line by connected component labeling.

3. a kind of track obstacle detection system based on machine vision according to claim 1, it is characterized in that, the detailed process of step 4 is: retain double track characteristic straight line according to connected component labeling, utilize fitting a straight line, least square method is adopted to realize extracting edge local to track, by two rail linear, obtain straight-line equation; With the position of the particular location determination detection window window of rail, size and dimension, rail portion is confined within the scope of it by detection window.

4. a kind of track obstacle detection system based on machine vision according to claim 1, it is characterized in that, the detailed process of step 5 is as follows:

Got rid of the feature of background based on image in class straight way track rule unchangeability and detection window, can ignore orbital motion, when there being suspicious barrier intrusion detection window, characteristics of image will certainly be caused to change, and this method chooses histogram feature;

Define threshold value T based on experience value ₁and T ₂for differentiating histogram variances variation delta σ _k ²with the value changing condition of histogram burr number BurrNum, by two-value monochrome pixels ratio r atio, Δ σ _k ²with the monitoring of BurrNum tri-parameters, determining whether barrier invasion may; Algorithm is as follows:

Histogram average: $\stackrel{\&OverBar;}{r_k}=\underset{r_k=0}{\overset{L-1}{\mathrm{\&Sigma;}}}{r}_k{p}_k\left(r_k\right)$

(wherein r _kfor histogram feature data, p _kprobability corresponding to it, L is the number of characteristic)

Histogram variances: ${\mathrm{\&sigma;}}_k^2=\underset{r_k=0}{\overset{L-1}{\mathrm{\&Sigma;}}}{(r_k-\stackrel{\&OverBar;}{r_k})}^2{p}_k\left(r_k\right)$

Two-value monochrome pixels ratio: $\mathrm{ratio}=\frac{n_b}{n_k}$

(n _bfor the quantity of black pixel, n _kquantity for white pixel)

From significance level and fiduciary level, ratio n _b/ n _kmore directly perceived, more can show actual conditions, therefore enumerate basis for estimation:

(1) during ratio<4.5, no matter Δ σ _k ²with the value of BurrNum how, all judge there is invasion barrier;

(2) during ratio>4.5, then Δ σ is observed _k ²how to change with the value of BurrNum, if both exceed threshold value, then judge there is invasion barrier;

(3) during ratio>4.5, and Δ σ _k ²all do not exceed threshold value with the value of BurrNum or one of them exceeds threshold value, then judging do not have barrier, may be the impact of the external interference such as illumination condition;

If there is no barrier, then return and start anew; If there is barrier, then carry out next step.

5. a kind of track obstacle detection system based on machine vision according to claim 1, it is characterized in that, the detailed process of step 6 is as follows:

Stationary obstruction also has relative motion to train, so differentiate static or moving obstacle by movement locus and motion feature;

Adopt the frame differential method improved through mathematical morphology, calculus of differences is carried out to the image in detection window, extract segmentation moving target; Frame differential method is exactly in sequence of video images, extracts adjacent two frame three frame or multiple images, and carries out to the two field picture extracted the image operation that pixel subtracts each other; If the difference of neighbor is less than the threshold value of setting, then think that this pixel is static background, otherwise, then extract this pixel as moving target, according to this principle, all pixels meeting moving target threshold value to be joined together the moving target that just can extract in scene, and remove jamming pattern, reach with this object extracting segmentation moving target;

Frame differential method principle is as follows:

D _k(x,y)＝|F _k(x,y)-F _k-1(x,y)|

F _k(x, y) represents continuous print image in video, D _k(x, y) represents the error image that two continuous frames image subtraction obtains, then by D _k(x, y) does following process:

$R_k(x,y)=\left\{\begin{array}{cc}1& D_k(x,y)>P\\ 0& D_k(x,y)<P\end{array}\right.$ (P is setting threshold value, and simulation obtains repeatedly by experiment)

R _k(x, y) is for judge the foundation whether target moves: if 1, then target discrimination is motion; Otherwise, for static;

Meanwhile, utilize background subtraction, the detected image in detection window and background image are carried out difference, then detects moving region by threshold value; If b (x, y) is background image, definition image sequence f (x, y, i), wherein (x, y) is image position coordinates, and i is number of image frames; The gray-scale value of the gray-scale value subtracting background of each two field picture is obtained error image:

id(x,y,i)＝f(x,y,i)-b(x,y)

A binaryzation difference image can be obtained by arranging threshold value T:

(threshold value T simulates by experiment and obtains)

Finally, carry out pixel and get common factor, extract more accurately to target the image of two kinds of algorithm gained, obtain the size of target, the shape of target, differentiation is static or motion.

6. a kind of track obstacle detection system based on machine vision according to claim 1, it is characterized in that, the detailed process of step 7 is as follows:

Corner Feature Matching pursuitalgorithm is adopted to carry out barrier size, speed and direction discernment, feature extraction adopts Harris Corner Detection Algorithm, it is a kind of interest point detect operator based on signal, in this method, the point enough large with adjoint point brightness contrast (intensity contrast of pixel neighborhoods point) is defined as angle point, pixel related function used is as follows:

$E(x,y)=\underset{u,v}{\mathrm{\&Sigma;}}w(u,v){[I(x+u,x+v)-I(x,y)]}^2=[u,v]M\left[\begin{array}{c}u\\ v\end{array}\right]$

$M=\underset{u,v}{\mathrm{\&Sigma;}}w(u,v)\left[\begin{array}{cc}{I}_x^2& I_x{I}_y\\ I_x{I}_y& I_y^2\end{array}\right]$

Wherein, w is the smoothing windows of carrying out noise reduction process, and (u, v) is offset coordinates, and I is the pixel value of image pixel matrix, I (x, y) value image mid point (x, y); I _xand I _ythe respectively single order partial differential of representative image pixel in the horizontal direction, in vertical direction, I _x ²and I _y ²then be respectively the second order Grad in both direction; Then the angle point in image just can be detected by calculating angle point response function:

R＝det(M)-k*tr ²(M)

Wherein, tr (M) and det (M) represents mark and the determinant of matrix M respectively, and Harris Corner Detection Algorithm recommends k to get 0.04;

Characteristic matching adopts based on the relevant Feature Points Matching algorithm of gray scale, is extracted as example with three frames:

First, for each unique point p _i∈ E ₁, p _j∈ E ₃, respectively centered by unique point, the window of structure N × N size, is designated as W respectively _i, W _j, then calculate the related function of each window respectively

$C_{\mathrm{cor}}(i,j)=\frac{\underset{n=1}{\overset{N\&times;N}{\mathrm{\&Sigma;}}}\left(W_i{W}_j\right)}{\underset{n=1}{\overset{N\&times;N}{\mathrm{\&Sigma;}}}{W}_i\underset{n=1}{\overset{N\&times;N}{\mathrm{\&Sigma;}}}{W}_j}$

Can according to C _corthe value of (i, j) can judge the correlation of individual features point, and this value is larger, and corresponding unique point neighborhood gray scale is more close; In the process of looking for match point, the mode of mutually coupling is adopted to mate, namely when the optimal match point of the unique point pi in E1 is the p in E3 _i, p simultaneously _joptimal match point be also p _itime, just think (p _ip _j) be a pair optimum matching unique point; Obtain final corners Matching result accordingly;

The type of preceding object thing is judged, for without dangerous (static the small-scale obstacle thing, fast across the moving obstacle of track) or dangerous (static large obstacle, affect the moving obstacle that train passes through) with this.