WO2022267266A1

WO2022267266A1 - Vehicle control method based on visual recognition, and device

Info

Publication number: WO2022267266A1
Application number: PCT/CN2021/123566
Authority: WO
Inventors: 曾云峰; 于延霞; 赵焱; 陈龙; 张朋; 王嵩淞; 邢淑梅; 王蒙蒙; 赵良; 刘嘉琛
Original assignee: 中车大连机车车辆有限公司
Priority date: 2021-06-24
Filing date: 2021-10-13
Publication date: 2022-12-29
Also published as: CN113269151A

Abstract

A vehicle control method based on visual recognition, and a device. The method comprises: acquiring an image in a vehicle running limit, and pre-processing the image (S100); performing binarization processing on the pre-processed image by using a difference algorithm between the current frame and a background frame, so as to acquire a feature image (S200); performing obstacle feature extraction on the feature image according to a neural network model (S300); and performing evaluation according to the obstacle feature, and performing braking control on a vehicle according to an evaluation result (S400). The solution of actively detecting an obstacle and controlling a vehicle is realized, thereby providing better and active vehicle safety early warning and braking.

Description

A train control method and device based on visual recognition

technical field

The invention belongs to the field of vehicle control, and in particular relates to a visual recognition-based train control method and equipment.

Background technique

In recent years, domestic unmanned driving projects have gradually increased. Vehicles are equipped with passive obstacle detection systems based on mechanical trigger principles. In subsequent domestic unmanned driving projects, passive obstacle detection systems have become standard configurations, but passive obstacle detection systems The object detection scheme cannot realize early detection and early warning of obstacles in the direction of the train, which will inevitably have a certain impact on the safety of vehicle equipment and driving safety.

Contents of the invention

In order to solve the above problems, the present invention proposes a train control method and computer equipment based on visual recognition, wherein the method includes:

Obtain images within the train running limits, and preprocess the images;

Performing binarization processing on the preprocessed image through a difference algorithm between the current frame and the background frame to obtain a feature image;

performing obstacle feature extraction on the feature image according to the neural network model; and

Evaluate according to the characteristics of the obstacle, and perform braking control on the train according to the evaluation result.

In some embodiments of the present invention, the algorithm of the neural network model is SSD algorithm.

In some embodiments of the present invention, the evaluating according to the obstacle characteristics includes:

Extracting the morphological features of the obstacles, classifying them according to the morphological features of the obstacles, and obtaining the size and distance of the classified obstacles; and

According to the classification, size and distance of the obstacle, it is evaluated whether it affects the safety of train operation.

In some implementations of the present invention, the evaluating according to the obstacle characteristics further includes:

Remove the characteristic influence of the original fixed objects within the train running limit.

In some embodiments of the present invention, the braking control of the train according to the evaluation result includes:

When the evaluation result affects the safety of train operation, emergency braking is performed on the train and an alarm is issued.

In some embodiments of the present invention, the performing brake control on the train according to the evaluation result further includes:

When the same obstacle is detected in multiple characteristic images, emergency braking and an alarm can be issued.

In some embodiments of the present invention, also include:

If an alarm occurs, images within the train operation limit acquired within a predetermined time before and after the alarm are saved.

In some implementations of the present invention, the acquisition of image information within the train operation limit includes:

Obtain multiple frames of images within the current train operating limits through high-definition cameras; and

Obtain the distance of the corresponding object within the current train operation limit through the millimeter wave radar.

Obtain multiple frames of images within the current train running limit and the distance of corresponding objects in the images through lidar.

Another aspect of the present invention also proposes a computer device, including a processor and a memory, the memory stores processor-executable instructions, and the instructions implement the above method when executed.

Through the method provided by the present invention, the real-time picture of the train operation limit is obtained, and the real-time picture is preprocessed, and then the current frame and the background frame of the picture of the preprocessed picture are differentially calculated to obtain a feature image for feature recognition, Extract the obstacles in the feature image, determine the distance of the obstacles through the radar, and judge whether the obstacles pose a threat to the safe operation of the train, and perform braking operations on the train when the obstacles threaten the safe operation of the train. Through the above scheme, a scheme for active obstacle detection and train control is realized, and a better active train safety control scheme is provided.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention, and those skilled in the art can obtain other embodiments according to these drawings without any creative effort.

FIG. 1 is a flowchart of a method according to an embodiment of the present invention.

detailed description

In order to make the object, technical solution and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

As shown in Figure 1, one aspect of the present invention proposes a train control method and computer equipment based on visual recognition, wherein the method includes:

Step S100, acquiring an image within the train running limit, and performing preprocessing on the image;

Step S200, using the difference algorithm between the current frame and the background frame, performing binarization processing on the preprocessed image to obtain a feature image;

Step S300, performing obstacle feature extraction on the feature image according to the neural network model; and

Step S400, perform evaluation according to the characteristics of the obstacle, and perform braking control on the train according to the evaluation result.

In this embodiment, in step S100, real-time acquisition of images within the train running limit, the so-called running limit refers to the range that has a certain impact on the safe running of the train when the train is running, and the video size is not less than 720P, the number of frames per second is not lower than 12 frames per second to obtain real-time images of the front of the train. At the same time, the image of each frame is preprocessed to reduce the interference of background noise such as lights and sunlight. In some embodiments, each frame of the collected image is preprocessed using algorithms such as grayscale and filtering to remove noise. In addition, in order to determine the effective range of the safe running of the train in the picture, it is also necessary to determine the track area that affects the safe running of the list through the edge detection sensitive window.

In step S200 , for each preprocessed image frame, a difference operation is performed between it and the background frame image, and binarization processing is performed to convert the image into a feature image composed of 0 and 1. Select the previous frame image as the background frame, subtract it from the current frame image, and then perform binarization as the feature image, which can provide the most complete feature data.

In some embodiments, since the field of view of the track area in the direction of train travel is the same in most scenarios, for example, the images of the track area captured by the camera within the train operation limit during straight-line driving are always "the same", because the train The running track is almost full of regularly appearing sleepers, poles for power supply, rails and other road facilities. Therefore, several common pictures in the direction of the train can be selected as the background frame of the background picture and the current frame is differentially calculated to obtain a binarized feature image. In addition, a plurality of feature images are obtained as a reference by performing a differential operation with a plurality of background images.

In some embodiments, pictures with different scenes can also be used as the background image in some cases, for example, in the case of a curve, or pictures in some special cases can be used as the background image. And select different scenes (curves, straight lines, etc.), and then combine multiple background pictures to make a difference, you can judge the road change trend of the track area within the train operation limit. For example, which of the multiple background pictures is closer to the current frame image, or the similarity of the feature image after the differential operation between the continuous multi-frame images and multiple diverse background frames within a period of time, if multiple consecutive multi-frames A higher overlap between the image and a certain background frame image indicates that the train is in the road scene to which the background frame image belongs.

In addition, in some embodiments, a comprehensive scene confirmation can be carried out through various background pictures under various scene types, and the safety of the current road is determined again by obstacle feature extraction under the scene confirmation. For example, the background frame is determined according to different ecological environments. Specifically, in the case of a train running on a curve, it is necessary to select pictures of the track area of the curve in different geographical locations as the background frame. More specifically, several curve scenes in arid areas are selected as background frames, and several outlander scenes in humid areas in the south are selected as regions, because in the curve scenes, the camera has a wider field of view, so there may be differences due to geographical differences. Factors misjudgment of some obstacles in the outer lane. Therefore, common images of the same scene in different climates can be selected as background frames. That is, the background frame when multiple scenes of the same track type are different. And it can contain multiple track types, such as curved roads, straight roads, bridges, tunnels, etc., and can judge the road change trend of the train running based on multiple background frames, for example, when the current frame image starts to match multiple background frames of the until type , in the next frame and several subsequent frames more background frames matching the curve type, it can be determined that the current train is heading to the outer lane.

In step S300, the obtained feature image is input into the trained neural network model, and the obstacle in the feature image is identified through the neural network model, and the feature information of the obstacle is extracted, such as the position and size of the obstacle, etc. .

In step S400, it is analyzed whether the location of the obstacle affects the running of the train, and whether the size and distance of the obstacle affect the driving safety of the vehicle.

In some embodiments of the present invention, the algorithm of the neural network model is the SSD algorithm.

In this embodiment, the model used to identify obstacles in the feature image is obtained by training with the SSD algorithm. The SSD algorithm takes into account both speed and accuracy, and can be used for detection of multiple categories, and has better detection performance for small target objects. In addition, avoiding the use of fully connected layers in the network structure accelerates the processing speed.

In some embodiments of the present invention, evaluating according to the obstacle characteristics includes:

In this embodiment, the features identified by the neural network model are evaluated, and the types of obstacles are analyzed, including trains, people and small obstacles. And the distance from the corresponding obstacle in the characteristic image of the train track area to the current vehicle position is obtained through the radar. And based on the type, size and distance of the obstacle, evaluate the impact of the obstacle on the train running safety, as shown in the following table,

Obstacles with a width greater than 2800mm are classified as trains, obstacles with a width of 400mm are classified as people, and obstacles with a width of less than 300mm are classified as small obstacles. For train-type obstacles, when the distance is less than 280m, it is regarded as a threat to the safety of train running. For obstacles classified as people, when the distance between them and the train is less than 200m, it is considered to affect the safety of the train. When it is classified as a small obstacle, its distance from the current train is less than 100 meters, and it is considered to affect the driving safety of the train. When there is an obstacle in the direction of the train that affects the safe running of the train (there is an obstacle or there is a risk of rear-end collision), the sound and light alarm will be triggered immediately, and a control command will be output to brake the train.

In some embodiments of the present invention, the evaluating according to the obstacle characteristics further includes:

In this embodiment, since there are some commonly used facilities for trains in the track area in front of the train during the running of the train, it is necessary to remove the influence caused by the original facilities within the train running limit. Therefore, in this embodiment, it is necessary to mark a large number of images with common implementations of train operation limits, and then train through the neural network model. When the "obstacle" appears again in the feature image, it will be classified as Classes are normal fixed objects. Evaluate the object's unsafety.

In this embodiment, after visually analyzing the images captured by the high-definition camera, if obstacles are found in the track area, alarm data will be output to the TCMS system through the TRDP interface. At the same time, real-time images of obstacles in the front track area are transmitted through the PIS network interface. After receiving the alarm information and image information, the three-layer switch of the vehicle transmits the information to the ground OCC through the vehicle-ground communication system. After OCC obtains the alarm information and image information of obstacle detection, it forwards them to the ground vehicle health management system, and the ground health management system forwards the information to the ground intelligent operation and maintenance platform.

Further, evaluate the obstacle, classify the obstacle, and judge the distance between the obstacle and the train to determine whether it affects the safety of the train. When the obstacle affects the safety of the vehicle, it will automatically trigger the sound and light warning immediately. Notify the train driver and passengers that there are obstacles in front of the train that affect the safety of the train. Simultaneously apply the emergency brake. Control the train to stop or reduce the speed to a safe level.

In some embodiments of the present invention, the braking control of the train according to the evaluation result further includes:

In this embodiment, in some cases, the braking operation is performed immediately when there is an obstacle in only one image, which will instead cause unnecessary losses. For example, the obstacle in front is only a light obstacle that suddenly appears or drifts by. Therefore, in the next frame of the video image, the obstacle may leave the train track area, and if the brake is forced to cause adverse effects. For this reason, in this embodiment, by judging a plurality of characteristic images of a plurality of video frames that are adjacent or at intervals of a certain time, if the obstacle affects the safety of the train in a plurality of characteristic images, then further Apply brakes and other safety control operations to trains to reduce the safety risks caused by accidental misjudgments of single-frame feature images.

In some embodiments of the present invention, also include:

If an alarm is generated, images within the train operation limit acquired within a predetermined time before and after the alarm are saved.

In this embodiment, if there is an obstacle in the characteristic image of a certain frame in the image collected from the video to trigger an alarm, then the video content captured by the train within 2 minutes before and after the video frame is saved, and the image quality is not high. Lower than 1080P, 12 frames per second, for subsequent analysis.

In this embodiment, the high-definition camera is used to obtain the image information within the train operation limit, and at the same time, the millimeter-wave radar is used to obtain the distance between the object within the train operation limit and the train.

In this embodiment, the image information within the train running limit and the distance between the object in front and the train can be obtained by means of lidar scanning.

Claims

A visual recognition-based train control method, characterized in that it comprises:

Obtain images within the train running limits, and preprocess the images;

Performing binarization processing on the preprocessed image through a difference algorithm between the current frame and the background frame to obtain a feature image;

performing obstacle feature extraction on the feature image according to the neural network model; and

Evaluate according to the characteristics of the obstacle, and perform braking control on the train according to the evaluation result.
The train control method according to claim 1, wherein the algorithm of the neural network model is an SSD algorithm.
The train control method according to claim 2, wherein said evaluating according to said obstacle characteristics comprises:

Extracting the morphological features of the obstacles, classifying them according to the morphological features of the obstacles, and obtaining the size and distance of the classified obstacles; and

According to the classification, size and distance of the obstacle, it is evaluated whether it affects the safety of train operation.
The train control method according to claim 3, wherein said evaluating according to said obstacle characteristics further comprises:

Remove the characteristic influence of the original fixed objects within the train running limit.
The train control method according to claim 3, wherein the braking control of the train according to the evaluation result comprises:

When the evaluation result affects the safety of train operation, emergency braking is performed on the train and an alarm is issued.
The train control method according to claim 5, wherein said performing brake control on the train according to the evaluation result further comprises:

When the same obstacle is detected in multiple characteristic images, emergency braking and an alarm can be issued.
The train control method according to claim 5 or 6, further comprising:

If an alarm is generated, images within the train operation limit acquired within a predetermined time before and after the alarm are saved.
The train control method according to claim 1, wherein said acquiring image information within the train operation limit comprises:

Obtain multiple frames of images within the current train operating limits through high-definition cameras; and

Obtain the distance of the corresponding object within the current train operation limit through the millimeter wave radar.
The train control method according to claim 1, wherein said acquiring image information within the train operation limit comprises:

Obtain multiple frames of images within the current train running limit and the distance of corresponding objects in the images through lidar.
A computer device comprising a processor and a memory storing processor-executable instructions which, when executed, implement the method of any one of claims 1-9.