CN113223090A - Dynamic visual monitoring method for railway shunting - Google Patents
Dynamic visual monitoring method for railway shunting Download PDFInfo
- Publication number
- CN113223090A CN113223090A CN202110412963.2A CN202110412963A CN113223090A CN 113223090 A CN113223090 A CN 113223090A CN 202110412963 A CN202110412963 A CN 202110412963A CN 113223090 A CN113223090 A CN 113223090A
- Authority
- CN
- China
- Prior art keywords
- coordinate system
- millimeter wave
- wave radar
- monocular camera
- formula
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention provides a dynamic visual monitoring method for shunting a railway, which compares pixel coordinates identified by a millimeter wave radar with a locomotive position identified by a monocular camera through a millimeter wave radar and a coordinate conversion algorithm, takes the pixel coordinates of a point closest to a reference point of the locomotive position in Euclidean distance as a final target point, and can assist the monocular camera to perform locomotive positioning operation through auxiliary information positioning and algorithm of the millimeter wave radar relative to a single-point laser range finder, so that the locomotive positioning is more accurate, the precision loss is less, when the locomotive distance is less than 30m, the millimeter wave radar is turned off, the laser range finder is used as distance measuring equipment, the short-distance laser range finder is more accurate, and the distance measuring precision of short-distance locomotive butt joint is improved.
Description
Technical Field
The invention relates to the field of visual identification, in particular to a dynamic visual monitoring method for railway shunting.
Background
The railway shunting generally relates to the butt joint between the train vehicles, the original butt joint method is that a worker observes the distance between the vehicles to estimate, whether the train head is in butt joint is judged through experience and intuition, along with the development of a vision technology, the existing train butt joint work is replaced by a vision and distance measurement system, image information of a target object is obtained through vision, specific shape and position information of the target object is obtained through a vision image processing algorithm such as machine learning and deep learning, then the distance between the butted train heads is measured through a distance measurement system, the butt joint of the train is completed, the distance measurement of the target object is usually realized through a laser distance meter and a monocular camera in the prior art, and the distance measurement method has the advantage that the distance measurement capability of the laser distance meter and the target object identification capability of the monocular camera can be fully exerted. However, the method has low ranging precision, and because the laser range finder is used for single-point ranging and cannot be matched with an algorithm and a monocular camera to realize precise locomotive position location, the ranging method has certain precision loss and can generate certain precision influence on the butt joint of the locomotives of the train.
Disclosure of Invention
According to the technical problem, the invention provides a dynamic visual monitoring method for railway shunting, which is characterized by comprising the following steps: the specific operation method comprises the following steps:
the method comprises the following steps: completing the calibration of the monocular camera by using a Zhangyingyou calibration method:
and selecting a calibration plate with the square of 52mm and the length and width of 9 multiplied by 6, and calibrating the calibration plate images at different positions by using calibretor application in Matlab software to obtain calibration results of 4 basic multiplying powers. Finally, internal parameter matrixes such as focal length, optical axis offset and the like of the monocular camera are obtained;
step two: installation and calibration of the millimeter wave radar:
calibrating the pitch angle and the roll angle of the millimeter wave radar to be horizontal by using a small-sized horizontal ruler, determining the inverse value of a metal angle anti-coordinate of a certain specific area to be 0 in the X direction and the inverse value in the Y direction to be a set target value by using an angle ruler during the calibration of a yaw angle;
step three: and (3) converting a millimeter wave radar and monocular camera coordinate system:
the millimeter wave radar and the monocular camera are respectively and independently calibrated, so that the normal vector of the detection surface of the millimeter wave radar and the optical axis of the monocular camera are respectively parallel to the longitudinal symmetrical plane of the train body, and a millimeter wave radar projection coordinate system O is established on the basisrw-xrwyrwzrwThe origin of the coordinate system is a projection point of a millimeter wave radar detection center point on the ground, and then a monocular camera projection seat is establishedSystem of symbols Ocw-xcwycwzcwThe origin of the coordinate system is a projection point of the monocular camera optical center on the ground, and the millimeter wave radar projection coordinate system and the monocular camera projection coordinate system are 2 coordinate systems which are parallel to each other in space;
establishing an image coordinate system op-xpyp(ii) a The origin of the coordinate system is positioned at the upper left corner of the image, and a monocular camera coordinate system O is establishedc-xcyczc(ii) a The origin of the coordinate system is the lens of the monocular camera, and the image coordinate system o is based on the pinhole imaging modelp-xpypAnd monocular camera coordinate system Oc-xcyczcThe following relations exist between the following components:
the formula I is as follows:
and according to the monocular camera coordinate system Oc-xcyczcAnd monocular camera projection coordinate system Ocw-xcwycwzcwThe translation and rotation relationship between the two can be obtained as follows:
the formula II is as follows:
the ground is a plane and the target is on the ground, the target projects a coordinate system O on the monocular cameracw-xcwycwzcwIn which there is zcwWhen the value is 0, the first formula and the second formula are combined to obtain the image coordinate system op-xpypProjection coordinate system O to monocular cameracw-xcwycwzcwThe conversion formula between:
the formula III is as follows:
based on the calibration work of the millimeter wave radar and the monocular cameraMetric wave radar projection coordinate system Orw-xrwyrwzrwAnd monocular camera projection coordinate system Ocw-xcwycwzcwThe interconversion can be realized only by translation, and the conversion formula is as follows:
the formula four is as follows:
obtaining a millimeter wave radar projection coordinate system O according to the formula I, the formula III and the formula IVrw-xrwyrwzrwAny point in the image is converted into an image coordinate system op-xpypThe conversion relationship in (1):
the formula five is as follows:
wherein a and b in the formulas I to V are variables, cx、cyIs the amount of optical axis deviation, fx、fyIs a focal length, Lx、LyRespectively millimeter wave radar projection coordinate system Orw-xrwyrwzrwAnd monocular camera projection coordinate system Ocw-xcwycwzcwThe spacing between the X-axes and the spacing between the y-axes;
the conversion relation between the monocular camera coordinate system and the millimeter wave radar coordinate system can be finally obtained by using the formula, and the millimeter wave radar coordinate is finally converted into the pixel coordinate;
step four: and (3) a middle-long distance ranging process:
identifying a target locomotive by using a monocular camera, wherein the identification mode of the monocular camera is realized by adopting a YOLO (YOLO) depth neural network algorithm, the locomotive identification of the neural network is carried out by using a train locomotive database which is open on the network, then a running track corresponding to the train is selected by using a track detection mode, and a target in the track is selected as the target locomotive and marked by using a rectangular frame;
converting message information identified by the millimeter wave radar into actual physical information, performing type conversion from hexadecimal system to binary system on message information data, finding out required position information through a communication protocol list corresponding to the millimeter wave radar, and finally converting the position information into a decimal physical signal for output;
and converting all the detected physical signals of all the positions in the millimeter wave radar coordinate system into a pixel coordinate system of the camera through a formula V and displaying the converted physical signals. Taking the midpoint of the bottom edge of the rectangular frame detected by the monocular camera as a reference point, selecting the pixel coordinate of the point which is closest to the Euclidean distance between all converted camera pixel coordinates and the monocular camera reference point as a final target point, and then enabling the millimeter wave radar to detect the displacement data of the final target point as final vehicle head position information;
further: the type of the millimeter wave radar is ARS 408;
further: the number of the millimeter wave radars is two, the data processing of the two millimeter wave radars adopts a weighted average algorithm, the weighted average of the transverse distance data and the longitudinal distance data of the two millimeter wave radars is selected as the final target distance, and if a certain radar fails or a target point is not detected due to environmental problems, the detection data of the other radar is selected as the final target;
further:
at 30m short-range ranging: and closing the ranging function of the millimeter wave radar, starting the laser range finder as ranging equipment, displaying the image of the head position of the target vehicle by using the monocular camera as an auxiliary visual tool, and displaying the distance information obtained by the laser range finder as final position information in a pixel coordinate system.
The invention has the beneficial effects that:
according to the invention, the pixel coordinate identified by the millimeter wave radar is compared with the head position identified by the monocular camera through the millimeter wave radar and the coordinate conversion algorithm, the pixel coordinate of the point closest to the reference point of the head position in Euclidean distance is taken as a final target point, and relative to a single-point laser range finder, medium-distance and long-distance ranging can be carried out by the auxiliary information positioning and algorithm auxiliary monocular camera to carry out head positioning operation, so that the head positioning is more accurate, the precision loss is less, when the head distance is less than 30m, the millimeter wave radar is turned off, the laser range finder is used as ranging equipment, the short-distance laser range finder is more accurate, and the distance measurement precision of short-distance head butt joint is increased.
According to the invention, two millimeter wave radars are used as the distance measuring equipment, and the weighted average value of the two millimeter wave radars is adopted for the data of medium-distance and long-distance measurement, so that the problem that a single equipment fails or a certain radar cannot be identified due to vehicle shaking is avoided, and the accuracy of distance measurement is improved. Meanwhile, the detection visual angles of the two radars also enlarge the detection range of the target object, and the problem of loss of the target object is better avoided.
Drawings
FIG. 1 is a diagram of a monocular camera internal parameter matrix within the Metlad software of the present invention;
FIG. 2 is a diagram of the coordinate system transformation relationship between the millimeter wave radar and the monocular camera according to the present invention;
FIG. 3 is a table of data type conversion communication protocols for converting millimeter wave radar message data hexadecimal into binary according to the present invention;
FIG. 4 is a data type conversion protocol list for binary to decimal conversion of millimeter wave radar message data according to the present invention;
FIG. 5 is a software application interface of the present invention.
Detailed Description
Example 1
The invention provides a dynamic visual monitoring method for railway shunting, which is characterized by comprising the following steps: the specific operation method comprises the following steps:
the method comprises the following steps: completing the calibration of the monocular camera by using a Zhangyingyou calibration method:
and selecting a calibration plate with the square of 52mm and the length and width of 9 multiplied by 6, and calibrating the calibration plate images at different positions by using calibretor application in Matlab software to obtain calibration results of 4 basic multiplying powers. Finally, internal parameter matrixes such as focal length, optical axis offset and the like of the monocular camera are obtained;
step two: installation and calibration of the millimeter wave radar:
calibrating the pitch angle and the roll angle of the millimeter wave radar to be horizontal by using a small-sized horizontal ruler, determining the inverse value of a metal angle anti-coordinate of a certain specific area to be 0 in the X direction and the inverse value in the Y direction to be a set target value by using an angle ruler during the calibration of a yaw angle;
step three: and (3) converting a millimeter wave radar and monocular camera coordinate system:
the millimeter wave radar and the monocular camera are respectively and independently calibrated, so that the normal vector of the detection surface of the millimeter wave radar and the optical axis of the monocular camera are respectively parallel to the longitudinal symmetrical plane of the train body, and a millimeter wave radar projection coordinate system O is established on the basisrw-xrwyrwzrwThe origin of the coordinate system is a projection point of the detection central point of the millimeter wave radar on the ground, and then a monocular camera projection coordinate system O is establishedcw-xcwycwzcwThe origin of the coordinate system is a projection point of the monocular camera optical center on the ground, and the millimeter wave radar projection coordinate system and the monocular camera projection coordinate system are 2 coordinate systems which are parallel to each other in space;
establishing an image coordinate system op-xpyp(ii) a The origin of the coordinate system is positioned at the upper left corner of the image, and a monocular camera coordinate system O is establishedc-xcyczc(ii) a The origin of the coordinate system is the lens of the monocular camera, and the image coordinate system o is based on the pinhole imaging modelp-xpypAnd monocular camera coordinate system Oc-xcyczcThe following relations exist between the following components:
the formula I is as follows:
and according to the monocular camera coordinate system Oc-xcyczcAnd monocular camera projection coordinate system Ocw-xcwycwzcwThe translation and rotation relationship between the two can be obtained as follows:
the formula II is as follows:
the ground is a plane and the target is on the ground, the target projects a coordinate system O on the monocular cameracw-xcwycwzcwIn which there is zcwWhen the value is 0, the first formula and the second formula are combined to obtain the image coordinate system op-xpypProjection coordinate system O to monocular cameracw-xcwycwzcwThe conversion formula between:
the formula III is as follows:
based on the calibration work of the millimeter wave radar and the monocular camera, the millimeter wave radar projects a coordinate system Orw-xrwyrwzrwAnd monocular camera projection coordinate system Ocw-xcwycwzcwThe interconversion can be realized only by translation, and the conversion formula is as follows:
the formula four is as follows:
obtaining a millimeter wave radar projection coordinate system O according to the formula I, the formula III and the formula IVrw-xrwyrwzrwAny point in the image is converted into an image coordinate system op-xpypThe conversion relationship in (1):
the formula five is as follows:
wherein a and b in the formulas I to V are variables, cx、cyIs the amount of optical axis deviation, fx、fyIs a focal length, Lx、LyRespectively millimeter wave radar projection coordinate system Orw-xrwyrwzrwAnd monocular camera projection coordinate system Ocw-xcwycwzcwBetween the X-axesThe spacing between the pitch and the y-axis;
the conversion relation between the monocular camera coordinate system and the millimeter wave radar coordinate system can be finally obtained by using the formula, and the millimeter wave radar coordinate is finally converted into the pixel coordinate;
step four: and (3) a middle-long distance ranging process:
identifying a target locomotive by using a monocular camera, wherein the identification mode of the monocular camera is realized by adopting a YOLO (YOLO) depth neural network algorithm, the locomotive identification of the neural network is carried out by using a train locomotive database which is open on the network, then a running track corresponding to the train is selected by using a track detection mode, and a target in the track is selected as the target locomotive and marked by using a rectangular frame;
converting message information identified by the millimeter wave radar into actual physical information, performing type conversion from hexadecimal system to binary system on message information data, finding out required position information through a communication protocol list corresponding to the millimeter wave radar, and finally converting the position information into a decimal physical signal for output;
and converting all the detected physical signals of all the positions in the millimeter wave radar coordinate system into a pixel coordinate system of the camera through a formula V and displaying the converted physical signals. Taking the midpoint of the bottom edge of the rectangular frame detected by the monocular camera as a reference point, selecting the pixel coordinate of the point which is closest to the Euclidean distance between all converted camera pixel coordinates and the monocular camera reference point as a final target point, and then enabling the millimeter wave radar to detect the displacement data of the final target point as final vehicle head position information;
further: the type of the millimeter wave radar is ARS 408;
further: the method comprises the steps that two millimeter wave radars are set, a weighted average algorithm is adopted for data processing of the two millimeter wave radars, the weighted average value of the data of the transverse distance and the longitudinal distance of the two millimeter wave radars is selected as a final target distance, and if a certain radar fails or a target point is not detected due to environmental problems, detection data of the other radar is selected as a final target;
further:
at 30m short-range ranging: and closing the ranging function of the millimeter wave radar, starting the laser range finder as ranging equipment, displaying the image of the head position of the target vehicle by using the monocular camera as an auxiliary visual tool, and displaying the distance information obtained by the laser range finder as final position information in a pixel coordinate system.
Example 2
The invention mainly realizes the problem of locomotive distance measurement in the process of train butt joint by fusing four sensors, namely two millimeter wave radars, a monocular vision camera and a laser distance meter.
(1) The millimeter wave radar mainly undertakes the task of measuring the middle and long distances of the target, and the purpose of arranging the two millimeter wave radars is to increase the accuracy of the measuring result and the robustness of fault diagnosis.
(2) The monocular vision camera mainly undertakes the task of identifying and detecting the butted locomotive, and comprehensively obtains the image information of the locomotive by utilizing a deep learning algorithm and a rail detection algorithm of a station.
(3) The laser range finder mainly undertakes the distance measurement task of short-distance straight lines and plays a role in distance measurement when a train is about to be butted.
The invention uses the upper computer software which is assembled by various sensors and designed by the sensors to carry out real-time train distance measurement tasks, and the main method comprises the following steps:
firstly, a monocular vision camera and a millimeter wave radar are used for calibration, and the calibration is unified to a set world coordinate system. The monocular vision camera is calibrated by using a Zhang Zhengyou calibration method to obtain internal parameters and external parameters of the camera. The internal parameter matrixes such as the focal length, the optical axis offset and the like of the monocular camera obtained after calibration are shown in figure 1. The calibration of the millimeter wave radar and the camera are completed in the same world coordinate system, and the coordinate system conversion relationship of the millimeter wave radar and the monocular camera is shown in fig. 2.
Firstly, in the process of long-distance ranging, position information of all targets is obtained through a millimeter wave radar, and message data obtained through the millimeter wave radar (the German continental ARS408 millimeter wave radar is used in the invention) is analyzed according to a data type conversion communication protocol list shown in the figures 3 and 4, so that information including the transverse distance, the longitudinal distance, the transverse speed, the longitudinal speed and the like of the targets is obtained. Then, the data information of the millimeter wave radar is converted to the pixel coordinate system of the visual camera through the coordinate conversion matrix of the millimeter wave radar and the monocular camera, and is displayed in the form of dots.
Then, the monocular vision camera is used for identifying the target locomotive, and a rectangular frame is marked in the image. At this time, a series of target point information obtained by the radar exists in the image. And selecting a radar coordinate point which is closest to the midpoint of the lower end of the rectangular frame as a target vehicle head. And acquiring target position information corresponding to the two millimeter-wave radars, and finally acquiring distance information of the target locomotive through weighting with equal weight.
And finally, in the process of 30m short-distance measurement, because the track is changed into a straight line, the distance measurement of the target locomotive is realized by utilizing the measurement function of the laser distance meter, and the monocular camera is used for displaying in real time.
The invention designs corresponding upper computer software to realize the distance measurement task. The method comprises the steps of message data analysis of radar, serial port communication, image capture and image capture of a monocular camera and the fusion function of a millimeter wave radar, a camera and a laser range finder, well achieves the target locomotive range finding task in the train docking engineering, and the software interface is shown in figure 5.
The foregoing shows and describes the general principles and broad features of the present invention and advantages thereof. While the invention has been described with respect to the above embodiments, it will be understood by those skilled in the art that the invention is not limited to the above embodiments, which are described in the specification and illustrated only to illustrate the principles of the invention, but that various changes and modifications may be made without departing from the spirit and scope of the invention as defined by the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (4)
1. A dynamic visual monitoring method for railway shunting is characterized in that: the specific operation method comprises the following steps:
the method comprises the following steps: completing the calibration of the monocular camera by using a Zhangyingyou calibration method:
and selecting a calibration plate with the square of 52mm and the length and width of 9 multiplied by 6, and calibrating the calibration plate images at different positions by using calibretor application in Matlab software to obtain calibration results of 4 basic multiplying powers. Finally, internal parameter matrixes such as focal length, optical axis offset and the like of the monocular camera are obtained;
step two: installation and calibration of the millimeter wave radar:
calibrating the pitch angle and the roll angle of the millimeter wave radar to be horizontal by using a small-sized horizontal ruler, determining the inverse value of a metal angle anti-coordinate of a certain specific area to be 0 in the X direction and the inverse value in the Y direction to be a set target value by using an angle ruler during the calibration of a yaw angle;
step three: and (3) converting a millimeter wave radar and monocular camera coordinate system:
the millimeter wave radar and the monocular camera are respectively and independently calibrated, so that the normal vector of the detection surface of the millimeter wave radar and the optical axis of the monocular camera are respectively parallel to the longitudinal symmetrical plane of the train body, and a millimeter wave radar projection coordinate system O is established on the basisrw-xrwyrwzrwThe origin of the coordinate system is a projection point of the detection central point of the millimeter wave radar on the ground, and then a monocular camera projection coordinate system O is establishedcw-xcwycwzcwThe origin of the coordinate system is a projection point of the monocular camera optical center on the ground, and the millimeter wave radar projection coordinate system and the monocular camera projection coordinate system are 2 coordinate systems which are parallel to each other in space;
establishing an image coordinate system op-xpyp(ii) a The origin of the coordinate system is positioned at the upper left corner of the image, and a monocular camera coordinate system O is establishedc-xcyczc(ii) a The origin of the coordinate system is monocular phaseAt the lens of the camera, according to the pinhole imaging model, the image coordinate system op-xpypAnd monocular camera coordinate system Oc-xcyczcThe following relations exist between the following components:
the formula I is as follows:
and according to the monocular camera coordinate system Oc-xcyczcAnd monocular camera projection coordinate system Ocw-xcwycwzcwThe translation and rotation relationship between the two can be obtained as follows:
the formula II is as follows:
the ground is a plane and the target is on the ground, the target projects a coordinate system O on the monocular cameracw-xcwycwzcwIn which there is zcwWhen the value is 0, the first formula and the second formula are combined to obtain the image coordinate system op-xpypProjection coordinate system O to monocular cameracw-xcwycwzcwThe conversion formula between:
the formula III is as follows:
based on the calibration work of the millimeter wave radar and the monocular camera, the millimeter wave radar projects a coordinate system Orw-xrwyrwzrwAnd monocular camera projection coordinate system Ocw-xcwycwzcwThe interconversion can be realized only by translation, and the conversion formula is as follows:
the formula four is as follows:
according toThe millimeter wave radar projection coordinate system O can be obtained by the formula I, the formula III and the formula IVrw-xrwyrwzrwAny point in the image is converted into an image coordinate system op-xpypThe conversion relationship in (1):
the formula five is as follows:
wherein a and b in the formulas I to V are variables, cx、cyIs the amount of optical axis deviation, fx、fyIs a focal length, Lx、LyRespectively millimeter wave radar projection coordinate system Orw-xrwyrwzrwAnd monocular camera projection coordinate system Ocw-xcwycwzcwThe spacing between the X-axes and the spacing between the y-axes;
the conversion relation between the monocular camera coordinate system and the millimeter wave radar coordinate system can be finally obtained by using the formula, and the millimeter wave radar coordinate is finally converted into the pixel coordinate;
step four: and (3) a middle-long distance ranging process:
identifying a target locomotive by using a monocular camera, wherein the identification mode of the monocular camera is realized by adopting a YOLO (YOLO) depth neural network algorithm, the locomotive identification of the neural network is carried out by using a train locomotive database which is open on the network, then a running track corresponding to the train is selected by using a track detection mode, and a target in the track is selected as the target locomotive and marked by using a rectangular frame;
converting message information identified by the millimeter wave radar into actual physical information, performing type conversion from hexadecimal system to binary system on message information data, finding out required position information through a communication protocol list corresponding to the millimeter wave radar, and finally converting the position information into a decimal physical signal for output;
and converting all the detected physical signals of all the positions in the millimeter wave radar coordinate system into a pixel coordinate system of the camera through a formula V and displaying the converted physical signals. And taking the midpoint of the bottom edge of the rectangular frame detected by the monocular camera as a reference point, selecting the pixel coordinate of the point which is closest to the Euclidean distance between all converted camera pixel coordinates and the monocular camera reference point as a final target point, and then enabling the millimeter wave radar to detect the displacement data of the final target point as final vehicle head position information.
2. The dynamic visual monitoring method for shunting trains of claim 1, wherein the type of said millimeter wave radar is ARS 408.
3. The dynamic visual monitoring method for railway shunting according to claim 1, characterized in that the number of the millimeter wave radars is two, the data processing of the two millimeter wave radars adopts a weighted average algorithm, the weighted average of the transverse distance data and the longitudinal distance data of the two millimeter wave radars is selected as the final target distance, and if a certain radar fails or a target point is not detected due to environmental problems, the detection data of the other radar is selected as the final target.
4. The dynamic visual monitoring method for railway shunting of claim 1, characterized in that:
the monitoring method further comprises the following steps:
at 30m short-range ranging: and closing the ranging function of the millimeter wave radar, starting the laser range finder as ranging equipment, displaying the image of the head position of the target vehicle by using the monocular camera as an auxiliary visual tool, and displaying the distance information obtained by the laser range finder as final position information in a pixel coordinate system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110412963.2A CN113223090A (en) | 2021-04-16 | 2021-04-16 | Dynamic visual monitoring method for railway shunting |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110412963.2A CN113223090A (en) | 2021-04-16 | 2021-04-16 | Dynamic visual monitoring method for railway shunting |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113223090A true CN113223090A (en) | 2021-08-06 |
Family
ID=77087600
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110412963.2A Pending CN113223090A (en) | 2021-04-16 | 2021-04-16 | Dynamic visual monitoring method for railway shunting |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113223090A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114194250A (en) * | 2021-12-02 | 2022-03-18 | 天津开发区文博电子有限公司 | Dynamic visual monitoring device for railway shunting |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0899580A2 (en) * | 1997-08-29 | 1999-03-03 | Kabushiki Kaisha Toshiba | Target locating system and approach guidance system |
| US20080106462A1 (en) * | 2006-11-06 | 2008-05-08 | Toyota Jidosha Kabushiki Kaisha | Object detection system and object detection method |
| CN111098815A (en) * | 2019-11-11 | 2020-05-05 | 武汉市众向科技有限公司 | ADAS front vehicle collision early warning method based on monocular vision fusion millimeter waves |
| CN111123260A (en) * | 2018-11-01 | 2020-05-08 | 北京自动化控制设备研究所 | Method for recognizing state of environmental object by using millimeter wave radar and visible light camera |
| CN111191625A (en) * | 2020-01-03 | 2020-05-22 | 浙江大学 | Object identification and positioning method based on laser-monocular vision fusion |
| WO2020135810A1 (en) * | 2018-12-29 | 2020-07-02 | 华为技术有限公司 | Multi-sensor data fusion method and device |
-
2021
- 2021-04-16 CN CN202110412963.2A patent/CN113223090A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0899580A2 (en) * | 1997-08-29 | 1999-03-03 | Kabushiki Kaisha Toshiba | Target locating system and approach guidance system |
| US20080106462A1 (en) * | 2006-11-06 | 2008-05-08 | Toyota Jidosha Kabushiki Kaisha | Object detection system and object detection method |
| CN111123260A (en) * | 2018-11-01 | 2020-05-08 | 北京自动化控制设备研究所 | Method for recognizing state of environmental object by using millimeter wave radar and visible light camera |
| WO2020135810A1 (en) * | 2018-12-29 | 2020-07-02 | 华为技术有限公司 | Multi-sensor data fusion method and device |
| CN111098815A (en) * | 2019-11-11 | 2020-05-05 | 武汉市众向科技有限公司 | ADAS front vehicle collision early warning method based on monocular vision fusion millimeter waves |
| CN111191625A (en) * | 2020-01-03 | 2020-05-22 | 浙江大学 | Object identification and positioning method based on laser-monocular vision fusion |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114194250A (en) * | 2021-12-02 | 2022-03-18 | 天津开发区文博电子有限公司 | Dynamic visual monitoring device for railway shunting |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109270534B (en) | Intelligent vehicle laser sensor and camera online calibration method | |
| CN103487034B (en) | Method for measuring distance and height by vehicle-mounted monocular camera based on vertical type target | |
| CN110390695B (en) | Laser radar and camera fusion calibration system and calibration method based on ROS | |
| CN103499337B (en) | Vehicle-mounted monocular camera distance and height measuring device based on vertical target | |
| CN103424112B (en) | A kind of motion carrier vision navigation method auxiliary based on laser plane | |
| CN107796373B (en) | Distance measurement method based on monocular vision of front vehicle driven by lane plane geometric model | |
| CN110174059B (en) | Monocular image-based pantograph height and pull-out value measuring method | |
| CN112363167A (en) | Extended target tracking method based on fusion of millimeter wave radar and monocular camera | |
| KR20200001471A (en) | Apparatus and method for detecting lane information and computer recordable medium storing computer program thereof | |
| CN112748421B (en) | Laser radar calibration method based on automatic driving of straight road section | |
| CN110850859B (en) | Robot and obstacle avoidance method and obstacle avoidance system thereof | |
| CN113850102B (en) | Vehicle-mounted vision detection method and system based on millimeter wave radar assistance | |
| CN112017248B (en) | 2D laser radar camera multi-frame single-step calibration method based on dotted line characteristics | |
| WO2020215254A1 (en) | Lane line map maintenance method, electronic device and storage medium | |
| CN115079143B (en) | Multi-radar external parameter quick calibration method and device for double-bridge steering mine card | |
| CN114295099B (en) | Ranging method based on monocular camera, vehicle-mounted ranging equipment and storage medium | |
| CN113791074A (en) | Unmanned aerial vehicle bridge crack inspection system and method based on multi-sensor fusion | |
| CN113223090A (en) | Dynamic visual monitoring method for railway shunting | |
| CN110136186A (en) | A kind of detection target matching method for mobile robot object ranging | |
| CN111830519B (en) | Multi-sensor fusion ranging method | |
| CN110091962B (en) | Monitoring method of 30 ten thousand-ton-level large-scale tanker berthing monitoring device based on virtual wall | |
| CN115100290B (en) | Monocular vision positioning method, monocular vision positioning device, monocular vision positioning equipment and monocular vision positioning storage medium in traffic scene | |
| CN110286366A (en) | Vehicle motion state information extraction method based on vision and radar cooperative perception | |
| CN116091603A (en) | Box workpiece pose measurement method based on point characteristics | |
| CN105841633B (en) | Large area optical profilometry scaling method based on double wave face interference fringe array |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210806 |