CN114170320B - Automatic positioning and working condition self-adaption method of pile driver based on multi-sensor fusion - Google Patents
Automatic positioning and working condition self-adaption method of pile driver based on multi-sensor fusion Download PDFInfo
- Publication number
- CN114170320B CN114170320B CN202111269206.0A CN202111269206A CN114170320B CN 114170320 B CN114170320 B CN 114170320B CN 202111269206 A CN202111269206 A CN 202111269206A CN 114170320 B CN114170320 B CN 114170320B
- Authority
- CN
- China
- Prior art keywords
- pile driver
- information
- pile
- vehicle
- adopting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C22/00—Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers, using pedometers
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
- G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
- G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
- G01S19/42—Determining position
- G01S19/43—Determining position using carrier phase measurements, e.g. kinematic positioning; using long or short baseline interferometry
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention discloses a method for automatically positioning a pile driver and self-adapting to working conditions based on multi-sensor fusion, which realizes the pre-sensing of a long-distance laser inertia odometer to a road environment through a multi-sensor fusion scheme, obtains the running track information and the pose information of a pile driver vehicle through the inertia odometer, accurately plans and marks the position of each pile driving point, optimizes the pile driving position by adopting a short-distance pile driving positioning identification system, provides the position information of the pile driver vehicle and the pile driving points and the pose information of the pile driver vehicle in real time, and adjusts the pile driver vehicle in real time by a central control system according to the information. The method can use other sensors to perform efficient and stable operation even in the area with weak GPS signals, realizes accurate positioning of the piling position, greatly improves the piling operation efficiency and effectively reduces the position error during piling, and greatly improves the automation and intelligence level of mechanical equipment of the pile driver.
Description
Technical Field
The invention relates to the technical field of intelligent control of a pile driver, in particular to an automatic positioning and working condition self-adaption method of the pile driver based on multi-sensor fusion.
Background
Two sides of an expressway are mostly dangerous areas, and in order to effectively reduce safety problems caused by accidents, safety protective guards are generally arranged on two sides of the expressway to ensure the safety of vehicles coming and going. When the protective guard is installed, piles need to be placed and piled so as to ensure the stability of the protective guard. If the installation position of the ground pile is not reasonable or the accuracy is not enough, the installation error of the protective guard is large, the installation fails, and the effect of safety protection cannot be achieved.
The positioning and installing process of the guardrail pile mainly depends on manual operation in the prior art, pile placing and piling are carried out in a manual measuring mode, the method is time-consuming and labor-consuming, the distance between the ground pile and the ground pile cannot be well guaranteed, and the problems that the guardrail is installed in the later period in a failed mode, and is serious or even needs to be reworked are likely to occur. In addition, due to the terrain, the problems of deflection of ground piles, excessive or insufficient ground penetration depth and the like can also occur in the piling process, the road has complicated conditions of straight lines, curves, irregular radians, upward and downward slopes and the like, the operation of the piling machine is greatly influenced, and the requirement on the skill level of an operator is high.
At present, a plurality of domestic research institutions research the positioning technology of the pile driver to a certain extent, for example, a GPS pile driving positioning system proposed by the medium-iron and large-bridge bureau mainly depends on a GPS to perform positioning measurement, but the GPS measurement is difficult to perform efficient and accurate work in a place with weak signals; wuhan Hua Chuang BeiDou technique limited company provides a big dipper pile driving system and is used for realizing the automatic function of putting the stake, nevertheless also has the unstable problem of part area signal equally, and the guarantee that the operation precision can not be fine leads to the phenomenon of unable normal work to appear in the machine. Most of the tasks of pile placing and driving on the expressway are mainly manual measurement and operation at present.
Disclosure of Invention
The invention aims to provide a pile driver automatic positioning and working condition self-adaption method based on multi-sensor fusion, which can realize accurate positioning of a pile driving position and solve the problem of guardrail installation failure caused by insufficient pile placing and driving precision manually.
In order to achieve the purpose, the automatic positioning and working condition self-adaption method of the pile driver comprises the following steps of:
s1, collecting data through a plurality of sensors arranged on a pile driver vehicle and carrying out combined calibration;
s2, recognizing lane lines by adopting a deep learning target detection algorithm and enabling a pile driver vehicle to perform lane line tracking driving;
s3, generating a laser inertia odometer by adopting a slam algorithm based on factor graph optimization, correcting by using a GPS-RTK measurement technology, and acquiring and storing the running track information of the pile driver vehicle and the pose information of the pile driver vehicle at each position;
s4, accurately planning and marking the position of each pile driving point according to the information output by the laser inertia odometer;
s5, the pile driver starts to work, the pile driver vehicle runs along the laser inertia odometer by adopting a lane line tracking and GPS-RTK mixed positioning strategy, and the position information and the pose information of the pile driver vehicle are output in real time;
s6, optimizing the piling position by adopting a positioning identification system for short-distance piling, and providing position information of a pile driver vehicle and a piling point and pose information of the pile driver vehicle in real time;
and S7, transmitting the position information of the pile driver vehicle, the position information of the pile driving point and the position and posture information of the pile driver vehicle to a central control system of the pile driver, and adjusting the pile driver vehicle in real time by the central control system according to the information.
Further, in step S1, the sensor mounted on the pile driver vehicle includes: a lidar, at least 2 cameras, an inertial sensor, and at least 2 GPS receivers; the GPS receivers are arranged in front of and behind the pile driver; the laser radar scans the surrounding environment of the pile driver by 360 degrees to collect point cloud data; the camera collects images covering the lane line, the road surface edge and the ground pile area; the inertial sensor collects the six-degree-of-freedom pose information and the acceleration information of the pile driver; the GPS receiver collects the coordinate information and azimuth information of the pile driver body.
Further, in step S1, the method for performing joint calibration on the collected data includes: establishing a geometric constraint relation between the visual extraction feature points and the radar extraction edges by adopting a pnp external reference calibration mode; performing external reference calibration between the camera and the inertial sensor, and between the laser radar and the inertial sensor by adopting a hand-eye calibration method, and establishing a geometric constraint relation; and establishing a geometric constraint relation between the inertial sensor and the GPS receiver by using calibration information in the combined inertial navigation.
Further, the method for recognizing the lane line and enabling the pile driver vehicle to perform lane line tracking driving by adopting the deep learning target detection algorithm in the step S2 comprises the following steps: s21, collecting lane line data at a piling operation place, performing labeling training on the obtained data by adopting an improved deep learning target detection algorithm based on LanNet, continuously adjusting weight parameters, and improving the accuracy and robustness of lane line identification detection;
step S22, adopting a double-camera mode, using the improved deep learning target detection algorithm of the step S21 to identify and detect the lane lines on the expressway, and then further segmenting the lane line parts by using an image segmentation algorithm;
and S23, transmitting the acquired lane line information into a central control system of the vehicle, and controlling the excavator vehicle to autonomously run at a position away from the lane line by the central control system to realize lane line tracking running.
Further, the method for generating the laser inertia odometer by adopting the slam algorithm based on factor graph optimization in the step S3, correcting by using a GPS-RTK measurement technology, and acquiring and storing the running track information of the pile driver vehicle and the pose information of the vehicle at each position comprises the following steps:
s31, starting a slam algorithm based on factor graph optimization in the lane line tracking driving process, acquiring the track of the odometer and the pose of the vehicle body at each position in a mode of tightly coupling a laser radar and an inertial sensor, and providing a good initial value for the laser odometer by the inertial sensor in a pre-integration processing mode, so that the accuracy is ensured and the calculation efficiency is greatly improved;
step S32, carrying out real-time differential GPS dynamic measurement positioning by adopting a GPS-RTK measurement technology, then converting the obtained longitude and latitude high coordinate into a northeast coordinate system through a coordinate system, and measuring the coordinate and the azimuth angle of the pile driver vehicle in real time;
and S33, inserting a laser odometer factor, an inertial sensor pre-integration factor and a Bayes tree (isam 2) used when a GPS-RTK factor node is inserted into the factor graph to perform increment smooth joint optimization, and acquiring and storing the driving track information of the pile driver vehicle and the pose information of the vehicles at all positions.
Further, the method for accurately planning and marking the position of each piling point according to the information output by the odometer in the step S4 comprises the following steps: and acquiring the road condition including road curvature and gradient information and track information of the whole expressway according to the information output by the odometer, then manually setting the initial positions of the piling points, and accurately planning and marking the piling point positions on the whole expressway by analyzing the vertical distance between the first piling point and the laser inertia odometer and the preset distance between the adjacent piling points.
Further, in step S5, the pile driver vehicle starts pile releasing and pile driving operations from the starting point of the laser inertia odometer, and during the traveling process, the pile driver vehicle tracks along a lane line and autonomously travels, and at the same time, the vehicle is observed by using a GPS-RTK positioning technology, and compared with information reserved in the laser inertia odometer, when a deviation occurs during traveling, the vehicle is timely corrected, the trajectory of the vehicle is always kept the same as that of the laser inertia original odometer, and the position information and the pose information of the pile driver vehicle are output in real time during the pile driver operation process.
Further, step S6 is to optimize the pile driving position by using a positioning and identifying system for short distance pile driving, and the method for providing the position information of the pile driving point in real time includes the following steps:
and S61, fusing data of the laser radar and the camera:
expanding point clouds scanned by a laser radar according to a mode of 0-360 degrees to form a depth map with a resolution of 1800 multiplied by 16, storing corresponding point set coordinates in kdtree, performing coordinate conversion according to external parameters between the laser radar and a camera, intercepting a part with the resolution of 600 multiplied by 16 in the depth map, performing feature matching with an image obtained by the camera, and searching three points closest to the feature points in the kdtree to obtain an average value of depths as an image feature point depth value;
triangulation is adopted for the depth of the feature point for the feature points which are not successfully associated, the depths of other associated feature points within the radius range of 1cm are selected for comparison, and if the depth distance exceeds 10cm or no associated feature point exists within the radius range, the feature point is used as an external point and does not participate in subsequent work;
step S62, acquiring the position information of the pile driving point and the road edge information through the camera:
detecting the ground piles and the ground piles to be driven in real time according to the image information acquired by the camera by adopting the deep learning target detection algorithm in the step S2, marking the positions of the ground piles and the ground piles to be driven in the image, further detecting the road edges by adopting an edge detection algorithm based on pixel gradient change, and accurately acquiring the position information of the ground piles and the ground piles to be driven and the relative position information of the ground piles and the road edges according to the image information after depth correlation;
step S63, converting the position information of the driven ground pile and the ground pile to be driven and the relative position information of the ground pile and the road edge into a geographic coordinate system, and carrying out combined optimization with the coordinate position output by the laser inertia odometer:
the position information of the ground pile and the ground pile to be driven and the relative position information of the ground pile and the road edge are converted into a geographic coordinate system, the driving point provided by a laser inertia odometer is used as an initial value, the LM nonlinear optimization algorithm is adopted, the geographic coordinate information obtained according to the image is utilized to continuously carry out iterative optimization on the coordinate position of the driving point, and the position information and the pose information of the pile driver and the driving point are continuously output.
Compared with the prior art, the invention has the following beneficial effects:
(1) The invention adopts a scheme of fusing a plurality of sensors such as a laser radar, a camera, an inertial sensor, a GPS receiver and the like to realize the pre-sensing of the long-distance laser inertial odometer on the road environment, and obtains the track information of the driving distance and the pose information of the vehicle through the laser inertial odometer, wherein the track information comprises a pitch angle, a yaw angle, a roll angle, acceleration information of three degrees of freedom and the position information of the pile driver in real time. Compared with the traditional method, the method can sense the change of the posture of the vehicle body and make effective adjustment, and can use other sensors to perform efficient and stable operation even in the area with weak GPS signals, thereby greatly solving the adverse effect of various road conditions on the working process of the pile driver.
(2) In the aspects of pile placing and pile driving positioning, the method adopts a strategy of remote pre-positioning and close-range optimization, firstly uses a laser inertia odometer to carry out long-range planning and marking on the position of the ground pile, then adopts a close-range positioning perception optimization system to carry out real-time optimization on the position of the ground pile when a pile driver works, realizes accurate positioning of the pile driving position, greatly improves the pile driving working efficiency, effectively reduces the position error generated in pile driving, solves the problems of guardrail mounting failure caused by manual pile placing and pile driving precision insufficiency and large consumption of manpower and material resources to a great extent, can reach high precision requirements even in long-distance and complicated-landform areas through the pre-positioning of the ground pile, and greatly improves the automation and intelligence level of pile driver mechanical equipment.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.
FIG. 1 is a flow chart of a method for automatic positioning of a pile driver and adaptive operation based on multi-sensor fusion;
FIG. 2 is a schematic view of multiple sensors mounted on an excavator vehicle;
in the figure, 1-first camera, 2-first GPS receiver, 3-inertial sensor, 4-second camera, 5-second GPS receiver, 6-lidar, 7-central controller.
FIG. 3 is a flow chart of generating a laser inertial odometer;
fig. 4 is a flow chart of optimizing the pile driving position using a position identification system for close-range pile driving.
Detailed Description
The following detailed description of the present invention is provided in conjunction with the accompanying drawings, but it should be understood that the scope of the present invention is not limited to the specific embodiments.
Throughout the specification and claims, unless explicitly stated otherwise, the term "comprise" or variations such as "comprises" or "comprising", etc., will be understood to imply the inclusion of a stated element or component but not the exclusion of any other element or component.
Example 1
Referring to fig. 1, an automatic positioning and working condition adaptive method for a pile driver comprises the following steps:
step S1, collecting data through a plurality of sensors arranged on a pile driver vehicle and carrying out combined calibration:
referring to fig. 2, the sensor mounted on the pile driver vehicle includes: a laser radar 6, a first camera 1, a second camera 4, an inertial sensor 3, a first GPS receiver 2 and a second GPS receiver 5; the first camera 1 and the second camera 4 are respectively installed in front of and behind the pile driver and collect images covering the lane line, the road surface edge and the ground pile area; the first GPS receiver 2 and the second GPS receiver 5 are respectively arranged in front of and behind the pile driver and used for collecting coordinate information and azimuth angle information of a pile driver body; the laser radar 6 scans the surrounding environment of the pile driver by 360 degrees to collect point cloud data; the inertial sensor 3 collects the six-degree-of-freedom pose information and the acceleration information of the pile driver. A central controller 7 is also mounted on the pile driver vehicle, and the central controller 7 is an industrial control public machine provided with a ros system and used for receiving information transmitted by each sensor and issuing instructions.
The method for carrying out combined calibration on the collected data comprises the following steps: establishing a geometric constraint relation between the visual extraction feature points and the radar extraction edges by adopting a pnp external reference calibration mode; carrying out external reference calibration between the camera and the inertial sensor and between the laser radar and the inertial sensor by adopting a hand-eye calibration method, and establishing a geometric constraint relation; and establishing a geometric constraint relation between the inertial sensor and the GPS receiver by using calibration information in the combined inertial navigation.
S2, recognizing lane lines by adopting a deep learning target detection algorithm and enabling a pile driver vehicle to perform lane line tracking driving:
s21, collecting lane line data at a piling operation place, performing labeling training on the obtained data by adopting an improved deep learning target detection algorithm based on LanNet, continuously adjusting weight parameters, and improving the accuracy and robustness of lane line identification detection;
step S22, adopting a double-camera mode, using the improved deep learning target detection algorithm of the step S21 to identify and detect the lane lines on the expressway, and then further segmenting the lane line parts by using an image segmentation algorithm;
and S23, transmitting the acquired lane line information into a central control system of the vehicle, and controlling the excavator vehicle to autonomously run at a position away from the lane line by the central control system to realize lane line tracking running.
S3, generating a laser inertia odometer by adopting a slam algorithm based on factor graph optimization, correcting by using a GPS-RTK measurement technology, and acquiring and storing the running track information of the pile driver vehicle and the pose information of the pile driver vehicle at each position:
step S31, please refer to fig. 3, starting a slam algorithm based on factor graph optimization in the lane line tracking driving process, acquiring the track of the odometer and the pose of the vehicle body at each position by adopting a tight coupling mode of a laser radar and an inertial sensor, and simultaneously providing a good initial value for the laser odometer by adopting a pre-integration processing mode through the inertial sensor, so that the accuracy is ensured and the calculation efficiency is greatly improved;
step S32, carrying out real-time differential GPS dynamic measurement positioning by adopting a GPS-RTK measurement technology, then converting the obtained longitude and latitude high coordinate into a northeast coordinate system through a coordinate system, and measuring the coordinate and the azimuth angle of the pile driver vehicle in real time;
and S33, inserting a laser odometer factor, an inertial sensor pre-integration factor and a Bayes tree (isam 2) used when a GPS-RTK factor node is inserted into the factor graph to perform increment smooth joint optimization, and acquiring and storing the driving track information of the pile driver vehicle and the pose information of the vehicles at all positions.
S4, accurately planning and marking the position of each pile driving point according to the information output by the laser inertia odometer:
the method comprises the steps of obtaining road conditions including road curvature, gradient information and track information of the whole expressway according to information output by an odometer, then manually setting initial positions of piling points, and accurately planning and marking the positions of the piling points on the whole expressway by analyzing the vertical distance between a first piling point and a laser inertia odometer and the preset distance between adjacent piling points (the distance between guardrail piles on the expressway is 2 m). Adopt this kind of mode to plan the position of piling can effectively avoid longer distance during operation because the complicated pile position of piling that brings of topography sets up the unreasonable problem that leads to the guardrail not to install, and meanwhile the staff can make in advance and predict the quantity and the type of required material, saves a large amount of manpower and materials.
S5, the pile driver starts to work, the pile driver vehicle runs along the laser inertia odometer by adopting a lane line tracking and GPS-RTK hybrid positioning strategy, and the position information and the pose information of the pile driver vehicle are output in real time:
the pile driver vehicle starts pile releasing and pile driving operation from the starting point of the laser inertia odometer, the pile driver vehicle automatically runs by adopting lane line tracking in the running process, meanwhile, the vehicle is observed by using a GPS-RTK positioning technology and is compared with the information reserved in the laser inertia odometer, the vehicle is corrected in time when deviation occurs in running, the track of the vehicle is always kept the same as that of the laser inertia original odometer, and the position information and the pose information of the pile driver vehicle are output in real time in the pile driver operation process.
Step S6, referring to fig. 4, optimizing the piling position by using the positioning and identifying system for short-distance piling, and providing position information of the pile driver vehicle and the piling point and pose information of the pile driver vehicle in real time:
and S61, fusing data of the laser radar and the camera:
when the environmental characteristic points are extracted by using the pictures provided by the camera, a large error exists in a mode of completing the triangular depth distance estimation through the movement of the camera, so that the point cloud picture acquired by adopting the multi-frame laser radar is subjected to depth correlation with the images. Firstly, point clouds scanned by a laser radar are unfolded according to a mode of 0-360 degrees to form a depth map with the resolution ratio of 1800 multiplied by 16, corresponding point set coordinates are stored in kdtree, coordinate conversion is carried out according to external parameters between the laser radar and a camera, then a part with the resolution ratio of 600 multiplied by 16 in the depth map is intercepted, feature matching is carried out on an image obtained by the camera, the visual angle of the camera is 30 degrees at the moment, three points nearest to feature points are searched in the kdtree, and the mean value of the depths is obtained and used as the depth value of feature points of the image;
triangulation is adopted to measure the depth of the feature point for the feature points which are not successfully associated, the depths of other associated feature points within the radius range of 1cm are selected for comparison, and if the depth distance exceeds 10cm or no associated feature point exists within the radius range, the feature point is taken as an external point and does not participate in subsequent work;
step S62, acquiring the position information of the pile driving point and the road edge information through the camera:
detecting the ground piles and the ground piles to be driven in real time by adopting the deep learning target detection algorithm in the step S2 according to the image information acquired by the camera, marking the positions of the ground piles and the ground piles to be driven in the image (equivalent to acquiring the pixel coordinates of the ground piles), further detecting the road edges by adopting an edge detection algorithm based on pixel gradient change, and accurately acquiring the position information of the ground piles and the ground piles to be driven and the relative position information of the ground piles and the road edges according to the image information after depth correlation;
step S63, converting the position information of the driven ground pile and the ground pile to be driven and the relative position information of the ground pile and the road edge into a geographic coordinate system, and carrying out combined optimization with the coordinate position output by the laser inertia odometer:
the position information of the ground pile and the ground pile to be driven and the relative position information of the ground pile and the road edge are converted into a geographic coordinate system, the driving point provided by a laser inertia odometer is used as an initial value, the LM nonlinear optimization algorithm is adopted, the geographic coordinate information obtained according to the image is utilized to continuously carry out iterative optimization on the coordinate position of the driving point, and the position information and the pose information of the pile driver and the driving point are continuously output.
And S7, transmitting the position information of the pile driver vehicle, the position information of the pile driving point and the position and posture information of the pile driver vehicle to a central control system of the pile driver, and adjusting the pile driver vehicle in real time by the central control system according to the information.
The foregoing description of specific exemplary embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain certain principles of the invention and its practical application to enable one skilled in the art to make and use various exemplary embodiments of the invention and various alternatives and modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims and their equivalents.
Claims (7)
1. A pile driver automatic positioning and working condition self-adaption method based on multi-sensor fusion is characterized by comprising the following steps:
s1, collecting data through a plurality of sensors arranged on a pile driver vehicle and carrying out combined calibration;
s2, recognizing lane lines by adopting a deep learning target detection algorithm and enabling a pile driver vehicle to perform lane line tracking driving;
s3, generating a laser inertia odometer by adopting a slam algorithm based on factor graph optimization, correcting by using a GPS-RTK measurement technology, and acquiring and storing the running track information of the pile driver vehicle and the pose information of the pile driver vehicle at each position;
s4, accurately planning and marking the position of each pile driving point according to the information output by the laser inertia odometer;
s5, the pile driver starts to work, the pile driver vehicle travels along the laser inertia odometer by adopting a lane line tracking and GPS-RTK hybrid positioning strategy, and position information and pose information of the pile driver vehicle are output in real time;
s6, optimizing the piling position by adopting a positioning identification system for short-distance piling, and providing position information of a pile driver vehicle and a piling point and pose information of the pile driver vehicle in real time;
s7, transmitting the position information of the pile driver vehicle, the position information of the pile driving point and the pose information of the pile driver vehicle to a central control system of the pile driver, and adjusting the pile driver vehicle in real time by the central control system according to the information;
the method for optimizing the piling position by adopting the positioning and identifying system for short-distance piling in the step S6 and providing the position information of the piling point in real time comprises the following steps:
and S61, fusing data of the laser radar and the camera:
expanding point clouds scanned by a laser radar according to a mode of 0-360 degrees to form a depth map with a resolution of 1800 multiplied by 16, storing corresponding point set coordinates in kdtree, performing coordinate conversion according to external parameters between the laser radar and a camera, intercepting a part with the resolution of 600 multiplied by 16 in the depth map, performing feature matching with an image obtained by the camera, and searching three points closest to the feature points in the kdtree to obtain an average value of depths as an image feature point depth value;
triangulation is adopted for the depth of the feature point for the feature points which are not successfully associated, the depths of other associated feature points within the radius range of 1cm are selected for comparison, and if the depth distance exceeds 10cm or no associated feature point exists within the radius range, the feature point is used as an external point and does not participate in subsequent work;
step S62, acquiring the position information of the pile driving point and the road edge information through the camera:
detecting the ground piles and the ground piles to be driven in real time according to the image information acquired by the camera by adopting the deep learning target detection algorithm in the step S2, marking the positions of the ground piles and the ground piles to be driven in the image, further detecting the road edges by adopting an edge detection algorithm based on pixel gradient change, and accurately acquiring the position information of the ground piles and the ground piles to be driven and the relative position information of the ground piles and the road edges according to the image information after depth correlation;
step S63, converting the position information of the driven ground pile and the ground pile to be driven and the relative position information of the ground pile and the road edge into a geographic coordinate system, and carrying out joint optimization with the coordinate position output by the laser inertia odometer:
the position information of the ground pile and the ground pile to be driven and the relative position information of the ground pile and the road edge are converted into a geographic coordinate system, the driving point provided by a laser inertia odometer is used as an initial value, the LM nonlinear optimization algorithm is adopted, the geographic coordinate information obtained according to the image is utilized to continuously carry out iterative optimization on the coordinate position of the driving point, and the position information and the pose information of the pile driver and the driving point are continuously output.
2. The automatic positioning and operating condition adaptive method for the pile driver as recited in claim 1, wherein:
in step S1, a sensor mounted on a pile driver vehicle includes: a lidar, at least 2 cameras, an inertial sensor, and at least 2 GPS receivers; the GPS receivers are arranged in front of and behind the pile driver; the laser radar scans the surrounding environment of the pile driver by 360 degrees to collect point cloud data; the method comprises the following steps that a camera collects images covering a lane line, a road surface edge and a ground pile area; the inertial sensor collects pose information and acceleration information of the pile driver in six degrees of freedom; the GPS receiver collects coordinate information and azimuth information of the pile driver body.
3. The automatic pile driver positioning and condition adaptive method as recited in claim 1, wherein:
in step S1, the method for performing the joint calibration on the collected data includes: establishing a geometric constraint relation between the visual extraction feature points and the radar extraction edges by adopting a pnp external reference calibration mode; carrying out external reference calibration between the camera and the inertial sensor and between the laser radar and the inertial sensor by adopting a hand-eye calibration method, and establishing a geometric constraint relation; and establishing a geometric constraint relation between the inertial sensor and the GPS receiver by using calibration information in the combined inertial navigation.
4. The automatic pile driver positioning and condition adaptive method according to claim 1, wherein the method for performing lane line identification and lane line tracking driving of pile driver vehicles by using a deep learning target detection algorithm in step S2 comprises the steps of:
s21, collecting lane line data at a piling operation place, performing labeling training on the obtained data by adopting an improved deep learning target detection algorithm based on LanNet, continuously adjusting weight parameters, and improving the accuracy and robustness of lane line identification detection;
step S22, adopting a double-camera mode, using the improved deep learning target detection algorithm of the step S21 to identify and detect the lane lines on the expressway, and then further segmenting the lane line parts by using an image segmentation algorithm;
and S23, transmitting the acquired lane line information into a central control system of the vehicle, and controlling the excavator vehicle to autonomously run at a position away from the lane line by the central control system to realize lane line tracking running.
5. The automatic positioning and working condition adaptive method for the pile driver as claimed in claim 1, wherein the step S3 adopts a slam algorithm based on factor graph optimization to generate a laser inertia odometer, uses a GPS-RTK measurement technology to carry out correction, and the method for acquiring and storing the running track information of the pile driver vehicle and the pose information of each position vehicle comprises the following steps:
s31, starting a slam algorithm based on factor graph optimization in the lane line tracking driving process, acquiring the track of the odometer and the pose of the vehicle body at each position in a mode of tightly coupling a laser radar and an inertial sensor, and providing a good initial value for the laser odometer by the inertial sensor in a pre-integration processing mode, so that the accuracy is ensured and the calculation efficiency is greatly improved;
step S32, carrying out real-time differential GPS dynamic measurement positioning by adopting a GPS-RTK measurement technology, then converting the obtained longitude and latitude high coordinate into a northeast coordinate system through a coordinate system, and measuring the coordinate and the azimuth angle of the pile driver vehicle in real time;
and S33, performing increment smooth joint optimization on the Bayes tree used when the laser odometer factor, the inertial sensor pre-integration factor and the GPS-RTK factor node are inserted into the factor graph, and acquiring and storing the running track information of the pile driver vehicle and the pose information of the vehicles at all positions.
6. Pile driver automatic positioning and condition adaptation method according to claim 1,
step S4, the method for accurately planning and marking the position of each pile driving point according to the information output by the odometer comprises the following steps: and acquiring the road condition including road curvature and gradient information and track information of the whole expressway according to the information output by the odometer, then manually setting the initial positions of the piling points, and accurately planning and marking the piling point positions on the whole expressway by analyzing the vertical distance between the first piling point and the laser inertia odometer and the preset distance between the adjacent piling points.
7. The automatic pile driver positioning and condition adaptive method as recited in claim 1, wherein: and step S5, the pile driver vehicle starts pile releasing and pile driving operation from the starting point of the laser inertia odometer, the pile driver vehicle tracks and autonomously travels by adopting a lane line in the traveling process, meanwhile, the vehicle is observed by using a GPS-RTK positioning technology and compared with the information reserved in the laser inertia odometer, the vehicle is timely corrected when deviation occurs in traveling, the track of the vehicle is always kept the same as that of the laser inertia original odometer, and the position information and the pose information of the pile driver vehicle are output in real time in the pile driver operation process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111269206.0A CN114170320B (en) | 2021-10-29 | 2021-10-29 | Automatic positioning and working condition self-adaption method of pile driver based on multi-sensor fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111269206.0A CN114170320B (en) | 2021-10-29 | 2021-10-29 | Automatic positioning and working condition self-adaption method of pile driver based on multi-sensor fusion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114170320A CN114170320A (en) | 2022-03-11 |
| CN114170320B true CN114170320B (en) | 2022-10-28 |
Family
ID=80477676
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111269206.0A Active CN114170320B (en) | 2021-10-29 | 2021-10-29 | Automatic positioning and working condition self-adaption method of pile driver based on multi-sensor fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114170320B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113805479B (en) * | 2021-10-09 | 2023-08-18 | 上海亥伯智能科技有限公司 | Automatic operation control method and control system for punching and grabbing machine |
| CN114703851A (en) * | 2022-03-16 | 2022-07-05 | 中国黄金集团建设有限公司 | Pile foundation positioning system |
| CN114838726A (en) * | 2022-04-20 | 2022-08-02 | 哈尔滨理工大学 | GPS data correction algorithm based on multi-sensor data fusion |
| CN114993310A (en) * | 2022-05-12 | 2022-09-02 | 广西大学 | Automatic positioning method of intelligent pile driver |
| JP7256576B1 (en) * | 2022-12-12 | 2023-04-12 | 株式会社Ｄａチャレンジャーズ | Pile construction method and pile measuring device |
| CN117351338B (en) * | 2023-12-04 | 2024-02-13 | 北京理工大学前沿技术研究院 | Automatic pile correction method, system and equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107728175A (en) * | 2017-09-26 | 2018-02-23 | 南京航空航天大学 | The automatic driving vehicle navigation and positioning accuracy antidote merged based on GNSS and VO |
| CN111595333A (en) * | 2020-04-26 | 2020-08-28 | 武汉理工大学 | Modularized unmanned vehicle positioning method and system based on visual inertial laser data fusion |
| CN112304307A (en) * | 2020-09-15 | 2021-02-02 | 浙江大华技术股份有限公司 | Positioning method and device based on multi-sensor fusion and storage medium |
| CN112819711A (en) * | 2021-01-20 | 2021-05-18 | 电子科技大学 | Monocular vision-based vehicle reverse positioning method utilizing road lane line |
| CN112904395A (en) * | 2019-12-03 | 2021-06-04 | 青岛慧拓智能机器有限公司 | Mining vehicle positioning system and method |
| CN113199474A (en) * | 2021-04-25 | 2021-08-03 | 广西大学 | Robot walking and operation intelligent cooperative motion planning method |
| CN113252033A (en) * | 2021-06-29 | 2021-08-13 | 长沙海格北斗信息技术有限公司 | Positioning method, positioning system and robot based on multi-sensor fusion |
| CN113358112A (en) * | 2021-06-03 | 2021-09-07 | 北京超星未来科技有限公司 | Map construction method and laser inertia odometer |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107505644B (en) * | 2017-07-28 | 2020-05-05 | 武汉理工大学 | Three-dimensional high-precision map generation system and method based on vehicle-mounted multi-sensor fusion |
| CN110207714B (en) * | 2019-06-28 | 2021-01-19 | 广州小鹏自动驾驶科技有限公司 | Method for determining vehicle pose, vehicle-mounted system and vehicle |
| CN111522043B (en) * | 2020-04-30 | 2023-07-25 | 北京联合大学 | Unmanned vehicle laser radar quick re-matching positioning method |
| CN112083725B (en) * | 2020-09-04 | 2021-11-30 | 湖南大学 | Structure-shared multi-sensor fusion positioning system for automatic driving vehicle |
| CN113238576A (en) * | 2021-05-07 | 2021-08-10 | 北京中科遥数信息技术有限公司 | Positioning method for unmanned aerial vehicle and related device |
-
2021
- 2021-10-29 CN CN202111269206.0A patent/CN114170320B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107728175A (en) * | 2017-09-26 | 2018-02-23 | 南京航空航天大学 | The automatic driving vehicle navigation and positioning accuracy antidote merged based on GNSS and VO |
| CN112904395A (en) * | 2019-12-03 | 2021-06-04 | 青岛慧拓智能机器有限公司 | Mining vehicle positioning system and method |
| CN111595333A (en) * | 2020-04-26 | 2020-08-28 | 武汉理工大学 | Modularized unmanned vehicle positioning method and system based on visual inertial laser data fusion |
| CN112304307A (en) * | 2020-09-15 | 2021-02-02 | 浙江大华技术股份有限公司 | Positioning method and device based on multi-sensor fusion and storage medium |
| CN112819711A (en) * | 2021-01-20 | 2021-05-18 | 电子科技大学 | Monocular vision-based vehicle reverse positioning method utilizing road lane line |
| CN113199474A (en) * | 2021-04-25 | 2021-08-03 | 广西大学 | Robot walking and operation intelligent cooperative motion planning method |
| CN113358112A (en) * | 2021-06-03 | 2021-09-07 | 北京超星未来科技有限公司 | Map construction method and laser inertia odometer |
| CN113252033A (en) * | 2021-06-29 | 2021-08-13 | 长沙海格北斗信息技术有限公司 | Positioning method, positioning system and robot based on multi-sensor fusion |
Non-Patent Citations (2)
| Title |
|---|
| 《Design and Experimental Verification of an Intelligent Fire-fighting Robot》;Xinyu Gao，et al;《IEEE International Conference on Advanced Robotics and Mechatronics》;20210731;全文 * |
| 《LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping》;Tixiao Shan,et al;《arXiv:2007.00258v3》;20200714;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114170320A (en) | 2022-03-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114170320B (en) | Automatic positioning and working condition self-adaption method of pile driver based on multi-sensor fusion | |
| CN111551958B (en) | Mining area unmanned high-precision map manufacturing method | |
| Akai et al. | Robust localization using 3D NDT scan matching with experimentally determined uncertainty and road marker matching | |
| CN100494900C (en) | Environment sensing one-eye visual navigating method adapted to self-aid moving vehicle | |
| CN108345005B (en) | Real-time continuous autonomous positioning and orienting system and navigation positioning method of tunnel boring machine | |
| CN105719284A (en) | Data processing method, device and terminal | |
| CN102368158B (en) | Navigation positioning method of orchard machine | |
| CN207752371U (en) | A kind of robot autonomous navigation device and robot | |
| WO2017149813A1 (en) | Sensor calibration system | |
| CN107422730A (en) | The AGV transportation systems of view-based access control model guiding and its driving control method | |
| CN110208842A (en) | Vehicle high-precision locating method under a kind of car networking environment | |
| CN112346463B (en) | Unmanned vehicle path planning method based on speed sampling | |
| US20220363263A1 (en) | Automated bump and/or depression detection in a roadway | |
| CN114199240B (en) | Two-dimensional code, laser radar and IMU fusion positioning system and method without GPS signal | |
| CN111693050A (en) | Indoor medium and large robot navigation method based on building information model | |
| CN112924955B (en) | Roadside laser radar point cloud coordinate dynamic correction method | |
| CN106127787A (en) | A kind of camera calibration method based on Inverse projection | |
| CN110108269A (en) | AGV localization method based on Fusion | |
| CN111506069B (en) | All-weather all-ground crane obstacle identification system and method | |
| US20210240195A1 (en) | Systems and methods for utilizing images to determine the position and orientation of a vehicle | |
| KR20210058640A (en) | Vehicle navigaton switching device for golf course self-driving cars | |
| CN109839634A (en) | A kind of subject fusion method of vehicle-mounted camera and radar | |
| CN116704019A (en) | Drilling and anchoring robot monocular vision positioning method based on anchor rod network | |
| JP2020204501A (en) | Self position estimation device of vehicle, and vehicle | |
| JP7217483B1 (en) | Pile core position investigation system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |