CN110147106A - Has the intelligent Mobile Service robot of laser and vision fusion obstacle avoidance system - Google Patents
Has the intelligent Mobile Service robot of laser and vision fusion obstacle avoidance system Download PDFInfo
- Publication number
- CN110147106A CN110147106A CN201910454798.XA CN201910454798A CN110147106A CN 110147106 A CN110147106 A CN 110147106A CN 201910454798 A CN201910454798 A CN 201910454798A CN 110147106 A CN110147106 A CN 110147106A
- Authority
- CN
- China
- Prior art keywords
- data
- laser
- barrier
- module
- obstacle avoidance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 53
- 230000004888 barrier function Effects 0.000 claims abstract description 87
- 230000033001 locomotion Effects 0.000 claims abstract description 50
- 238000006243 chemical reaction Methods 0.000 claims abstract description 24
- 238000004422 calculation algorithm Methods 0.000 claims description 26
- 238000012545 processing Methods 0.000 claims description 20
- 238000000034 method Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 8
- 238000003062 neural network model Methods 0.000 claims description 6
- 238000013135 deep learning Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000012549 training Methods 0.000 claims description 4
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 238000000465 moulding Methods 0.000 claims 1
- 230000003044 adaptive effect Effects 0.000 abstract description 5
- 230000000007 visual effect Effects 0.000 abstract description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 238000012152 algorithmic method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000002604 ultrasonography Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002459 sustained effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0238—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0255—Control of position or course in two dimensions specially adapted to land vehicles using acoustic signals, e.g. ultra-sonic singals
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0257—Control of position or course in two dimensions specially adapted to land vehicles using a radar
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- General Physics & Mathematics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Automation & Control Theory (AREA)
- Electromagnetism (AREA)
- Acoustics & Sound (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Traffic Control Systems (AREA)
Abstract
The present invention relates to intelligent Mobile Service robot fields, has the intelligent Mobile Service robot of laser and vision fusion obstacle avoidance system, laser and vision fusion obstacle avoidance system include hardware system and navigation obstacle avoidance system, hardware system includes depth camera, laser radar, airborne PC, motion control board and motor driver, and navigation obstacle avoidance system includes barrier positioning and data conversion module, obstacle classification identification module, data fusion module, laser navigation frame, chassis drive module and bobbin movement control module；Navigation obstacle avoidance system operates on motion control board in addition to bobbin movement control module, remaining is operated on airborne PC；Laser radar, depth camera and motion control board connect airborne PC, motor driver connects motion control board, the system merges visual sensor in two-dimensional laser avoidance, the type of barrier and the three-dimensional information of barrier are considered in obstacle, improve accuracy, robustness and the adaptive capacity to environment of intelligent Mobile Service robot obstacle-avoiding.
Description
Technical field
The present invention relates to intelligent Mobile Service robot fields, more particularly to the avoidance system of intelligent Mobile Service robot
Command domain.
Background technique
Intelligent Mobile Service robot can pass through voice, vision, tactile etc. in the continuous real-time autonomous of multiple scenes
Multiple sensors interact with environment and personnel and provide corresponding service can to substitute the role of lobby manager, waiter
Applied to scenes such as family, hotel, bank, airports.In order to guarantee safety, service robot needs during autonomous
The barriers such as tables and chairs, animal, the people in scene are avoided in time.The mode of the avoidance of intelligent Mobile Service robot generally passes through biography
Sensor obtains the location information of outer barrie object, and a local path is cooked up on cost map, passes through kinetic control system
It is moved along the path of planning, so as to avoiding obstacles.
The avoidance mode of service robot mainly includes avoiding obstacles by supersonic wave and TOF avoidance at present；Avoiding obstacles by supersonic wave by
Ultrasonic wave module, directional transmissions and reception ultrasonic wave are accessed in robot, and barrier avoiding function is realized according to the range information detected；
For TOF avoidance by infrared ray or the light beam of the specific wavelength of Laser emission, record reflection interval is poor, calculates barrier nearby
Range distribution situations such as；Existing avoidance mode uses the single-sensors such as ultrasound, two-dimensional laser, since single-sensor has
There is limitation, perfect obstacle information can not be obtained by only relying on a kind of sensor；Finishing journey of the avoiding obstacles by supersonic wave to reflecting surface
Spend more demanding, when in face of the barrier of no albedo or weak albedo, safety problem cannot be protected；Two dimension
Laser avoidance cannot achieve the Classification and Identification of barrier in environment, so that different Robot dodge strategies is taken for different classifications,
And since two-dimensional laser is only capable of capturing the ambient condition information with laser sensor riding position for sustained height plane,
For special-shaped barrier, such as the door of hollow desk, lower hollow, its specific position can not be judged.
Summary of the invention
The purpose of the present invention is to provide one kind to merge visual sensor in two-dimensional laser avoidance, and barrier is considered in obstacle
Hinder the type of object and the three-dimensional information of barrier, improves accuracy, robustness and the environment of intelligent Mobile Service robot obstacle-avoiding
A kind of laser vision of intelligent Mobile Service robot of adaptability merges obstacle avoidance system.
To achieve the above object, the technical scheme is that the intelligent mobile of tool laser and vision fusion obstacle avoidance system
Service robot, the laser and vision fusion obstacle avoidance system include the hardware being laid in Information Mobile Service robot fuselage main body
System and navigation obstacle avoidance system, the hardware system includes depth camera, laser radar, airborne PC, motion control board and electricity
Machine driver, the navigation obstacle avoidance system include barrier positioning and data conversion module, obstacle classification identification module, data
Fusion Module, laser navigation frame and bobbin movement control module；The bobbin movement control module operates in motion control board
On, the barrier positioning and data conversion module, obstacle classification identification module, data fusion module and laser navigation frame
It operates on airborne PC；The laser radar, depth camera and motion control board connect airborne PC, the motor driver connection
Motion control board.
The navigation of the laser and vision fusion obstacle avoidance system and barrier-avoiding method are the vision numbers obtained by depth camera
According to barrier positioning and data conversion module and obstacle classification identification module is sent respectively to, swashed by what laser radar obtained
Light data is sent to data fusion module, and vision data is carried out processing and obtains barrier by the barrier positioning and data conversion module
The location information data of object are hindered to be sent to data fusion module, the obstacle classification identification module handles vision data
Show that the type information data of barrier is sent to data fusion module, the data fusion module is by laser data and barrier
Location information data and the type information data of barrier carry out processing and obtain the location information data of fused barrier
It is sent to laser navigation frame, the location information data of barrier is carried out processing and obtains new part by the laser navigation frame
Cost map, the laser navigation frame carry out local paths planning and show that local paths planning data are sent to chassis driving mould
Local paths planning data are carried out processing and show that bobbin movement control data are sent to chassis and transport by block, the chassis drive module
Dynamic control module, the bobbin movement control module send motion control commands to motor driver.
The obstacle classification identification module is instructed by the deep learning algorithm of convolutional neural networks using yolo V3 in advance
Practice model the barrier of special scenes is trained to obtain trained neural network model, the trained neural network
Model can carry out processing judgement according to the color image information data captured in the vision data obtained by depth camera and obtain
The type information data of the barrier.
The barrier positioning and data conversion module include three-dimensional information detection and the virtual two-dimensional laser letter of barrier
Breath conversion, three-dimensional information detection convert what airborne PC can be identified for the depth image that depth camera acquires by image pick-up card
Digital picture simultaneously carries out the depth information that image preprocessing obtains 3-D image, and the conversion of virtual two-dimensional laser intelligence passes through
The depth information of 3-D image is converted to virtual two-dimensional laser intelligence data, institute by depthimage_to_laserscan algorithm
State the location information data that virtual two-dimensional laser intelligence data is the barrier.
The data fusion module merges virtual two-dimensional laser intelligence data, the class of barrier by Kalman filtering algorithm
Point cloud information data in type information data and laser data obtain the location information data of the barrier.
The hardware system further includes ultrasonic sensor and mechanical obstacle avoidance sensor, the ultrasonic sensor and machinery
Obstacle avoidance sensor connects the bobbin movement control module on motion control board, the ultrasonic sensor and mechanical obstacle avoidance sensor
The data that sensing obtains are sent to bobbin movement control module, the bobbin movement control module receive ultrasonic sensor and
It is stopping that the data that mechanical obstacle avoidance sensor sensing obtains, which send control command to motor driver when being barrier sensing data,.
Barrier sensing data is sent when the distance sensing of the ultrasonic sensor is 10-50 centimetres；It is described mechanical anti-
Hit transmission barrier sensing data when sensor collides.
By using above-mentioned technical proposal, the beneficial effects of the present invention are: the present invention has laser and vision merges avoidance system
The intelligent Mobile Service robot of system is combined using laser radar and depth camera by above-mentioned by above-mentioned system structure
The navigation of system of the present invention and barrier-avoiding method realize a kind of laser and vision merge carry out laser navigation frame
The navigation of the local paths planning of frame and barrier-avoiding method, to realize that the present invention improves the essence of intelligent Mobile Service robot obstacle-avoiding
True property, the purpose of robustness and adaptive capacity to environment.
The laser data obtained by the application of laser radar obtains accurate environment orientation, range information, establishes
Accurate environmental map is broken the barriers positioning and data conversion module and obstacle by the data that the application of depth camera obtains
The processing of object Classification and Identification module carries out depth perception to ambient enviroment, by deep learning algorithm integration in intelligent Mobile Service
On robot platform, to show that the location information data of barrier obtain the target type and three-dimensional position of peripheral obstacle
Information, the i.e. three-dimensional information of vision, by will be by the way that the three dimensional local information of vision to be converted to the location information of barrier
Data, that is, laser two-dimensional surface map, and the type information of barrier is combined, it is sent to laser navigation frame and handle
New local cost map out, then local paths planning is carried out to reach the avoidance navigation that laser is merged with vision, realize this
Invention merges visual sensor in two-dimensional laser avoidance, and the type of barrier and the three-dimensional letter of barrier are considered in obstacle
Breath improves accuracy, the purpose of robustness and adaptive capacity to environment of intelligent Mobile Service robot obstacle-avoiding.
Detailed description of the invention
Fig. 1 and Fig. 2 is the structural schematic diagram of intelligent Mobile Service robot of the present invention；
Fig. 3 is the structural block diagram of hardware system of the present invention；
Fig. 4 is the structural block diagram of laser of the present invention and vision fusion obstacle avoidance system.
In figure:
Fuselage main body 1；Walking mechanism 2；Depth camera 31；Laser radar 32；
Airborne PC33；Motion control board 34；Motor driver 35；
Barrier positioning and data conversion module 41；Obstacle classification identification module 42；
Data fusion module 43；Laser navigation frame 44；
Bobbin movement control module 45；Ultrasonic sensor 36；Mechanical obstacle avoidance sensor 37.
Specific embodiment
In order to further explain the technical solution of the present invention, being explained in detail below by specific embodiment the present invention
It states.
The intelligent Mobile Service robot of tool laser disclosed by the invention and vision fusion obstacle avoidance system, as shown in Figure 1, Figure 2, schemes
Shown in 3 and Fig. 4, the intelligent Mobile Service machine in figure is artificially similar to the fuselage main body 1 of the contour structures of mankind's figure, fuselage master
The bottom of body 1 is walking mechanism 2, and intelligent Mobile Service robot of the invention and existing intelligent Mobile Service robot are not
It is that the intelligent Mobile Service robot invented includes laser and vision fusion obstacle avoidance system with place, which includes laying
Hardware system and navigation obstacle avoidance system in fuselage main body, the specific hardware system includes depth camera 31, laser thunder
Up to 32, airborne PC33, motion control board 34 and motor driver 35, the navigation obstacle avoidance system includes barrier positioning and data
Conversion module 41, obstacle classification identification module 42, data fusion module 43, laser navigation frame 44 and bobbin movement control mould
Block 45, the bobbin movement control module 45 operate on motion control board 34, the barrier positioning and data conversion module
41, obstacle classification identification module 42, data fusion module 43 and laser navigation frame 44 operate on airborne PC33；It is described to swash
Optical radar 32, depth camera 31 and motion control board 34 connect airborne PC33, and network interface communication can be used in connection here, USB leads to
The modes such as letter, serial communication, signal wire connect, and the motor driver 35 connects motion control board 34 for driving walking mechanism
2, airborne PC33, motion control board 34 and motor driver 35 are generally arranged inside fuselage main body 1, depth camera 31 and swash
Optical radar 32 is then arranged on the shell of fuselage main body 1, working end outwardly, as illustrated in the drawing, depth camera 31 and laser
Intelligent Mobile Service robot is arranged in similar to the leading flank of the contour structures of mankind's figure, intelligent Mobile Service machine in radar 32
The equipment such as also settable display screen being connect with airborne PC on the shell of the fuselage main body 1 of people, usual intelligent Mobile Service machine
People, which may also include, loudspeaker, sound receiver etc., since these are not main improvements of the invention, this specific embodiment party
Description no longer just is illustrated to other component equipment in formula.
The navigation of laser of the invention and vision fusion obstacle avoidance system and barrier-avoiding method are such that
Barrier positioning and data conversion module 41 and barrier are sent respectively to by the vision data that depth camera 31 obtains
Hinder object Classification and Identification module 42；The obstacle classification identification module 42 has passed through the deep learning algorithm of convolutional neural networks (
The neural network deep learning algorithm known, relevant information can be arrived on network, is just not described in detail here) utilize yolo V3
(yolo is a kind of known algorithmic methods, and full name is the yolo that You Only Look Once, yolo V3 are yolo series
V3 is the latest algorithm and algorithm known of the series, relevant information can be arrived on network, is just not described in detail here,
In addition, illustratively can be realized can also apply with the calculation method of the similar or identical algorithm effect of yolo V3 algorithm here, this
A kind of achievable yolo V3 algorithm suitable for intelligent Mobile Service robot is only disclosed in embodiment, any algorithm is adopted
With being within the scope of the present invention) pre-training model the barrier of special scenes is trained to obtain it is trained
Neural network model, the trained neural network model can be according to capturing in the vision data obtained by depth camera
Color image information data carry out the type information data that processing judgement obtains the barrier；The barrier positioning and data
Conversion module 41 includes the three-dimensional information detection and the conversion of virtual two-dimensional laser intelligence of barrier, and three-dimensional information detection passes through image
The digital picture that capture card (existing product) converts airborne PC33 for the depth image that depth camera 31 acquires and can identify is gone forward side by side
Row image preprocessing obtains the depth information of 3-D image, and the conversion of virtual two-dimensional laser intelligence passes through depthimage_to_
Laserscan algorithm (a kind of known algorithmic methods can arrive relevant information on network, just be not described in detail here, in addition,
Here it illustratively can be realized the calculation method with the similar or identical algorithm effect of depthimage_to_laserscan algorithm
It can also apply, a kind of achievable depthimage_to_ suitable for intelligent Mobile Service robot is only disclosed in the present embodiment
The use of laserscan algorithm, any algorithm is within the scope of the present invention) depth information of 3-D image is turned
It is changed to virtual two-dimensional laser intelligence data, the virtual two-dimensional laser intelligence data is the location information number of the barrier
According to.
Data fusion module 43, the barrier positioning and data are sent to by the laser data that laser radar 32 obtains
Vision data is carried out processing and show that the location information data of barrier are sent to data fusion module 43 by conversion module 41, described
Vision data is carried out processing and show that the type information data of barrier is sent to data fusion by obstacle classification identification module 42
Module 43, the data fusion module 43 is by the type information number of the location information data and barrier of laser data and barrier
Show that the location information data of fused barrier is sent to laser navigation frame 44 according to processing is carried out, we first explain here
Laser navigation frame 44 once.
Laser navigation frame 44 is a kind of known laser navigation technology, the frame generally included map service module,
Move_base module, chassis drive module, tf variation module, amcl locating module, odometer module etc., the move_base
Module generally includes global cost map, global path planning, local cost map, local paths planning etc., one to navigate
Kind of method briefly describe here by grating map importing map service module carry out Map Data Transmission that processing obtains to
Global cost map, navigation target position determine by global path obtain global cost map progress global path planning and
Global path planning data are calculated through local paths planning in tf variation module, amcl locating module, odometer resume module
It is sent to chassis drive module, when there is the case where part changes map, show that local cost map is replaced into global cost
The corresponding part of map simultaneously goes out local path replacement into global path planning, using other modules by local paths planning
Processing show that Route Planning Data is sent to chassis drive module, to reach navigation function, in existing some airmanships
Be also applied to laser radar, but the application of the prior art is the use for establishing map early period, and it is of the invention then not
Together, the present invention is that laser radar and depth camera connected applications are continued with description embodiments of the present invention.
The data fusion module 43 merges point cloud information data in laser data, virtual by Kalman filtering algorithm
The type information data of two-dimensional laser information data and barrier obtains the location information data of the barrier, illustrates one here
Under can be realized the calculation method of the data fusion there are also other algorithm, can also be applied to the present invention, only disclosed in the present embodiment
A kind of achievable Kalman filtering algorithm suitable for intelligent Mobile Service robot, actually can be according to live use environment institute
Algorithm effect to be achieved determines used algorithm, and the use of any algorithm is within the scope of the present invention.
The laser navigation frame 44 can be same as above published method by the location information data of barrier carry out processing obtain it is new
Local cost map replace the corresponding part into global cost map, carry out the calculating of local path path planning etc., obtain
Local paths planning data, or replace into global path planning, the laser navigation frame 44 carries out local paths planning and obtains
Local paths planning data (data for as reaching navigation avoidance effect) are sent to chassis drive module, the chassis driving
Local paths planning data are carried out processing and show that bobbin movement control data are sent to bobbin movement control module 45, institute by module
It states bobbin movement control module 45 and sends motion control commands to motor driver 35, to reach control intelligent Mobile Service machine
The navigation avoidance of device people, and be to merge visual sensor in two-dimensional laser avoidance, the type of barrier is considered in obstacle
With the three-dimensional information of barrier, accuracy, robustness and the adaptive capacity to environment of intelligent Mobile Service robot obstacle-avoiding are improved.
In the present invention, the hardware system may also include ultrasonic sensor 36 and mechanical obstacle avoidance sensor 37, described super
Sonic sensor 36 and mechanical obstacle avoidance sensor 37 connect the bobbin movement control module on motion control board 34, the ultrasonic wave
The data that sensor 36 and the mechanical sensing of obstacle avoidance sensor 37 obtain can be transmitted directly to bobbin movement control module, transport on chassis
It is that barrier senses number that dynamic control module, which receives ultrasonic sensor 36 and the data of the mechanical sensing of obstacle avoidance sensor 37 acquisition,
According to when to the transmission control command of motor driver 35 to stop, the distance sensing of ultrasonic sensor 36 is set as will be described
Barrier sensing data is sent when 10-50 centimetres (for 20 centimetres or can preferably be set according to actual conditions), makes intelligent sliding
Dynamic service robot stop motion, the machinery obstacle avoidance sensor 37 then send barrier when touching or colliding and sense number
According to, make the stop motion of intelligent Mobile Service robot,.
As shown in the figure, the ultrasonic sensor 36 and mechanical obstacle avoidance sensor 37 and depth camera 31 and laser radar
The 32 identical intelligent Mobile Service robots that are arranged in are shown in figure similar to the leading flank of the contour structures of mankind's figure, described
Ultrasonic sensor 36 and mechanical obstacle avoidance sensor 37 and depth camera 31 and laser radar 32 are in intelligent Mobile Service robot
The leading flank of the contour structures of similar mankind's figure is that interval is laid from top to down, and for depth camera 31 in the top, machinery is anti-
Hit sensor 37 bottom, laser radar 32 and ultrasonic sensor 36 depth camera 31 and mechanical obstacle avoidance sensor 37 it
Between, lay principle is advisable with being conducive to expand visual range, expansion of laser light scanning range, expanding sensing scope, the ultrasound
The setting of wave sensor 36 and mechanical obstacle avoidance sensor 37 can further increase the accurate of intelligent Mobile Service robot obstacle-avoiding
Property, robustness and adaptive capacity to environment.
Above-described embodiment and schema and non-limiting product form and style of the invention, any technical field it is common
The appropriate changes or modifications that technical staff does it all should be regarded as not departing from patent category of the invention.
Claims (10)
1. having the intelligent Mobile Service robot of laser and vision fusion obstacle avoidance system, it is characterised in that: the laser and vision
Fusion obstacle avoidance system include the hardware system being laid in Information Mobile Service robot fuselage main body and navigation obstacle avoidance system, it is described
Hardware system includes depth camera, laser radar, airborne PC, motion control board and motor driver, the navigation obstacle avoidance system
Including barrier positioning and data conversion module, obstacle classification identification module, data fusion module, laser navigation frame and bottom
Disk motion-control module；The bobbin movement control module operates on motion control board, and the barrier positioning and data turn
Mold changing block, obstacle classification identification module, data fusion module and laser navigation frame operate on airborne PC；The laser thunder
It reaches, depth camera connects airborne PC with motion control board, the motor driver connection motion control board.
2. the intelligent Mobile Service robot of tool laser and vision fusion obstacle avoidance system, feature exist as described in claim 1
In the navigation of: the laser and vision fusion obstacle avoidance system and barrier-avoiding method be the vision data difference obtained by depth camera
It is sent to barrier positioning and data conversion module and obstacle classification identification module, the laser data obtained by laser radar
It is sent to data fusion module, vision data is carried out processing and obtains barrier by the barrier positioning and data conversion module
Location information data are sent to data fusion module, and vision data is carried out processing and obtains barrier by the obstacle classification identification module
The type information data of object is hindered to be sent to data fusion module, the data fusion module is by the positioning of laser data and barrier
The type information data of information data and barrier carries out processing and show that the location information data of fused barrier is sent to
The location information data of barrier is carried out processing with obtaining new local cost by laser navigation frame, the laser navigation frame
Figure, the laser navigation frame carry out local paths planning and show that local paths planning data are sent to chassis drive module, institute
State chassis drive module by local paths planning data carry out processing obtain bobbin movement control data be sent to bobbin movement control
Molding block, the bobbin movement control module send motion control commands to motor driver.
3. the intelligent Mobile Service robot of tool laser and vision fusion obstacle avoidance system, feature exist as claimed in claim 2
In: the obstacle classification identification module utilizes pre-training Model Calculating Method by the deep learning algorithm of convolutional neural networks
The barrier of special scenes is trained to obtain trained neural network model, the trained neural network model can
Processing judgement, which is carried out, according to the color image information data captured in the vision data obtained by depth camera obtains the barrier
Hinder the type information data of object.
4. the intelligent Mobile Service robot of tool laser and vision fusion obstacle avoidance system, feature exist as claimed in claim 3
In: the barrier positions and data conversion module includes that the three-dimensional information detection of barrier and virtual two-dimensional laser intelligence turn
It changes, three-dimensional information detection converts the number that airborne PC can be identified for the depth image that depth camera acquires by image pick-up card
Image simultaneously carries out the depth information that image preprocessing obtains 3-D image, and the conversion of virtual two-dimensional laser intelligence will by calculation method
The depth information of 3-D image is converted to virtual two-dimensional laser intelligence data, and the virtual two-dimensional laser intelligence data is described
The location information data of barrier.
5. the intelligent Mobile Service robot of tool laser and vision fusion obstacle avoidance system, feature exist as claimed in claim 4
In: the data fusion module passes through the point cloud information data in calculation method fusion laser data, virtual two-dimensional laser intelligence
The type information data of data and barrier obtains the location information data of the barrier.
6. the intelligent Mobile Service robot of tool laser and vision fusion obstacle avoidance system, feature exist as claimed in claim 2
In: the barrier positions and data conversion module includes that the three-dimensional information detection of barrier and virtual two-dimensional laser intelligence turn
It changes, three-dimensional information detection converts the number that airborne PC can be identified for the depth image that depth camera acquires by image pick-up card
Image simultaneously carries out the depth information that image preprocessing obtains 3-D image, and the conversion of virtual two-dimensional laser intelligence will by calculation method
The depth information of 3-D image is converted to virtual two-dimensional laser intelligence data, and the virtual two-dimensional laser intelligence data is described
The location information data of barrier.
7. the intelligent Mobile Service robot of tool laser and vision fusion obstacle avoidance system, feature exist as claimed in claim 2
In: the data fusion module passes through the point cloud information data in calculation method fusion laser data, virtual two-dimensional laser intelligence
The type information data of data and barrier obtains the location information data of the barrier.
8. the intelligent Mobile Service robot of tool laser and vision fusion obstacle avoidance system, feature exist as claimed in claim 3
In: the data fusion module passes through the point cloud information data in calculation method fusion laser data, virtual two-dimensional laser intelligence
The type information data of data and barrier obtains the location information data of the barrier.
9. the intelligent Mobile Service machine of tool laser and vision fusion obstacle avoidance system as described in claim 1-8 any one
People, it is characterised in that: the hardware system further includes ultrasonic sensor and mechanical obstacle avoidance sensor, the ultrasonic sensor
The bobbin movement control module on motion control board, the ultrasonic sensor and mechanical anticollision are connected with mechanical obstacle avoidance sensor
The data that sensor senses obtain are sent to bobbin movement control module, and the bobbin movement control module receives ultrasonic wave biography
Control command is sent to motor driver when the data that sensor and mechanical obstacle avoidance sensor sensing obtain are barrier sensing data
To stop.
10. the intelligent Mobile Service robot of tool laser and vision fusion obstacle avoidance system, feature exist as claimed in claim 5
In: the pre-training model method is yolo V3 pre-training Model Calculating Method；The depth information by 3-D image is converted
Calculation method for virtual two-dimensional laser intelligence data is depthimage_to_laserscan algorithm；The fusion laser number
The calculation method of the type information data of point cloud information data, virtual two-dimensional laser intelligence data and barrier in is karr
Graceful filtering algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910454798.XA CN110147106A (en) | 2019-05-29 | 2019-05-29 | Has the intelligent Mobile Service robot of laser and vision fusion obstacle avoidance system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910454798.XA CN110147106A (en) | 2019-05-29 | 2019-05-29 | Has the intelligent Mobile Service robot of laser and vision fusion obstacle avoidance system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110147106A true CN110147106A (en) | 2019-08-20 |
Family
ID=67593563
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910454798.XA Pending CN110147106A (en) | 2019-05-29 | 2019-05-29 | Has the intelligent Mobile Service robot of laser and vision fusion obstacle avoidance system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110147106A (en) |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110502019A (en) * | 2019-09-06 | 2019-11-26 | 北京云迹科技有限公司 | A kind of barrier-avoiding method and device of Indoor Robot |
| CN110562354A (en) * | 2019-10-21 | 2019-12-13 | 上海沃迪智能装备股份有限公司 | barrier heavy load AGV is kept away in space |
| CN110604597A (en) * | 2019-09-09 | 2019-12-24 | 李胜利 | Method for intelligently acquiring fetal cardiac cycle images based on ultrasonic four-cavity cardiac section |
| CN110733039A (en) * | 2019-10-10 | 2020-01-31 | 南京驭行科技有限公司 | Automatic robot driving method based on VFH + and vision auxiliary decision |
| CN110764511A (en) * | 2019-11-13 | 2020-02-07 | 苏州大成有方数据科技有限公司 | Mobile robot with multi-sensor fusion and control method thereof |
| CN110908374A (en) * | 2019-11-14 | 2020-03-24 | 华南农业大学 | Mountain orchard obstacle avoidance system and method based on ROS platform |
| CN111413970A (en) * | 2020-03-18 | 2020-07-14 | 天津大学 | Ultra-wideband and vision integrated indoor robot positioning and autonomous navigation method |
| CN111426326A (en) * | 2020-01-17 | 2020-07-17 | 深圳市镭神智能系统有限公司 | Navigation method, device, equipment, system and storage medium |
| CN112034861A (en) * | 2020-09-15 | 2020-12-04 | 航天科工智能机器人有限责任公司 | Bionic autonomous robot autonomous obstacle avoidance system and obstacle avoidance method thereof |
| CN112083730A (en) * | 2020-09-28 | 2020-12-15 | 双擎科技(杭州)有限公司 | Method for avoiding obstacles in complex environment by fusing multiple groups of sensor data |
| CN112113565A (en) * | 2020-09-22 | 2020-12-22 | 温州科技职业学院 | Robot positioning system for agricultural greenhouse environment |
| CN112132929A (en) * | 2020-09-01 | 2020-12-25 | 北京布科思科技有限公司 | Grid map marking method based on depth vision and single line laser radar |
| CN112157662A (en) * | 2020-09-04 | 2021-01-01 | 江汉大学 | Blasting robot |
| CN112155487A (en) * | 2019-08-21 | 2021-01-01 | 追创科技(苏州)有限公司 | Sweeping robot, control method of sweeping robot and storage medium |
| CN112486171A (en) * | 2020-11-30 | 2021-03-12 | 中科院软件研究所南京软件技术研究院 | Robot obstacle avoidance method based on vision |
| CN112558045A (en) * | 2020-12-07 | 2021-03-26 | 福建（泉州）哈工大工程技术研究院 | Offline acceptance method for multi-line laser radar function of automatic driving equipment |
| CN112570906A (en) * | 2020-12-10 | 2021-03-30 | 苏州阿甘机器人有限公司 | Efficient intelligent cutting robot and working method thereof |
| CN112606211A (en) * | 2020-12-17 | 2021-04-06 | 湖南机电职业技术学院 | Stirring and transporting robot |
| CN112622049A (en) * | 2020-12-17 | 2021-04-09 | 湖南机电职业技术学院 | Stirring and transporting robot with automatic obstacle avoidance function |
| CN112859880A (en) * | 2021-02-25 | 2021-05-28 | 湖南擎谱数字科技有限公司 | Indoor automatic positioning and intelligent obstacle avoidance system and method for mobile robot |
| CN112859873A (en) * | 2021-01-25 | 2021-05-28 | 山东亚历山大智能科技有限公司 | Semantic laser-based mobile robot multi-stage obstacle avoidance system and method |
| CN113353173A (en) * | 2021-06-01 | 2021-09-07 | 福勤智能科技（昆山）有限公司 | Automatic guided vehicle |
| CN113409631A (en) * | 2021-06-18 | 2021-09-17 | 上海锡鼎智能科技有限公司 | AI auxiliary teaching robot |
| CN113641103A (en) * | 2021-08-13 | 2021-11-12 | 广东工业大学 | Adaptive robot treadmill control method and system |
| CN113697001A (en) * | 2021-08-31 | 2021-11-26 | 武汉铁路职业技术学院 | Sharing carrier |
| TWI757999B (en) * | 2020-12-04 | 2022-03-11 | 國立陽明交通大學 | Real-time obstacle avoidance system, real-time obstacle avoidance method and unmanned vehicle with real-time obstacle avoidance function |
| CN114475861A (en) * | 2022-01-26 | 2022-05-13 | 上海合时智能科技有限公司 | Robot and control method thereof |
| CN117539268A (en) * | 2024-01-09 | 2024-02-09 | 吉林省吉邦自动化科技有限公司 | VGA autonomous obstacle avoidance system based on fusion of machine vision and laser radar |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6285778B1 (en) * | 1991-09-19 | 2001-09-04 | Yazaki Corporation | Vehicle surroundings monitor with obstacle avoidance lighting |
| JP2005242552A (en) * | 2004-02-25 | 2005-09-08 | Denso Corp | On-vehicle reception device, on-vehicle transmission device and server |
| CN106097348A (en) * | 2016-06-13 | 2016-11-09 | 大连理工大学 | A kind of three-dimensional laser point cloud and the fusion method of two dimensional image |
| CN106405555A (en) * | 2016-09-23 | 2017-02-15 | 百度在线网络技术（北京）有限公司 | Obstacle detecting method and device used for vehicle-mounted radar system |
| CN106772435A (en) * | 2016-12-12 | 2017-05-31 | 浙江华飞智能科技有限公司 | A kind of unmanned plane barrier-avoiding method and device |
| CN108958250A (en) * | 2018-07-13 | 2018-12-07 | 华南理工大学 | Multisensor mobile platform and navigation and barrier-avoiding method based on known map |
| CN109002800A (en) * | 2018-07-20 | 2018-12-14 | 苏州索亚机器人技术有限公司 | The real-time identification mechanism of objective and recognition methods based on Multi-sensor Fusion |
| CN109160452A (en) * | 2018-10-23 | 2019-01-08 | 西安中科光电精密工程有限公司 | Unmanned transhipment fork truck and air navigation aid based on laser positioning and stereoscopic vision |
| CN109506658A (en) * | 2018-12-26 | 2019-03-22 | 广州市申迪计算机系统有限公司 | Robot autonomous localization method and system |
| CN109804223A (en) * | 2016-10-11 | 2019-05-24 | 御眼视觉技术有限公司 | Based on the barrier navigation vehicle detected |
-
2019
- 2019-05-29 CN CN201910454798.XA patent/CN110147106A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6285778B1 (en) * | 1991-09-19 | 2001-09-04 | Yazaki Corporation | Vehicle surroundings monitor with obstacle avoidance lighting |
| JP2005242552A (en) * | 2004-02-25 | 2005-09-08 | Denso Corp | On-vehicle reception device, on-vehicle transmission device and server |
| CN106097348A (en) * | 2016-06-13 | 2016-11-09 | 大连理工大学 | A kind of three-dimensional laser point cloud and the fusion method of two dimensional image |
| CN106405555A (en) * | 2016-09-23 | 2017-02-15 | 百度在线网络技术（北京）有限公司 | Obstacle detecting method and device used for vehicle-mounted radar system |
| CN109804223A (en) * | 2016-10-11 | 2019-05-24 | 御眼视觉技术有限公司 | Based on the barrier navigation vehicle detected |
| CN106772435A (en) * | 2016-12-12 | 2017-05-31 | 浙江华飞智能科技有限公司 | A kind of unmanned plane barrier-avoiding method and device |
| CN108958250A (en) * | 2018-07-13 | 2018-12-07 | 华南理工大学 | Multisensor mobile platform and navigation and barrier-avoiding method based on known map |
| CN109002800A (en) * | 2018-07-20 | 2018-12-14 | 苏州索亚机器人技术有限公司 | The real-time identification mechanism of objective and recognition methods based on Multi-sensor Fusion |
| CN109160452A (en) * | 2018-10-23 | 2019-01-08 | 西安中科光电精密工程有限公司 | Unmanned transhipment fork truck and air navigation aid based on laser positioning and stereoscopic vision |
| CN109506658A (en) * | 2018-12-26 | 2019-03-22 | 广州市申迪计算机系统有限公司 | Robot autonomous localization method and system |
Cited By (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112155487A (en) * | 2019-08-21 | 2021-01-01 | 追创科技(苏州)有限公司 | Sweeping robot, control method of sweeping robot and storage medium |
| CN110502019A (en) * | 2019-09-06 | 2019-11-26 | 北京云迹科技有限公司 | A kind of barrier-avoiding method and device of Indoor Robot |
| CN110604597B (en) * | 2019-09-09 | 2020-10-27 | 李胜利 | Method for intelligently acquiring fetal cardiac cycle images based on ultrasonic four-cavity cardiac section |
| CN110604597A (en) * | 2019-09-09 | 2019-12-24 | 李胜利 | Method for intelligently acquiring fetal cardiac cycle images based on ultrasonic four-cavity cardiac section |
| CN110733039A (en) * | 2019-10-10 | 2020-01-31 | 南京驭行科技有限公司 | Automatic robot driving method based on VFH + and vision auxiliary decision |
| CN110562354A (en) * | 2019-10-21 | 2019-12-13 | 上海沃迪智能装备股份有限公司 | barrier heavy load AGV is kept away in space |
| CN110764511A (en) * | 2019-11-13 | 2020-02-07 | 苏州大成有方数据科技有限公司 | Mobile robot with multi-sensor fusion and control method thereof |
| CN110908374B (en) * | 2019-11-14 | 2021-04-23 | 华南农业大学 | Mountain orchard obstacle avoidance system and method based on ROS platform |
| CN110908374A (en) * | 2019-11-14 | 2020-03-24 | 华南农业大学 | Mountain orchard obstacle avoidance system and method based on ROS platform |
| CN111426326A (en) * | 2020-01-17 | 2020-07-17 | 深圳市镭神智能系统有限公司 | Navigation method, device, equipment, system and storage medium |
| CN111413970A (en) * | 2020-03-18 | 2020-07-14 | 天津大学 | Ultra-wideband and vision integrated indoor robot positioning and autonomous navigation method |
| CN112132929A (en) * | 2020-09-01 | 2020-12-25 | 北京布科思科技有限公司 | Grid map marking method based on depth vision and single line laser radar |
| CN112132929B (en) * | 2020-09-01 | 2024-01-26 | 北京布科思科技有限公司 | Grid map marking method based on depth vision and single-line laser radar |
| CN112157662A (en) * | 2020-09-04 | 2021-01-01 | 江汉大学 | Blasting robot |
| CN112157662B (en) * | 2020-09-04 | 2023-07-07 | 江汉大学 | Blasting robot |
| CN112034861A (en) * | 2020-09-15 | 2020-12-04 | 航天科工智能机器人有限责任公司 | Bionic autonomous robot autonomous obstacle avoidance system and obstacle avoidance method thereof |
| CN112113565A (en) * | 2020-09-22 | 2020-12-22 | 温州科技职业学院 | Robot positioning system for agricultural greenhouse environment |
| CN112083730A (en) * | 2020-09-28 | 2020-12-15 | 双擎科技(杭州)有限公司 | Method for avoiding obstacles in complex environment by fusing multiple groups of sensor data |
| CN112486171A (en) * | 2020-11-30 | 2021-03-12 | 中科院软件研究所南京软件技术研究院 | Robot obstacle avoidance method based on vision |
| TWI757999B (en) * | 2020-12-04 | 2022-03-11 | 國立陽明交通大學 | Real-time obstacle avoidance system, real-time obstacle avoidance method and unmanned vehicle with real-time obstacle avoidance function |
| CN112558045A (en) * | 2020-12-07 | 2021-03-26 | 福建（泉州）哈工大工程技术研究院 | Offline acceptance method for multi-line laser radar function of automatic driving equipment |
| CN112558045B (en) * | 2020-12-07 | 2024-03-15 | 福建（泉州）哈工大工程技术研究院 | Offline acceptance method for multi-line laser radar function of automatic driving equipment |
| CN112570906A (en) * | 2020-12-10 | 2021-03-30 | 苏州阿甘机器人有限公司 | Efficient intelligent cutting robot and working method thereof |
| CN112606211A (en) * | 2020-12-17 | 2021-04-06 | 湖南机电职业技术学院 | Stirring and transporting robot |
| CN112622049A (en) * | 2020-12-17 | 2021-04-09 | 湖南机电职业技术学院 | Stirring and transporting robot with automatic obstacle avoidance function |
| CN112859873A (en) * | 2021-01-25 | 2021-05-28 | 山东亚历山大智能科技有限公司 | Semantic laser-based mobile robot multi-stage obstacle avoidance system and method |
| CN112859880A (en) * | 2021-02-25 | 2021-05-28 | 湖南擎谱数字科技有限公司 | Indoor automatic positioning and intelligent obstacle avoidance system and method for mobile robot |
| CN113353173A (en) * | 2021-06-01 | 2021-09-07 | 福勤智能科技（昆山）有限公司 | Automatic guided vehicle |
| CN113409631A (en) * | 2021-06-18 | 2021-09-17 | 上海锡鼎智能科技有限公司 | AI auxiliary teaching robot |
| CN113641103A (en) * | 2021-08-13 | 2021-11-12 | 广东工业大学 | Adaptive robot treadmill control method and system |
| CN113641103B (en) * | 2021-08-13 | 2023-04-25 | 广东工业大学 | Running machine control method and system of self-adaptive robot |
| CN113697001A (en) * | 2021-08-31 | 2021-11-26 | 武汉铁路职业技术学院 | Sharing carrier |
| CN114475861A (en) * | 2022-01-26 | 2022-05-13 | 上海合时智能科技有限公司 | Robot and control method thereof |
| CN117539268A (en) * | 2024-01-09 | 2024-02-09 | 吉林省吉邦自动化科技有限公司 | VGA autonomous obstacle avoidance system based on fusion of machine vision and laser radar |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110147106A (en) | Has the intelligent Mobile Service robot of laser and vision fusion obstacle avoidance system | |
| US20240035827A1 (en) | Methods for finding the perimeter of a place using observed coordinates | |
| US11673269B2 (en) | Method of identifying dynamic obstacle and robot implementing same | |
| US11407116B2 (en) | Robot and operation method therefor | |
| CN110275538A (en) | Intelligent cruise vehicle navigation methods and systems | |
| KR102298582B1 (en) | Artificial intelligence robot for determining cleaning route using sensor data and method for the same | |
| JP5963372B2 (en) | How to make a mobile robot follow people | |
| CN112867424B (en) | Navigation and cleaning area dividing method and system, and moving and cleaning robot | |
| CN109773783B (en) | Patrol intelligent robot based on space point cloud identification and police system thereof | |
| CN106020201A (en) | Mobile robot 3D navigation and positioning system and navigation and positioning method | |
| CN112415998A (en) | Obstacle classification and obstacle avoidance control system based on TOF camera | |
| CN109541535A (en) | A method of AGV indoor positioning and navigation based on UWB and vision SLAM | |
| CN106530946A (en) | Indoor map editing method and device | |
| GB2527207A (en) | Mobile human interface robot | |
| CN208953962U (en) | A kind of robot tracking control and robot | |
| CN106584472A (en) | Method and device for controlling autonomous mobile equipment | |
| CN110488811B (en) | Method for predicting pedestrian track by robot based on social network model | |
| CN112859873B (en) | Semantic laser-based mobile robot multi-stage obstacle avoidance system and method | |
| CN109917788A (en) | A kind of control method and device of Robot wall walking | |
| US11579626B2 (en) | Method of localization by synchronizing multi sensors and robot implementing same | |
| CN108459596A (en) | A kind of method in mobile electronic device and the mobile electronic device | |
| CN109760064A (en) | A kind of method of adjustment and device of mobile robot self-position | |
| CN111258311A (en) | Obstacle avoidance method of underground mobile robot based on intelligent vision | |
| CN111781929A (en) | AGV trolley and 3D laser radar positioning and navigation method | |
| US20220139086A1 (en) | Device and method for generating object image, recognizing object, and learning environment of mobile robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190820 |
|
| RJ01 | Rejection of invention patent application after publication |