CN109814564A - Detection, barrier-avoiding method, electronic equipment and the storage medium of target object - Google Patents
Detection, barrier-avoiding method, electronic equipment and the storage medium of target object Download PDFInfo
- Publication number
- CN109814564A CN109814564A CN201910088973.8A CN201910088973A CN109814564A CN 109814564 A CN109814564 A CN 109814564A CN 201910088973 A CN201910088973 A CN 201910088973A CN 109814564 A CN109814564 A CN 109814564A
- Authority
- CN
- China
- Prior art keywords
- floor
- depth image
- barrier
- electronic equipment
- target object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The present embodiments relate to a kind of detection of target object, barrier-avoiding method, electronic equipment and storage mediums, including obtain depth image in real time；Determine the floor in the depth image；The target object is detected based on the floor, every group of depth image carries out real-time ground detection, guarantee that detection is using actual ground as reference frame every time, floor in depth image is determined using processor powerful calculating ability aided algorithm parallel processing, computational efficiency is promoted, guarantees the real-time of robot detected target object during traveling.
Description
Technical field
The present embodiments relate to robot field more particularly to a kind of detection of target object, barrier-avoiding method, electronics to set
Standby and storage medium.
Background technique
It needs to solve the problem of to position avoidance and decision in the scene of human-computer interaction robot.It is current general to keep away
Barrier sensor mainly has laser, and depth is infrared, ultrasonic wave etc..Wherein single line laser sensor detection latitude is low, can not cope with
Complex environment, and the higher cost of multi-thread radar；The precision of infrared ultrasonic wave is very low, is more suitable for the optical sensors such as glass not
The obstacle being readily detected.
Summary of the invention
In consideration of it, to solve above-mentioned technical problem or partial technical problems, the embodiment of the invention provides a kind of targets pair
Detection, barrier-avoiding method, electronic equipment and the storage medium of elephant.
In a first aspect, the embodiment of the present invention provides a kind of detection method of target object based on real-time ground detection, packet
It includes:
Depth image is obtained in real time；
Determine the floor in the depth image；
The target object is detected based on the floor.
In a possible embodiment, floor in the determination depth image, comprising:
The depth image that will acquire generates corresponding point cloud chart picture according to internal reference matrix；
The point cloud chart picture is converted according to calibrated outer ginseng matrix；
The floor in the point cloud chart picture after determining conversion.
In a possible embodiment, the floor in the point cloud chart picture after the determining conversion, comprising:
Based on the algorithm of setting from the point cloud chart picture selection and the highest layer data of the ground degree of approach after conversion
Determine floor；
Wherein, the algorithm includes at least following one:
RANSAC algorithm, discrete space enumeration or least-squares algorithm.
In a possible embodiment, the floor in the point cloud chart picture after the determining conversion, comprising:
The algorithm parallel processing is assisted to determine the floor using processor；
Wherein, the processor includes at least following one:
Graphics processor CPU, central processor CPU, field programmable gate array FGGA or digital signal processor DSP.
In a possible embodiment, the target object includes: barrier；
It is described that the target object is detected based on the floor, comprising:
Using the floor as the plane of reference, the mesh for being highly greater than first threshold in the floor part will exceed
Object is marked, barrier is determined as；
And/or
It will be less than the target object that depth in the floor part is greater than second threshold, be determined as barrier.
In a possible embodiment, the calibrated outer ginseng matrix obtains in the following manner:
Under the Training scene of setting, by image acquisition device multiple groups depth image, and electronic equipment is determined
Corresponding mileage information；
Described image acquisition device and the electronic equipment center are determined based on the depth image and the mileage information
The calibrated outer ginseng matrix of point.
In a possible embodiment, described that described image is determined based on the depth image and the mileage information
The calibrated outer ginseng matrix of acquisition device and the electronic equipment central point, comprising:
The motion profile of described image acquisition device is determined according to two groups of adjacent depth images；
Calibrated outer ginseng matrix is determined according to the motion profile and the mileage information.
Second aspect, the embodiment of the present invention provide a kind of barrier-avoiding method, comprising:
Depth image is obtained in real time；
Determine the floor in the depth image；
Corresponding obstacle information in the depth image is determined based on the floor；
Determine the information of the barrier corresponding coordinate information in navigation map；
The mark of barrier in depth image is projected in the navigation map according to the coordinate information.
In a possible embodiment, the method, further includes:
According to the mark of the barrier in the navigation map, determine the current location of electronic equipment to target location most
Shortest path；
The first control instruction to the electronic equipment is generated according to the optimal path；
First control instruction is sent to the driving motor of the electronic equipment, so that the driving motor is according to institute
It states the first control instruction and drives the electronic equipment to the target location according to the optimal path.
In a possible embodiment, the method, further includes:
Judge whether the floor meets setting condition；
When determining that the floor does not meet setting condition, the second control instruction is generated；
Second control instruction is sent to the driving motor of the electronic equipment, so that the driving motor is according to institute
The second control instruction is stated to control the electronic equipment deceleration or stop.
The third aspect, the embodiment of the present invention provide a kind of electronic equipment, comprising: memory, processor and are stored in storage
On device and the computer program that can run on a processor, wherein the processor realizes such as above-mentioned the when executing described program
On the one hand any barrier-avoiding method of detection method or above-mentioned second aspect of any target object.
Fourth aspect, the embodiment of the present invention provide a kind of storage medium, and the storage medium is stored with one or more journeys
Sequence, one or more of programs can be executed by one or more processors, to realize as described in above-mentioned first aspect is any
The detection method of target object or any barrier-avoiding method of above-mentioned second aspect.
The detection method of target object provided in this embodiment based on real-time ground detection, by obtaining depth map in real time
Picture；Determine the floor in the depth image；The target object is detected based on the floor, every group deep
Degree image carries out real-time ground detection, guarantees that detection is powerful using processor using actual ground as reference frame every time
Computing capability aided algorithm parallel processing determines the floor in depth image, promotes computational efficiency, guarantees that robot is expert at
Into the real-time of detected target object in the process.
Detailed description of the invention
The step of Fig. 1 is a kind of detection method of target object based on real-time ground detection provided in an embodiment of the present invention
Flow chart；
Fig. 2 be the present embodiments relate to the determination depth image in floor step flow chart；
Fig. 3 be the present embodiments relate to image capture device and robot between it is calibrated it is outer ginseng matrix step
Rapid flow chart；
Fig. 4 is a kind of flow diagram of barrier-avoiding method provided in an embodiment of the present invention；
Fig. 5 be the present embodiments relate to navigation map schematic diagram；
Fig. 6 is a kind of structure of the detection device of target object based on real-time ground detection provided in an embodiment of the present invention
Schematic diagram；
Fig. 7 is a kind of structural schematic diagram of obstacle avoidance apparatus provided in an embodiment of the present invention；
Fig. 8 is a kind of electronic equipment structural schematic diagram provided in an embodiment of the present invention；
Fig. 9 is another electronic devices structure schematic diagram provided in an embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
In order to facilitate understanding of embodiments of the present invention, it is further explained below in conjunction with attached drawing with specific embodiment
Bright, embodiment does not constitute the restriction to the embodiment of the present invention.
The step of Fig. 1 is a kind of detection method of target object based on real-time ground detection provided in an embodiment of the present invention
Flow chart, as shown in Figure 1, this method comprises:
S101, depth image is obtained in real time.
The detection method for the target object based on real-time ground detection that the present embodiment is related to, is advancing applied to robot
Road conditions detection in the process, robot obtain the depth map of the front road conditions of robot in traveling by image capture device in real time
Picture.
Wherein, image capture device may be, but not limited to: fisheye camera.
S102, floor in the depth image is determined.
The depth image got in real time is detected using processor, determines the floor in depth image, is joined
According to Fig. 2, following sub-step is specifically included:
S1021, the depth image that will acquire generate corresponding point cloud chart picture according to internal reference matrix.
Using processor to depth image carry out it is down-sampled, will be down-sampled according to the internal reference matrix of image capture device after
Depth image is converted into 3D point cloud figure.
S1022, the point cloud chart picture is converted according to calibrated outer ginseng matrix.
Calibrated outer ginseng matrix positional relationship between image capture device and robot, using calibrated outer ginseng square
Battle array converts 3D point cloud figure.
Specifically, calibrated outer ginseng matrix is used to switch to the 3D point cloud figure under image capture device coordinate system for correspondence
Robot coordinate under 3D point cloud figure.
After conversion 3D point cloud figure may there are certain angles with actual ground, therefore, in the present embodiment to obtaining every time
The depth image got is detected, it is ensured that the detection of each target object is reference with real-time ground plane, executes S1023
Determine the floor in every depth image.
The floor in the point cloud chart picture after S1023, determining conversion.
Specifically, it is chosen and highest one layer of the ground degree of approach based on the algorithm of setting from the point cloud chart picture after conversion
Floor is determined in data, and assists the algorithm parallel processing to determine the floor using processor.
Wherein, the algorithm includes at least following one: RANSAC algorithm, discrete space enumeration or least square
Algorithm.
The processor includes at least following one:
Graphics processor (Graphics Processing Unit, GPU), central processing unit (Central
Processing Unit, CPU), field programmable gate array (Field-Programmable Gate Array, FPGA), or
Digital signal processor (Digital Signal Processing, DSP).
Further, it when using RANSAC algorithm, is chosen based on RANSAC algorithm from the point cloud chart picture after conversion
With floor determining in the highest layer data of the ground degree of approach (that is, the floor under determining robot coordinate system is to z=
0), every layer data of the 3D point cloud figure after conversion is analyzed, using in the highest layer data of the ground degree of approach definitely
Facial plane, RANSAC algorithm determine the principle of floor for using determining optimal solution is repeatedly calculated, therefore, it is necessary to carry out repeatedly
Serial computing is in the present embodiment the real-time and accuracy that promote detection, is determined using processor boostrap algorithm parallel processing
The floor, by the way of parallel processing, the mode compared to serial process shortens processing time, guarantee in real time
Property.GPU can be used in the above-mentioned processor referred to, determines floor, the powerful calculating energy of GPU using GPU auxiliary RANSAC algorithm
Power can determine the multiple numerical procedure of floor by parallel processing RANSAC algorithm simultaneously, promoted and determined using RANSAC algorithm
The efficiency of floor.
Further, highest from the degree of approach using discrete space enumeration when using discrete space enumeration
Floor is determined in one layer data, the variation of the running angle of robot and ground level is subjected to discretization, according to GPU
The result of discretization is handled, and then determines floor.
Further, when using least-squares algorithm, using least-squares algorithm from the highest layer data of the degree of approach
Middle determining floor, which is fitted, determines floor, is assisted being fitted determining floor according to high-speed CPU.
It is put down it should be noted that the ground that one or more of combination progress the present embodiment is related to can be used in above-mentioned processor
The correlation step that face determines can be according to reality such as only with GPU as executing subject, or using the combination etc. of GPU and FPGA
Demand is set, in this regard, the present embodiment is not especially limited.
Such as, the depth image that image capture device is got for the first time be denoted as A1, subsequent acquisition to depth image successively remember
The An ... that makees A2, A3 ...；GPU auxiliary RANSAC algorithm is used to determine the floor in A1 for the floor in a1, A2 respectively
For the floor in a2, A3 be a3 ... the floor in An is an ....
S103, the target object is detected based on the floor.
In the present embodiment, target object may be, but not limited to: barrier.
When target object is barrier, the detection of barrier is specifically included based on floor:
Using the floor as the plane of reference, the mesh for being highly greater than first threshold in the floor part will exceed
Object is marked, barrier is determined as；And/or will be less than the target object that depth in the floor part is greater than second threshold,
It is determined as barrier.
Such as, first threshold and first threshold may be the same or different, and for example first threshold and first threshold are equal are as follows:
5cm, using floor determining in real time as the plane of reference, the object that will be above floor 5cm is determined as ground barrier (as
Table barrier), it will be less than object of the depth greater than 5cm in the floor part and be determined as lower ground barrier (such as pit).
The detection method of target object provided in this embodiment based on real-time ground detection, by obtaining depth map in real time
Picture；Determine the floor in the depth image；The target object is detected based on the floor, every group deep
Degree image carries out real-time ground detection, guarantees that detection is powerful using processor using actual ground as reference frame every time
Computing capability aided algorithm parallel processing determines the floor in depth image, promotes computational efficiency, guarantees that robot is expert at
Into the real-time of detected target object in the process.
Referring to Fig. 3, the calibrated outer ginseng matrix between image capture device and robot can as follows really
It is fixed, it specifically includes:
S301, under the Training scene of setting, pass through image acquisition device multiple groups depth image, and determine electronics
The corresponding mileage information of equipment.
The Training scene of setting may be, but not limited to: the vacant lot of about 10 square meter of area is distributed in the vacant lot
Mulitpath is travelled for robot, and barrier is respectively set in each path, and the interval for object of placing obstacles can be 2 meters, barrier
The type of object is hindered to can be the barrier that robot can be prevented to advance.
It should be noted that the type of the size in vacant lot of training, the interval of barrier and barrier can be according to reality
Border demand is set, in this regard, the present embodiment is not especially limited.
Image capture device and odometer are installed in robot, and robot is placed in the vacant lot, planning robot
Travel path, when robot advances according to the path of planning, image capture device acquires multiple groups depth image, and is shooting
The mileage information that recorder people advances while depth image.
S302, described image acquisition device and the electronic equipment are determined based on the depth image and the mileage information
The calibrated outer ginseng matrix of central point.
According to the multiple groups depth image got, determine the floor in every group of depth image (that is, first by depth image
It is converted into 3D point cloud figure, and determines the corresponding floor equation of the 3D point cloud figure), it is determined according to multiple groups depth image multiple
Floor determines the track of image capture device movement, and the mileage of the robot recorded according to the motion profile and odometer is believed
Breath can determine the (conversion that the track of image capture device movement passes through outer ginseng matrix in a practical situation of calibrated outer ginseng matrix
Result should in odometer record robot mileage information it is equal).
Specifically, determine that the position of the image collecting device in shooting picture interval becomes according to two adjacent groups depth image
Change, the motion profile of acquisition device is obtained by the location variation at each of shooting process interval of adding up.
Further, calibrated outer ginseng matrix are as follows:
Its, by a 4x4 matrix wherein have indicate rotate R matrix (3x3) and space three-dimensional translational movement t=[x,
Y, z] ' composition.
It further, is the accuracy for promoting calibrated outer ginseng matrix, the present embodiment calculates machine in every group of depth image
The pose of device people and the residual error of odometer, when residual values are less than given threshold, i.e., it is believed that image capture device and robot
Outer ginseng matrix be calibrated.
Residual error may is that
Fig. 4 is a kind of flow diagram of barrier-avoiding method provided in an embodiment of the present invention, as shown in figure 4, this method is specific
Include:
S401, depth image is obtained in real time.
S402, floor in the depth image is determined.
S403, corresponding obstacle information in the depth image is determined based on the floor.
The step of in S401-S403 and S101-S103 to detection of obstacles, is substantially identical, can refer to the phase in above-mentioned Fig. 1
Description is closed, for succinct description, therefore not to repeat here.
It should be noted that increasing the judgment step to floor, specifically in an optinal plan in the present embodiment
Include:
Judge whether the floor meets setting condition；When determining that the floor does not meet setting condition,
Generate the second control instruction；Second control instruction is sent to the driving motor of the electronic equipment, so that the driving
Motor, which controls the electronic equipment according to second control instruction, to be slowed down or stops.
Judge whether the floor meets setting condition and specifically include: the depth image (An) that judgement is currently got
In floor an and the floor an-1 in a upper depth image (An-1) similarity, or judgement is current obtains
The similarity of the floor a1 in the floor an and first depth image (A1) in depth image (An) arrived.
When determining similarity is less than given threshold (such as 80%), i.e. machine is inferred in the floor failure of current detection
There is serious jolt or there is huge inclination on ground or nearby higher barrier occurs and blocked view at this time in device people
Open country generates the second control instruction；Second control instruction is sent to the driving motor of the electronic equipment, so that the drive
Dynamic motor, which controls the electronic equipment according to second control instruction, to be slowed down or stops.
S404, the information of the barrier corresponding coordinate information in navigation map is determined.
The information of barrier may include: the distance of obstacle distance robot, type of barrier etc., according to barrier
The coordinate for the current robot that the GPS positioning module of distance and robot apart from robot determines, according to current robot
Coordinate and the distance of obstacle distance robot determine that barrier is located at corresponding coordinate information in navigation map.
Referring to Fig. 5, e.g., robot needs to advance to the position B2 from the position B1, need at this time by path are as follows: b1-b2-
B3 encounters barrier in the section robot b2.
S405, the mark of barrier in depth image is projected in the navigation map according to the coordinate information.
S406, according to the mark of the barrier in the navigation map, determine the current location of electronic equipment with arriving target
The optimal path of point.
When determining that the section b2 encounters barrier, the road for advancing to the position B2 from the position B1 in addition to current path is planned again
Outside diameter, such as path is revised as to eventually arrive at B2 by b2-c2-c3.
S407, the first control instruction to the electronic equipment is generated according to the optimal path.
S408, the driving motor that first control instruction is sent to the electronic equipment, so that the driving motor
The electronic equipment is driven according to the optimal path to the target location according to first control instruction.
Barrier-avoiding method provided in this embodiment based on real-time ground detection, by obtaining depth image in real time；Determine institute
State the floor in depth image；Corresponding obstacle information in the depth image is determined based on the floor；Really
The information of the fixed barrier corresponding coordinate information in navigation map；According to the coordinate information by obstacle in depth image
The mark of object projects in the navigation map, according to the mark of the barrier in the navigation map, determines electronic equipment
Optimal path of the current location to target location；It is generated according to the optimal path and the first control of the electronic equipment is referred to
It enables；First control instruction is sent to the driving motor of the electronic equipment, so that the driving motor is according to described
One control instruction drives the electronic equipment to the target location according to the optimal path, guarantees that detection is with reality every time
The ground on border is reference frame, determines the floor in depth image using GPU powerful calculating ability auxiliary RANSAC algorithm,
Computational efficiency is promoted, guarantees the real-time of robot detection of obstacles during traveling, when encountering barrier, automatically again
Planning path, it is ensured that robot is moved to target position in time.
Fig. 6 is a kind of structure of the detection device of target object based on real-time ground detection provided in an embodiment of the present invention
Schematic diagram, as shown in fig. 6, the device specifically includes:
Module 601 is obtained, for obtaining depth image in real time；
Determining module 602, for determining the floor in the depth image；
Detection module 603, for being detected based on the floor to the target object.
Optionally, the determining module 602 is generated specifically for the depth image that will acquire according to internal reference matrix
Corresponding point cloud chart picture；The point cloud chart picture is converted according to calibrated outer ginseng matrix；The point after determining conversion
The floor in cloud atlas picture.
Optionally, the determining module 602 is selected specifically for the algorithm based on setting from the point cloud chart picture after conversion
It takes and determines floor in the highest layer data of the ground degree of approach；Wherein, the algorithm includes at least following one:
RANSAC algorithm, discrete space enumeration or least-squares algorithm.
Optionally, the determining module 602, described in being determined using the algorithm parallel processing of processor auxiliary
Floor；Wherein, the processor includes at least following one: graphics processor CPU, central processor CPU, scene can compile
Journey gate array FGGA or digital signal processor DSP.
Optionally, the target object includes: barrier；
The detection module 603 is specifically used for will exceed the floor portion using the floor as the plane of reference
Height is greater than the target object of first threshold in point, is determined as barrier；And/or it will be less than deep in the floor part
Degree is greater than the target object of second threshold, is determined as barrier.
Optionally, described device further include: demarcating module 604, for passing through Image Acquisition under the Training scene of setting
Device acquires multiple groups depth image, and determines the corresponding mileage information of electronic equipment；Based on the depth image and it is described in
Journey information determines the calibrated outer ginseng matrix of described image acquisition device Yu the electronic equipment central point.
Optionally, the demarcating module 604, specifically for determining that described image acquires according to two groups of adjacent depth images
The motion profile of device；Calibrated outer ginseng matrix is determined according to the motion profile and the mileage information.
The detection device of target object provided in this embodiment based on real-time ground detection can be as shown in Figure 8
The target object as shown in figure 1 based on real-time ground detection can be performed in the detection device of target object based on real-time ground detection
Detection method all steps, and then realize Fig. 1 shown in based on real-time ground detection target object detection method skill
Art effect specifically please refers to Fig. 1 associated description, and for succinct description, therefore not to repeat here.
Fig. 7 is a kind of structural schematic diagram of obstacle avoidance apparatus provided in an embodiment of the present invention, as shown in fig. 7, the device is specific
Include:
Module 701 is obtained, for obtaining depth image in real time；
Determining module 702, for determining the floor in the depth image；
The determining module 702 is also used to determine corresponding barrier in the depth image based on the floor
Information；
The determining module 702 is also used to determine the information of the barrier corresponding coordinate information in navigation map；
Projection module 703, for the mark of barrier in depth image to be projected to described lead according to the coordinate information
It navigates in map.
Optionally, path planning module 704 determine electronics for the mark according to the barrier in the navigation map
Optimal path of the current location of equipment to target location；The first control to the electronic equipment is generated according to the optimal path
System instruction；First control instruction is sent to the driving motor of the electronic equipment, so that the driving motor is according to institute
It states the first control instruction and drives the electronic equipment to the target location according to the optimal path.
Optionally, judgment module 705, for judging whether the floor meets setting condition；When determining describedly
When facial plane does not meet setting condition, the second control instruction is generated；Second control instruction is sent to the electronic equipment
Driving motor slow down or stop so that the driving motor controls the electronic equipment according to second control instruction.
Obstacle avoidance apparatus provided in this embodiment can be obstacle avoidance apparatus as shown in Figure 7, can be performed such as avoidance side in Fig. 4
All steps of method, and then realize the technical effect of barrier-avoiding method shown in Fig. 4, specific associated description referring to figure 4., succinctly to retouch
It states, therefore not to repeat here.
Fig. 8 is a kind of electronic equipment structural schematic diagram provided in an embodiment of the present invention, as shown in figure 8, the electronic equipment has
Body includes:
Processor 810, memory 820 and transceiver 830.
Processor 810 can be graphics processor GPU, central processing unit (central processing unit, CPU),
Or the combination of CPU and hardware chip.Above-mentioned hardware chip can be specific integrated circuit (application-specific
Integrated circuit, ASIC), programmable logic device (programmable logic device, PLD) or its group
It closes.Above-mentioned PLD can be Complex Programmable Logic Devices (complex programmable logic device, CPLD), existing
Field programmable gate array (field-programmable gate array, FPGA), Universal Array Logic (generic array
Logic, GAL) or any combination thereof.
Memory 820 is for storing various applications, operating system and data.Memory 820 can pass the data of storage
It is defeated by processor 810.Memory 820 may include volatile memory, non-volatile dynamic random access memory
(nonvolatile random access memory, NVRAM), phase change random access memory (phase change RAM,
PRAM), magnetic-resistance random access memory (magetoresistive RAM, MRAM) etc., for example, at least a magnetic disk storage
Part, Electrical Erasable programmable read only memory (electrically erasable programmable read-only
Memory, EEPROM), flush memory device, such as anti-or flash memory (flash memory, NOR) or anti-and flash memory (flash
Memory, NAND), semiconductor devices, such as solid state hard disk (solid state disk, SSD) etc..Memory 820 can be with
The combination of memory including mentioned kind.
Transceiver 830, for sending and/or receiving data, transceiver 830 can be antenna etc..
The course of work of each device is as follows:
Processor 810, for obtaining depth image in real time；
The processor 810, is also used to determine the floor in the depth image；
The processor 810 is also used to detect the target object based on the floor.
Optionally, the processor 810 generates pair specifically for the depth image that will acquire according to internal reference matrix
The point cloud chart picture answered；The point cloud chart picture is converted according to calibrated outer ginseng matrix；Described cloud after determining conversion
The floor in image.
Optionally, the processor 810 is chosen specifically for the algorithm based on setting from the point cloud chart picture after conversion
With floor determining in the highest layer data of the ground degree of approach；Wherein, the algorithm includes at least following one: RANSAC
Algorithm, discrete space enumeration or least-squares algorithm.
Optionally, the processor 810, specifically for assisting the algorithm parallel processing to determine describedly using processor
Facial plane；Wherein, the processor includes at least following one: graphics processor CPU, central processor CPU, field-programmable
Gate array FGGA or digital signal processor DSP.
Optionally, the target object includes: barrier；
The processor 810 is specifically used for will exceed the floor part using the floor as the plane of reference
Middle height is greater than the target object of first threshold, is determined as barrier；And/or it will be less than depth in the floor part
Greater than the target object of second threshold, it is determined as barrier.
Optionally, the processor 810, specifically for passing through image acquisition device under the Training scene of setting
Multiple groups depth image, and determine the corresponding mileage information of electronic equipment；It is true based on the depth image and the mileage information
Determine the calibrated outer ginseng matrix of described image acquisition device Yu the electronic equipment central point.
Optionally, the processor 810, specifically for determining described image acquisition dress according to two groups of adjacent depth images
The motion profile set；Calibrated outer ginseng matrix is determined according to the motion profile and the mileage information.
Electronic equipment provided in this embodiment can be electronic equipment as shown in Figure 8, can be performed as shown in figure 1 based on real
When ground detection target object detection method all steps, and then realize Fig. 1 shown in based on real-time ground detection mesh
The technical effect for marking the detection method of object, specifically please refers to Fig. 1 associated description, and for succinct description, therefore not to repeat here.
Fig. 9 is another electronic devices structure schematic diagram provided in an embodiment of the present invention, as shown in figure 9, the positioning device
It specifically includes:
Processor 910, memory 920 and transceiver 930.
Processor 910 and processor 810 shown in Fig. 8, memory 920 and memory 820 shown in Fig. 8, transceiver 930
Similar with transceiver 830 shown in Fig. 8, for succinct description, therefore not to repeat here.
The course of work of each device is as follows:
Processor 910, for obtaining depth image in real time；
The processor 910, is also used to determine the floor in the depth image；
The processor 910 is also used to determine corresponding barrier letter in the depth image based on the floor
Breath；
The processor 910 is also used to determine the information of the barrier corresponding coordinate information in navigation map；
The processor 910 is also used to that the mark of barrier in depth image is projected to institute according to the coordinate information
It states in navigation map.
Optionally, the processor 910 is also used to determine electronics according to the mark of the barrier in the navigation map
Optimal path of the current location of equipment to target location；The first control to the electronic equipment is generated according to the optimal path
System instruction；
Transceiver 930, for first control instruction to be sent to the driving motor of the electronic equipment, so that described
Driving motor drives the electronic equipment to the target location according to the optimal path according to first control instruction.
Optionally, the processor 910, is also used to judge whether the floor meets setting condition；When determining
When stating floor and not meeting setting condition, the second control instruction is generated；
The transceiver 930 is also used to for second control instruction being sent to the driving motor of the electronic equipment, with
Make the driving motor control the electronic equipment according to second control instruction to slow down or stop.
Electronic equipment provided in this embodiment can be electronic equipment as shown in Figure 9, can be performed such as avoidance side in Fig. 4
All steps of method, and then realize the technical effect of barrier-avoiding method shown in Fig. 4, specific associated description referring to figure 4., succinctly to retouch
It states, therefore not to repeat here.
The embodiment of the invention also provides a kind of storage medium (computer readable storage mediums).Here storage medium is deposited
Contain one or more program.Wherein, storage medium may include volatile memory, such as random access memory；It deposits
Reservoir also may include nonvolatile memory, such as read-only memory, flash memory, hard disk or solid state hard disk；Memory
It can also include the combination of the memory of mentioned kind.
It is above-mentioned in base to realize when one or more program can be executed by one or more processor in storage medium
In the detection side for the target object based on real-time ground detection that the detection device side of the target object of real-time ground detection executes
Method.
The processor is used to execute the detection program of the target object based on real-time ground detection stored in memory,
To realize the following mesh based on real-time ground detection in the execution of the detection device side of the target object detected based on real-time ground
The step of marking the detection method of object:
Depth image is obtained in real time；
Determine the floor in the depth image；
The target object is detected based on the floor.
Optionally, the depth image that will acquire generates corresponding point cloud chart picture according to internal reference matrix；
The point cloud chart picture is converted according to calibrated outer ginseng matrix；
The floor in the point cloud chart picture after determining conversion.
Optionally, it is chosen and highest one layer of the ground degree of approach based on the algorithm of setting from the point cloud chart picture after conversion
Floor is determined in data；Wherein, the algorithm includes at least following one: RANSAC algorithm, discrete space enumeration,
Or least-squares algorithm.
Optionally, the algorithm parallel processing is assisted to determine the floor using processor；Wherein, the processor
Including at least following one:
Graphics processor CPU, central processor CPU, field programmable gate array FGGA or digital signal processor DSP.
Optionally, the target object includes: barrier；
Using the floor as the plane of reference, the mesh for being highly greater than first threshold in the floor part will exceed
Object is marked, barrier is determined as；
And/or
It will be less than the target object that depth in the floor part is greater than second threshold, be determined as barrier.
Optionally, under the Training scene of setting, by image acquisition device multiple groups depth image, and electricity is determined
The corresponding mileage information of sub- equipment；
Described image acquisition device and the electronic equipment center are determined based on the depth image and the mileage information
The calibrated outer ginseng matrix of point.
Optionally, the motion profile of described image acquisition device is determined according to two groups of adjacent depth images；
Calibrated outer ginseng matrix is determined according to the motion profile and the mileage information.
The embodiment of the invention also provides another storage medium (computer readable storage mediums).Here storage medium
It is stored with one or more program.Wherein, storage medium may include volatile memory, such as random access memory；
Memory also may include nonvolatile memory, such as read-only memory, flash memory, hard disk or solid state hard disk；Storage
Device can also include the combination of the memory of mentioned kind.
When one or more program can be executed by one or more processor in storage medium, to realize above-mentioned keeping away
Hinder the barrier-avoiding method that equipment side executes.
The processor is for executing the avoidance program stored in memory, with what is executed below realization in avoidance equipment side
The step of barrier-avoiding method:
Depth image is obtained in real time；
Determine the floor in the depth image；
Corresponding obstacle information in the depth image is determined based on the floor；
Determine the information of the barrier corresponding coordinate information in navigation map；
The mark of barrier in depth image is projected in the navigation map according to the coordinate information.
Optionally, according to the mark of the barrier in the navigation map, determine the current location of electronic equipment to target
The optimal path in place；
The first control instruction to the electronic equipment is generated according to the optimal path；
First control instruction is sent to the driving motor of the electronic equipment, so that the driving motor is according to institute
It states the first control instruction and drives the electronic equipment to the target location according to the optimal path.
Optionally, judge whether the floor meets setting condition；
When determining that the floor does not meet setting condition, the second control instruction is generated；
Second control instruction is sent to the driving motor of the electronic equipment, so that the driving motor is according to institute
The second control instruction is stated to control the electronic equipment deceleration or stop.
Professional should further appreciate that, described in conjunction with the examples disclosed in the embodiments of the present disclosure
Unit and algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, hard in order to clearly demonstrate
The interchangeability of part and software generally describes each exemplary composition and step according to function in the above description.
These functions are implemented in hardware or software actually, the specific application and design constraint depending on technical solution.
Professional technician can use different methods to achieve the described function each specific application, but this realization
It should not be considered as beyond the scope of the present invention.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can be executed with hardware, processor
The combination of software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only memory
(ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technical field
In any other form of storage medium well known to interior.
Above-described specific embodiment has carried out further the purpose of the present invention, technical scheme and beneficial effects
It is described in detail, it should be understood that being not intended to limit the present invention the foregoing is merely a specific embodiment of the invention
Protection scope, all within the spirits and principles of the present invention, any modification, equivalent substitution, improvement and etc. done should all include
Within protection scope of the present invention.
Claims (12)
1. a kind of detection method of the target object based on real-time ground detection characterized by comprising
Depth image is obtained in real time；
Determine the floor in the depth image；
The target object is detected based on the floor.
2. the method according to claim 1, wherein floor in the determination depth image, comprising:
The depth image that will acquire generates corresponding point cloud chart picture according to internal reference matrix；
The point cloud chart picture is converted according to calibrated outer ginseng matrix；
The floor in the point cloud chart picture after determining conversion.
3. according to the method described in claim 2, it is characterized in that, ground in the point cloud chart picture after the determining conversion
Plane, comprising:
It is chosen based on the algorithm of setting from the point cloud chart picture after conversion and is determined with the highest layer data of the ground degree of approach
Floor；
Wherein, the algorithm includes at least following one:
RANSAC algorithm, discrete space enumeration or least-squares algorithm.
4. according to the method described in claim 3, it is characterized in that, ground in the point cloud chart picture after the determining conversion
Plane, comprising:
The algorithm parallel processing is assisted to determine the floor using processor；
Wherein, the processor includes at least following one:
Graphics processor CPU, central processor CPU, field programmable gate array FGGA or digital signal processor DSP.
5. method according to claim 1 to 4, which is characterized in that the target object includes: barrier；
It is described that the target object is detected based on the floor, comprising:
Using the floor as the plane of reference, the target pair for being highly greater than first threshold in the floor part will exceed
As being determined as barrier；
And/or
It will be less than the target object that depth in the floor part is greater than second threshold, be determined as barrier.
6. according to the method described in claim 2, it is characterized in that, the calibrated outer ginseng matrix obtains in the following manner
:
Under the Training scene of setting, by image acquisition device multiple groups depth image, and determine that electronic equipment is corresponding
Mileage information；
Described image acquisition device and the electronic equipment central point are determined based on the depth image and the mileage information
Calibrated outer ginseng matrix.
7. according to the method described in claim 6, it is characterized in that, described true based on the depth image and the mileage information
Determine the calibrated outer ginseng matrix of described image acquisition device Yu the electronic equipment central point, comprising:
The motion profile of described image acquisition device is determined according to two groups of adjacent depth images；
Calibrated outer ginseng matrix is determined according to the motion profile and the mileage information.
8. a kind of barrier-avoiding method characterized by comprising
Depth image is obtained in real time；
Determine the floor in the depth image；
Corresponding obstacle information in the depth image is determined based on the floor；
Determine the information of the barrier corresponding coordinate information in navigation map；
The mark of barrier in depth image is projected in the navigation map according to the coordinate information.
9. according to the method described in claim 8, it is characterized in that, the method, further includes:
According to the mark of the barrier in the navigation map, determine the current location of electronic equipment to the optimal road of target location
Diameter；
The first control instruction to the electronic equipment is generated according to the optimal path；
First control instruction is sent to the driving motor of the electronic equipment, so that the driving motor is according to described
One control instruction drives the electronic equipment to the target location according to the optimal path.
10. according to the method described in claim 8, it is characterized in that, the method, further includes:
Judge whether the floor meets setting condition；
When determining that the floor does not meet setting condition, the second control instruction is generated；
Second control instruction is sent to the driving motor of the electronic equipment, so that the driving motor is according to described
Two control instructions control the electronic equipment and slow down or stop.
11. a kind of electronic equipment characterized by comprising memory, processor and storage are on a memory and can be in processor
The computer program of upper operation, wherein the processor is realized as described in claim 1 to 7 is any when executing described program
Any barrier-avoiding method of the detection method or claim 8 to 10 of target object.
12. a kind of storage medium, which is characterized in that the storage medium is stored with one or more programs, one or more
A program can be executed by one or more processors, to realize the detection side of the target object as described in claim 1 to 7 is any
Method or any barrier-avoiding method of claim 8 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910088973.8A CN109814564A (en) | 2019-01-29 | 2019-01-29 | Detection, barrier-avoiding method, electronic equipment and the storage medium of target object |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910088973.8A CN109814564A (en) | 2019-01-29 | 2019-01-29 | Detection, barrier-avoiding method, electronic equipment and the storage medium of target object |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109814564A true CN109814564A (en) | 2019-05-28 |
Family
ID=66605789
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910088973.8A Pending CN109814564A (en) | 2019-01-29 | 2019-01-29 | Detection, barrier-avoiding method, electronic equipment and the storage medium of target object |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109814564A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110377031A (en) * | 2019-06-28 | 2019-10-25 | 炬星科技（深圳）有限公司 | Motion model update method, device, electronic equipment and storage medium |
| CN110622085A (en) * | 2019-08-14 | 2019-12-27 | 珊口（深圳）智能科技有限公司 | Mobile robot and control method and control system thereof |
| CN111046776A (en) * | 2019-12-06 | 2020-04-21 | 杭州成汤科技有限公司 | Mobile robot traveling path obstacle detection method based on depth camera |
| CN111521184A (en) * | 2020-04-13 | 2020-08-11 | 轻客小觅机器人科技(成都)有限公司 | Map building method, device and system of sweeping robot |
| CN112034863A (en) * | 2020-09-21 | 2020-12-04 | 北京洛必德科技有限公司 | Ground property information detection method and device and computer equipment |
| CN112318496A (en) * | 2019-08-05 | 2021-02-05 | 乐歆机器人(东莞)有限公司 | Depth camera-based visual motion channel construction system and method |
| WO2021072720A1 (en) * | 2019-10-17 | 2021-04-22 | 深圳市大疆创新科技有限公司 | Method for obtaining position of radar, radar and movable platform |
| CN113252066A (en) * | 2020-02-13 | 2021-08-13 | 纳恩博（北京）科技有限公司 | Method and device for calibrating parameters of odometer equipment, storage medium and electronic device |
| WO2022011826A1 (en) * | 2020-07-15 | 2022-01-20 | 上海炬佑智能科技有限公司 | Tof camera and ground obstacle detection method therefor, and ground navigation device |
| CN114424240A (en) * | 2019-11-12 | 2022-04-29 | Oppo广东移动通信有限公司 | Target detection method and device, equipment and storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1617170A (en) * | 2003-09-19 | 2005-05-18 | 索尼株式会社 | Environment identification device and method, route design device and method and robot |
| CN104899869A (en) * | 2015-05-14 | 2015-09-09 | 浙江大学 | Plane and barrier detection method based on RGB-D camera and attitude sensor |
| CN105074600A (en) * | 2013-02-27 | 2015-11-18 | 夏普株式会社 | Surrounding environment recognition device, autonomous mobile system using same, and surrounding environment recognition method |
| CN106017320A (en) * | 2016-05-30 | 2016-10-12 | 燕山大学 | Bulk cargo stack volume measuring method based on image processing and system for realizing same |
| CN107016705A (en) * | 2016-01-05 | 2017-08-04 | 德州仪器公司 | Ground level estimation in computer vision system |
| CN107179768A (en) * | 2017-05-15 | 2017-09-19 | 上海木爷机器人技术有限公司 | A kind of obstacle recognition method and device |
| CN107862719A (en) * | 2017-11-10 | 2018-03-30 | 未来机器人（深圳）有限公司 | Scaling method, device, computer equipment and the storage medium of Camera extrinsic |
| US20180107218A1 (en) * | 2015-08-17 | 2018-04-19 | X Development Llc | Ground Plane Detection to Verify Depth Sensor Status for Robot Navigation |
| CN108872991A (en) * | 2018-05-04 | 2018-11-23 | 上海西井信息科技有限公司 | Target analyte detection and recognition methods, device, electronic equipment, storage medium |
| CN108932737A (en) * | 2018-06-15 | 2018-12-04 | 深圳地平线机器人科技有限公司 | In-vehicle camera pitch angle scaling method and device, electronic equipment and vehicle |
| CN109214348A (en) * | 2018-09-19 | 2019-01-15 | 北京极智嘉科技有限公司 | A kind of obstacle detection method, device, equipment and storage medium |
-
2019
- 2019-01-29 CN CN201910088973.8A patent/CN109814564A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1617170A (en) * | 2003-09-19 | 2005-05-18 | 索尼株式会社 | Environment identification device and method, route design device and method and robot |
| CN105074600A (en) * | 2013-02-27 | 2015-11-18 | 夏普株式会社 | Surrounding environment recognition device, autonomous mobile system using same, and surrounding environment recognition method |
| CN104899869A (en) * | 2015-05-14 | 2015-09-09 | 浙江大学 | Plane and barrier detection method based on RGB-D camera and attitude sensor |
| US20180107218A1 (en) * | 2015-08-17 | 2018-04-19 | X Development Llc | Ground Plane Detection to Verify Depth Sensor Status for Robot Navigation |
| CN107016705A (en) * | 2016-01-05 | 2017-08-04 | 德州仪器公司 | Ground level estimation in computer vision system |
| CN106017320A (en) * | 2016-05-30 | 2016-10-12 | 燕山大学 | Bulk cargo stack volume measuring method based on image processing and system for realizing same |
| CN107179768A (en) * | 2017-05-15 | 2017-09-19 | 上海木爷机器人技术有限公司 | A kind of obstacle recognition method and device |
| CN107862719A (en) * | 2017-11-10 | 2018-03-30 | 未来机器人（深圳）有限公司 | Scaling method, device, computer equipment and the storage medium of Camera extrinsic |
| CN108872991A (en) * | 2018-05-04 | 2018-11-23 | 上海西井信息科技有限公司 | Target analyte detection and recognition methods, device, electronic equipment, storage medium |
| CN108932737A (en) * | 2018-06-15 | 2018-12-04 | 深圳地平线机器人科技有限公司 | In-vehicle camera pitch angle scaling method and device, electronic equipment and vehicle |
| CN109214348A (en) * | 2018-09-19 | 2019-01-15 | 北京极智嘉科技有限公司 | A kind of obstacle detection method, device, equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 王雪峰: "基于视觉-里程计融合的移动机器人定位算法研究", 《中国硕士学位论文全文数据库 信息科技辑》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110377031B (en) * | 2019-06-28 | 2022-06-10 | 炬星科技（深圳）有限公司 | Motion model updating method and device, electronic equipment and storage medium |
| CN110377031A (en) * | 2019-06-28 | 2019-10-25 | 炬星科技（深圳）有限公司 | Motion model update method, device, electronic equipment and storage medium |
| CN112318496A (en) * | 2019-08-05 | 2021-02-05 | 乐歆机器人(东莞)有限公司 | Depth camera-based visual motion channel construction system and method |
| CN110622085A (en) * | 2019-08-14 | 2019-12-27 | 珊口（深圳）智能科技有限公司 | Mobile robot and control method and control system thereof |
| WO2021026831A1 (en) * | 2019-08-14 | 2021-02-18 | 珊口（深圳）智能科技有限公司 | Mobile robot, and control method and control system therefor |
| US11042760B2 (en) | 2019-08-14 | 2021-06-22 | Ankobot (Shanghai) Smart Technologies Co., Ltd. | Mobile robot, control method and control system thereof |
| WO2021072720A1 (en) * | 2019-10-17 | 2021-04-22 | 深圳市大疆创新科技有限公司 | Method for obtaining position of radar, radar and movable platform |
| CN114424240A (en) * | 2019-11-12 | 2022-04-29 | Oppo广东移动通信有限公司 | Target detection method and device, equipment and storage medium |
| CN111046776A (en) * | 2019-12-06 | 2020-04-21 | 杭州成汤科技有限公司 | Mobile robot traveling path obstacle detection method based on depth camera |
| CN111046776B (en) * | 2019-12-06 | 2023-06-09 | 杭州成汤科技有限公司 | Method for detecting obstacle of path of mobile robot based on depth camera |
| CN113252066A (en) * | 2020-02-13 | 2021-08-13 | 纳恩博（北京）科技有限公司 | Method and device for calibrating parameters of odometer equipment, storage medium and electronic device |
| CN113252066B (en) * | 2020-02-13 | 2024-04-09 | 纳恩博（北京）科技有限公司 | Calibration method and device for parameters of odometer equipment, storage medium and electronic device |
| CN111521184A (en) * | 2020-04-13 | 2020-08-11 | 轻客小觅机器人科技(成都)有限公司 | Map building method, device and system of sweeping robot |
| WO2022011826A1 (en) * | 2020-07-15 | 2022-01-20 | 上海炬佑智能科技有限公司 | Tof camera and ground obstacle detection method therefor, and ground navigation device |
| CN112034863A (en) * | 2020-09-21 | 2020-12-04 | 北京洛必德科技有限公司 | Ground property information detection method and device and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109814564A (en) | Detection, barrier-avoiding method, electronic equipment and the storage medium of target object | |
| CN109345596B (en) | Multi-sensor calibration method, device, computer equipment, medium and vehicle | |
| US10496093B2 (en) | Movement control system, movement control device, and computer-implemented program for movement control | |
| US11852729B2 (en) | Ground intensity LIDAR localizer | |
| EP3635500B1 (en) | Method of navigating a vehicle and system thereof | |
| CN109492507B (en) | Traffic light state identification method and device, computer equipment and readable medium | |
| US9495602B2 (en) | Image and map-based detection of vehicles at intersections | |
| CN111238494A (en) | Carrier, carrier positioning system and carrier positioning method | |
| CN110377025A (en) | Sensor aggregation framework for automatic driving vehicle | |
| KR20180093934A (en) | Autonomous visual navigation | |
| JP6456405B2 (en) | Three-dimensional information calculation device, three-dimensional information calculation method, and autonomous mobile device | |
| CN109429518A (en) | Automatic Pilot traffic forecast based on map image | |
| CN112539749B (en) | Robot navigation method, robot, terminal device, and storage medium | |
| CN108271408A (en) | Generating three-dimensional maps of scenes using passive and active measurements | |
| CN108628324A (en) | Unmanned vehicle navigation method, device, equipment based on map vector and storage medium | |
| CN109737971B (en) | Vehicle-mounted auxiliary navigation positioning system, method, equipment and storage medium | |
| US11295521B2 (en) | Ground map generation | |
| US11567497B1 (en) | Systems and methods for perceiving a field around a device | |
| CN110119138A (en) | For the method for self-locating of automatic driving vehicle, system and machine readable media | |
| CN106598063A (en) | Unmanned air vehicle and flight control method | |
| CN110390240A (en) | Lane post-processing in automatic driving vehicle | |
| CN105890610B (en) | Vehicle tracking air navigation aid and device | |
| CN110091342A (en) | Vehicle condition detection method, device and detection robot | |
| CN109895100A (en) | A kind of generation method of navigation map, device and robot | |
| CN112348845B (en) | System and method for parking space detection and tracking |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190528 |