CN108280401A - A kind of pavement detection method, apparatus, cloud server and computer program product - Google Patents
A kind of pavement detection method, apparatus, cloud server and computer program product Download PDFInfo
- Publication number
- CN108280401A CN108280401A CN201711446547.4A CN201711446547A CN108280401A CN 108280401 A CN108280401 A CN 108280401A CN 201711446547 A CN201711446547 A CN 201711446547A CN 108280401 A CN108280401 A CN 108280401A
- Authority
- CN
- China
- Prior art keywords
- road surface
- value
- coordinate system
- depth map
- depth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H3/00—Appliances for aiding patients or disabled persons to walk about
- A61H3/06—Walking aids for blind persons
- A61H3/061—Walking aids for blind persons with electronic detecting or guiding means
Abstract
A kind of pavement detection method, apparatus, cloud server and computer program product are provided in the embodiment of the present application, for solving the problems, such as that pavement detection accuracy is not high in the prior art.This method includes：Receive the image that depth transducer obtains, wherein described image is the depth map under depth transducer coordinate system；Depth map under depth transducer coordinate system is converted into the depth map under world coordinate system；Wherein, the value of each pixel includes horizontal distance of each pixel to the depth transducer in the depth map under the world coordinate system；Determine the row mean value of the depth map under the world coordinate system；Determine in described image whether include road surface according to the row mean value.Using the scheme in the application, road surface can be accurately detected, and provide road gradient, while the false interference of the road surface in obstacle detection can effectively be removed according to pavement detection result so that the result of obstacle detection is more reliable.
Description
Technical field
This application involves airmanship, more particularly to a kind of pavement detection method, apparatus, cloud server and computer journey
Sequence product.
Background technology
Because of visual perception obstacle, the daily life and trip of visually impaired people is very inconvenient.They are in addition in parent
With the help of people, generally relies on traditional blind man's stick and seeing-eye dog is assisted.
Traditional blind man's stick has three big limitations, first, detection range is limited, usually within the scope of one meter of front；Second is that
Investigative range is narrow, usually only more slightly wider than blind person shoulder breadth；Third, detection limited height, it is below to be typically only capable to one meter of ground of detection
Obstacle；Above-mentioned limitation usually can all cause greatly to threaten to the safety of blind person.And traditional seeing-eye dog, then because of cost
It is too high to cause popularity rate too low, and in certain specific occasions, such as airport, station, subway public place, the public is to seeing-eye dog
Acceptance level is not also high so that the application of seeing-eye dog is also very limited.
To solve the above-mentioned problems, the method for proposing to carry out road surface and obstacle detection using image recognition in the prior art.
In these methods, generally rely on the color of priori, the consensus information of shape and strong marginal information be split and
It compares and carries out road surface and obstacle detection, such as white line, body shape, tables and chairs shape, the car shaped etc. on dark color road surface and roadside,
It has been recognised by the inventors that these road surfaces in the prior art and obstacle detection mode only adapt to some simple application scenarios, multiple
Under heterocycle border, pavement detection accuracy is not high.
Invention content
A kind of pavement detection method, apparatus, cloud server and computer program product are provided in the embodiment of the present application,
For solving the problems, such as that pavement detection accuracy is not high in the prior art.
According to the first aspect of the embodiment of the present application, a kind of pavement detection method is provided, including：Receive depth sensing
The image that device obtains, wherein described image is the depth map under depth transducer coordinate system；It will be under depth transducer coordinate system
Depth map is converted to the depth map under world coordinate system；Wherein, each pixel in the depth map under the world coordinate system
Value includes horizontal distance of each pixel to the depth transducer；Determine that the row of the depth map under the world coordinate system is equal
Value；Determine in described image whether include road surface according to the row mean value.
According to the second aspect of the embodiment of the present application, a kind of road surface checking device is provided, including：Receiving module is used
In the image for receiving depth transducer acquisition, wherein described image is the depth map under depth transducer coordinate system；At depth map
Module is managed, for the depth map under depth transducer coordinate system to be converted to the depth map under world coordinate system；Wherein, the generation
The value of each pixel includes horizontal distance of each pixel to the depth transducer in depth map under boundary's coordinate system；Row is equal
It is worth determining module, the row mean value for determining the depth map under the world coordinate system；Pavement detection module, for according to
Whether it includes road surface that row mean value determines in described image.
In terms of according to the third of the embodiment of the present application, a kind of cloud server is additionally provided, including：Display, storage
Device, one or more processors；And one or more modules, the one or more module are stored in memory, and by
It is configured to be executed by one or more processors, which includes for executing according to the application the first aspect
In pavement detection method in each step instruction.
In terms of according to the third of the embodiment of the present application, a kind of computer program product is additionally provided, the computer program
For product to being encoded for executing a kind of instruction of process, which includes according to the road surface inspection in the application the first aspect
Survey method.
Using the pavement detection method in the embodiment of the present application, the depth under sensor coordinate system is obtained using depth transducer
Then degree figure, the depth map being reconverted under world coordinate system pass through the row mean value of the depth map under calculating world coordinate system again
Whether to judge in image including road surface, compared with pavement detection method in the prior art, accuracy higher.
Description of the drawings
Attached drawing described herein is used for providing further understanding of the present application, constitutes part of this application, this Shen
Illustrative embodiments and their description please do not constitute the improper restriction to the application for explaining the application.In the accompanying drawings：
The pavement detection method flow diagram according to the embodiment of the present application one is shown in Fig. 1；
The schematic diagram for the world coordinate system established in the embodiment of the present application one is shown in Fig. 2；
It is shown in Fig. 3 in the embodiment of the present application one between sensor coordinate system, image coordinate system and pixel coordinate system
Mapping relations schematic diagram；
Fig. 4 shows the schematic diagram of a depth map in the embodiment of the present application one；
Fig. 5 shows the schematic diagram of a row mean value figure in the embodiment of the present application one；
Fig. 6 shows the schematic diagram of pretreated row mean value figure in the embodiment of the present application one；
Fig. 7 shows the structural schematic diagram of the road surface checking device according to the embodiment of the present application two；
Fig. 8 shows the structural schematic diagram of the cloud server according to the embodiment of the present application three.
Specific implementation mode
During realizing the application, inventor has found, in the prior art to carry out road surface and barrier using image recognition
In the method for hindering detection, generally relies on the color of priori, the consensus information of shape and strong marginal information and be split
Road surface and obstacle detection are carried out with comparing, such as dark road surface and the white line in roadside, body shape, tables and chairs shape, car shaped
Deng, it has been recognised by the inventors that these ground in the prior art and obstacle detection mode only adapt to some simple application scenarios,
Under complex environment, because two dimensional image is lost the depth information of three-dimensional environment, the gradient and obstacle on ground can not be provided
Range information, pavement detection accuracy is high.
Inventor has found that in the related art, there is also following pavement detection methods：One, using seed point region growing
The method that method comes detection level road surface and barrier, but to think that this method calculates complicated by inventor, and can only detection level
Face can not detect the slope surface up and down there are the gradient；Two, road surface slope is obtained using random point least square method planar fit method
Degree, this method calculate complicated, are influenced by random point selection, and ground line gradient is easy erroneous judgement, and detection range is relatively narrow (is only capable of detecting
Front width is the slope surface of 1~1.5 meter of range)；Three, road surface is extracted using stochastical sampling coherence method, this method calculates
Complexity, it is computationally intensive, it cannot be satisfied real-time demand.
In view of the above-mentioned problems, provided in the embodiment of the present application a kind of method, apparatus of pavement detection, cloud server and
Computer program product obtains the depth map under sensor coordinate system using depth transducer, is reconverted under world coordinate system
Depth map, then judge in image whether to include road surface by calculating the row mean value of the depth map under world coordinate system again,
Compared with pavement detection method in the prior art, accuracy higher.
According to the scheme in the embodiment of the present application, a kind of road surface based on depth map and disorder detection method, energy are provided
It is enough to detect road surface ahead, road gradient, obstacle positions and distance in real time, realize road gradient prompting, obstruction forewarning function,
Testing result can assist visually impaired people to go on a journey, and also being available for robot obstacle-avoiding and unmanned avoidance etc. has avoidance and access detection
The scene of demand uses.
Using the pavement detection method in the embodiment of the present application, depth map is obtained based on depth transducer, and be based on depth
The internal reference and attitude angle of sensor are converted to the depth map under camera coordinates system (that is, depth transducer coordinate system) with optical center
For the depth map under the world coordinate system of origin；Then the row mean value in depth map is calculated, it is equal according to line number-row average generation row
Value is schemed, and detects road surface in mean value of being expert at figure, and calculate road gradient；Preceding object is finally detected in depth map, and
According to the road surface removal false interference detected.Compared with traditional blind-guide device and other householder methods, according to having for depth map
Range is imitated, detection range of the invention is reserved for road conditions prompting, obstruction forewarning, access prompt up to 5 to 8 meters of front range
In the sufficient reaction time, greatly improve the safety of visually impaired people.
Using the pavement detection method in the embodiment of the present application, compared with traditional blind-guide device and other householder methods, root
According to the depth camera angle of view, the present invention can detect the region of front full filed angular region, generally in 60 ° to 120 ° ranges
Between.Wider investigative range while providing more safe spaces for visually impaired people, also provides more for access
More flexible selection so that visually impaired people's trip avoidance is more flexible.
Using the pavement detection method in the embodiment of the present application, compared with traditional blind-guide device and other householder methods, this
Because calculating, simple, calculation amount is small for invention, and the real-time meter of 30fps or more can be reached on common embedded device or mobile terminal
Frame per second is calculated, therefore can not only detect the fixed obstacle (such as flower stand, trees, electric pole) in front, moreover it is possible to detect that low speed is transported
Dynamic obstacle (such as pedestrian, trolley) also has certain adaptability to the obstacle (such as automobile) of high-speed motion.
Using the pavement detection method in the embodiment of the present application, compared with traditional blind-guide device and other householder methods, this
Invention has high robustness because detecting road surface and road gradient by the way of row mean value figure, to depth map noise,
It can accurately detect road surface, and provide road gradient, while obstacle inspection can effectively be removed according to pavement detection result
Road surface false interference in survey so that the result of obstacle detection is more reliable.
In order to make technical solution in the embodiment of the present application and advantage be more clearly understood, below in conjunction with attached drawing to the application
Exemplary embodiment be described in more detail, it is clear that described embodiment be only the application a part implement
Example, rather than the exhaustion of all embodiments.It should be noted that in the absence of conflict, the embodiment in the application and reality
The feature applied in example can be combined with each other.
Embodiment one
The pavement detection method flow diagram according to the embodiment of the present application one is shown in Fig. 1.As shown in Figure 1, including following
Step：
S101 receives the image that depth transducer obtains, wherein image is the depth map under depth transducer coordinate system.
In the specific implementation, the depth transducer in the application (also known as depth camera) usually may include following three
Class：Three-dimension sensor based on structure light, such as Kinect, RealSense, LeapMotion, Orbbec；Or it is based on binocular
The three-dimension sensor of stereoscopic vision, such as ZED, Inuitive, Human+ department pupil；Or the depth transducer based on TOF principles,
Such as PMD, Panasonic.
Next, the abundant three-dimensional environment information that can be provided in above-mentioned all kinds of depth transducers, that is, the base of depth map
On plinth, a variety of detections such as pavement detection, slope detection, obstacle detection are realized.
Depth map under depth transducer coordinate system is converted to the depth map under world coordinate system by S102；Wherein, described
The value of each pixel includes horizontal distance of each pixel to the depth transducer in depth map under world coordinate system.
In the specific implementation, can be using depth transducer optical center as world coordinate system origin, it is X to the right to choose horizontalwAxis
Positive direction is vertically downward YwAxis positive direction, perpendicular to XwYwPlane is simultaneously directed straight ahead for ZwAxis positive direction is established such as Fig. 2 institutes
The world coordinate system shown.Because world coordinate system is overlapped with depth transducer coordinate origin, therefore only deposited between two coordinate systems
In rotation relationship, without translation relation, it is possible to will be under depth transducer coordinate system according to the attitude angle of depth transducer
Point P (Xc, Yc, Zc) it is transformed into the point P (X under world coordinate systemw, Yw, Zw), calculation formula is：
Wherein, XwFor each pixel in image world coordinate system X axis coordinate value；YwIt is alive for each pixel in image
The Y axis coordinate value of boundary's coordinate system；ZwFor each pixel in image world coordinate system Z axis coordinate value；α, beta, gamma are that depth passes
The attitude angle of sensor indicates the rotation angle of the X, Y, Z axis of the depth transducer around the X, Y, Z axis of world coordinate system respectively；Xc
For each pixel in image depth transducer coordinate system X axis coordinate value；YcIt is each pixel in image in depth transducer
The Y axis coordinate value of coordinate system；ZcFor each pixel in image depth transducer coordinate system Z axis coordinate value.
In the specific implementation, the attitude angle of depth transducer can be obtained by Inertial Measurement Unit IMU.
In the specific implementation, due to the three-dimensional point P (X under depth transducer coordinate systemc, Yc, Zc) under pixel coordinate system two
There are mapping relations as shown in Figure 3 between dimension point p (u, v)；Therefore, in image each pixel in depth transducer coordinate system
Coordinate value Xc、YcAnd ZcIt can be according to three-dimensional point P (X under depth transducer coordinate systemc, Yc, Zc) to two-dimensional points p under pixel coordinate system
Mapping relations between (u, v)：And the internal reference matrix M for the depth transducer being obtained ahead of time3×4, figure
Depth value D (u, the v) deformations of each pixel (u, v) obtain as in：
Wherein, M3×4It is the internal reference matrix of depth transducer；u0=M3×4(0,2), v0=M3×4(1,2) indicates depth sensing
Coordinate of the optical center of device under pixel coordinate system；fu=M3×4(0,0), fv=M3×4(1,1) indicates depth transducer respectively in picture
Equivalent focal length under plain coordinate system on u direction and the directions v；D (u, v) is the depth value at coordinate (u, v) in image.
It should be appreciated that in another specific implementation mode of S102, when image capture moment depth camera Z axis with
When the Z axis of world coordinate system is parallel, the conversion of reference axis can not also be carried out, directly according to each pixel in the image in depth
The Z axis coordinate value of sensor coordinate system, obtains depth map；The application is not restricted this.
So far, the depth map under world coordinate system is had been obtained for.
S103 determines the row mean value of the depth map under the world coordinate system.
If specifically, depth map ZwAs shown in Figure 4；It can be to the depth map Z under world coordinate systemwIt is pre-processed, so
Row mean value is calculated afterwards, and row mean value figure I is established with line number-row mean valuerowsMean, as shown in Figure 5.Specifically, the pretreatment can be with
Including processing such as smooth, filtering, denoisings.
S104 determines in the image whether include road surface according to the row mean value.
In the specific implementation, S104 may be used following sub-step and implement：
S1041 is worth to the row mean value column vector according to the row.
It specifically, can be first to row mean value figure IrowsMeanIt is pre-processed, can be specifically：Since road surface is usually in the world
Depth value Z in coordinate systemwCharacteristic with monotonic increase from the near to the distant, therefore can first remove row mean value figure IrowsMeanIn from
Under up non-monotonic incremental row mean value, acnode then is carried out to remaining row mean value and is filtered out, the operation of micro-cracks band connection obtains
To pretreated row mean value figure as shown in FIG. 6.
Then row mean value figure is reassembled as row mean value column vector VrowsMean。
S1042 determines doubtful road surface region according to the row mean value column vector.
Specifically, can by the depth map, row mean value column vector intermediate value be 0 row set to 0；And by each picture in the depth map
The difference of the depth value of vegetarian refreshments and the row mean value column vector analog value is more than or equal to the position of preset road surface fluctuating tolerance
Value set to 0；It is doubtful road surface region by the location determination not for 0 in the depth map.
Specifically, if VrowsMeanThe corresponding value of v rows is 0, then by depth map ZwIn v row zero setting；If | Zw(u, v)-
VrowsMean[v] | ＜ δ, wherein δ is the road surface fluctuating tolerance of setting；Then retain depth map ZwIn (u, v) coordinate position value,
Otherwise zero setting；Then it progressively scans from the bottom up, finally obtains doubtful road surface region.
S1043 determines the corresponding principal plane in the doubtful road surface region according to preset principal plane position threshold.
In the specific implementation, before this step, can also include：Morphological scale-space, filter are carried out to doubtful road surface region
The step of except small isolated island and connecting small fracture belt.
In the specific implementation, principal plane screening can be carried out to the doubtful road surface after Morphological scale-space, specifically chooses plan
It can slightly preset.For example, area maximum is chosen, and selection area bottom is apart from depth map ZwBottom is no more than εrows
Capable region.Specifically, can be arrangedWherein εrowsFor the principal plane position threshold of setting,For
Depth map ZwHeight.
S1044 is fitted each point on the principal plane, and normal direction and the world coordinate system Y-axis for obtaining the principal plane are positive
The angle of side.
It in the specific implementation, can be with least square method to all the points P (X on selected principal planew, Yw, Zw) be fitted and put down
Face AX+BY+CZ+D=0 obtains the normal direction of planeThen normal direction is calculatedWith world coordinate system Y-axis positive direction
Angle theta.
S1045 judges whether the doubtful road surface region includes road surface according to the size of the angle.
In the specific implementation, when the absolute value of the angle is greater than the set value, it can be determined that the principal plane is not road surface, should
Doubtful road surface region does not include road surface；The doubtful road surface may be other things on the non-road surface such as stair, metope at this time；When the folder
When angle absolute value is less than or equal to setting value, judge that the principal plane is road surface, which includes road surface.The setting value can
To be preset angle value, maximum road surface tilt threshold is indicated；For example, 30 degree, 25 degree etc..
Further, when angle is more than the first value, and is less than setting value, road surface can also be judged for upward trend；Work as folder
Angle is less than second value, and when more than negative setting value, judges road surface for downhill path；Wherein, second value is negative value.Specifically, should
First value and second value can opposite numbers each other；First value can be such as 0 degree, 3 degree, the small angles such as 5 degree.And work as angle
More than or equal to second value, when being less than or equal to the first value, it can be determined that road surface is level road.
For example, enabling θ0For the maximum road surface tilt threshold of setting, if | θ | ＞ θ0, then it is road surface to judge the principal plane not；If 0
＜ θ ＜ θ0, then judge that the road surface is the upward trend that the gradient is the angles θ；If-θ0＜ θ ＜ 0, then the road surface is the descending that the gradient is the angles θ
Road.
So far, pavement detection, road gradient detection are completed.
S105, obstacle detection.
Specifically, if image includes road surface, road surface can be removed from depth map；And according to removal road surface after
Depth map executes obstacle detection；If in image not including road surface, obstacle detection can be executed directly according to depth map.
In the specific implementation, S105 may be used following sub-step and implement：
S1051, with the minimax detecting distance D of settingmax, DminThreshold value is carried out to the image obtained in S101 to block,
Only retain Dmin≤D≤DmaxDepth value.
Specifically, minimax detecting distance can be preset by user according to the needs of barrier hint, when user not
When being configured, can also default setting be carried by system.It is 30 centimetres that minimum detection distance, which can generally be arranged,；Maximum detection
Distance could be provided as example, 3 meters, 4 meters, 5 meters, 8 meters etc.；The application is not restricted this.
S1052, in DrangeThe road surface detected in middle removal S104, avoids road surface at subsequent extracted barrier edge
Flase drop is obstacle.
It should be appreciated that when it does not include road surface to be detected in S104 in image, S1052 can not also be executed.
It, can also be to the depth map D after blocking before the execution of this steprangeSmothing filtering is carried out, reduces noise to rear
Influence when continuous extraction barrier edge.
S1053, barrier contours extract, chooses profile closing and profile inner area is more than the wheel of given threshold minArea
Exterior feature is candidate barrier.
It, can be with to the depth except above-mentioned removal road agitation, removal detecting distance range before the execution of this step
Figure carries out Morphological scale-space, further removes influence of the noises such as burrs on edges and isolated island to subsequent extracted barrier edge when.
S1054 obtains the essential information of barrier according to selected barrier profile：With minimum outsourcing rectangle by obstacle
Frame contour selects, and the position of barrier is indicated with this frame；Barrier is indicated with the average value of all non-zero pixels values in profile
Average distance；Barrier is obtained with a distance from depth transducer.
S1055, the coordinate (u, v) with the non-zero center of barrier outsourcing rectangle or the non-zero points nearest from zero center are
Barrier center, according to the internal reference matrix M of depth camera3×4Calculate horizontal deflection angle of the barrier in camera coordinates systemWith
Vertical deflection angleCalculation formula is：
It obtains barrier and is obtaining position of the barrier in depth transducer coordinate system.
So far, it is determined that the distance of barrier and position；Complete detection of obstacles.
It in the specific implementation, can also be further according to the profile of barrier, cognitive disorders object；For example, identifying barrier
It is bicycle, flower bed etc. to hinder object.
In the specific implementation, pavement detection, the result of obstacle detection can also prompt user, for example, can be with
Prompt the position on user road surface, the distance of obstacle and/or position etc..
It should be appreciated that in the specific implementation, the pavement detection method in the embodiment of the present application can be realized in end side,
It can realize in server beyond the clouds；It can also be realized with end side and cloud server, the application does not limit this
System.
Using the pavement detection method in the embodiment of the present application, the depth under sensor coordinate system is obtained using depth transducer
Then degree figure, the depth map being reconverted under world coordinate system pass through the row mean value of the depth map under calculating world coordinate system again
Whether to judge in image including road surface, compared with pavement detection method in the prior art, accuracy higher.
Based on same inventive concept, a kind of road surface checking device is additionally provided in the embodiment of the present application, due to the device solution
Certainly the principle of problem is similar to the method that the embodiment of the present application one is provided, therefore the implementation of the device may refer to the reality of method
It applies, overlaps will not be repeated.
Embodiment two
Fig. 7 shows the structural schematic diagram of the road surface checking device according to the embodiment of the present application two.
As shown in fig. 7, according to the road surface checking device 700 of the embodiment of the present application two, including：Receiving module 701, for connecing
Receive the image that depth transducer obtains, wherein described image is the depth map under depth transducer coordinate system；Coordinate transferring
702, for the depth map under depth transducer coordinate system to be converted to the depth map under world coordinate system；Wherein, the world
The value of each pixel includes horizontal distance of each pixel to the depth transducer in depth map under coordinate system；Row mean value
Determining module 703, the row mean value for determining the depth map under the world coordinate system；Pavement detection module 704 is used for basis
Whether it includes road surface that the row mean value determines in described image.
In the specific implementation, pavement detection module can specifically include：First computing unit, for calculating the depth
The row mean value of figure；Second computing unit, for being worth to the row mean value column vector according to the row；First determination unit, is used for
Doubtful road surface region is determined according to the row mean value column vector；Second determination unit, for according to preset principal plane position threshold
Determine the corresponding principal plane in the doubtful road surface region；Angle determination unit is obtained for being fitted to each point on the principal plane
The angle of the normal direction and world coordinate system Y-axis forward direction side of the principal plane；Judging unit, for judging to be somebody's turn to do according to the size of the angle
Whether doubtful road surface region includes road surface.
In the specific implementation, the first determination unit specifically can be used for：By in the depth map under the world coordinate system, row
The row that mean value column vector intermediate value is 0 is set to 0；And by the depth value of each pixel in the depth map under the world coordinate system with it is described
The difference of row mean value column vector analog value, the value more than or equal to the position of preset road surface fluctuating tolerance are set to 0；By the generation
The location determination in depth map under boundary's coordinate system not being 0 is doubtful road surface region.
In the specific implementation, judging unit specifically can be used for：When the absolute value of the angle is greater than the set value, judging should
Principal plane is not road surface, which does not include road surface；When the angle absolute value is less than or equal to setting value, judging should
Principal plane is road surface, which includes road surface.
In the specific implementation, judging unit can be also used for：When the angle is more than the first value, and is less than the setting value,
Judge the road surface for upward trend；When the angle is less than second value, and is more than the negative setting value, judge the road surface for descending
Road；Wherein, which is negative value.
In the specific implementation, can also include according to the road surface checking device of the embodiment of the present application：Obstacle detection module is used
In in the picture include road surface when, from world coordinate system under depth map in remove road surface；And according to the depth behind removal road surface
Figure executes obstacle detection；And when in the picture including road surface, then according to the depth map under world coordinate system, obstacle is executed
Detection.
Embodiment three
Based on same inventive concept, a kind of cloud server 800 as shown in Figure 8 is additionally provided in the embodiment of the present application.
The cloud server 800 includes：Display 801, memory 802, one or more processors 803；And one
Or multiple modules, which is stored in memory 802, and is configured to by one or more processors
803 execute, which includes for executing each step in the pavement detection method in the embodiment of the present application one
Instruction.
The side provided with the embodiment of the present application one by the principle that the method run in the cloud server solves the problems, such as
Method is similar, therefore the implementation of the cloud server may refer to the implementation of method, and overlaps will not be repeated.
Example IV
Based on same inventive concept, a kind of computer program product is additionally provided in the embodiment of the present application, the computer journey
For sequence product to being encoded for executing a kind of instruction of process, which includes the pavement detection side in the embodiment of the present application one
Method.
It is provided with the embodiment of the present application one by the principle that the method for computer program product operation solves the problems, such as
Method is similar, therefore the implementation of the computer program product may refer to the implementation of method, and overlaps will not be repeated.
It should be understood by those skilled in the art that, embodiments herein can be provided as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the application
Apply the form of example.Moreover, the application can be used in one or more wherein include computer usable program code computer
The computer program production implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
The form of product.
The application is with reference to method, the flow of equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram describe.It should be understood that can be realized by computer program instructions every first-class in flowchart and/or the block diagram
The combination of flow and/or box in journey and/or box and flowchart and/or the block diagram.These computer programs can be provided
Instruct the processor of all-purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine so that the instruction executed by computer or the processor of other programmable data processing devices is generated for real
The device for the function of being specified in present one flow of flow chart or one box of multiple flows and/or block diagram or multiple boxes.
These computer program instructions, which may also be stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that instruction generation stored in the computer readable memory includes referring to
Enable the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one box of block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device so that count
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, in computer or
The instruction executed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Although the preferred embodiment of the application has been described, created once a person skilled in the art knows basic
Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the application range.
Obviously, those skilled in the art can carry out the application essence of the various modification and variations without departing from the application
God and range.In this way, if these modifications and variations of the application belong to the range of the application claim and its equivalent technologies
Within, then the application is also intended to include these modifications and variations.
Claims (14)
1. a kind of pavement detection method, which is characterized in that including：
Receive the image that depth transducer obtains, wherein described image is the depth map under depth transducer coordinate system；
Depth map under depth transducer coordinate system is converted into the depth map under world coordinate system；Wherein, the world coordinates
The value of each pixel includes horizontal distance of each pixel to the depth transducer in depth map under system；
Determine the row mean value of the depth map under the world coordinate system；
Determine in described image whether include road surface according to the row mean value.
2. according to the method described in claim 1, it is characterized in that, according to the row mean value determine in described image whether include
Road surface specifically includes：
It is worth to the row mean value column vector according to the row；
Doubtful road surface region is determined according to the row mean value column vector；
The corresponding principal plane in the doubtful road surface region is determined according to preset principal plane position threshold；
Each point on the principal plane is fitted, the folder of the normal direction and world coordinate system Y-axis forward direction side of the principal plane is obtained
Angle；
Judge whether the doubtful road surface region includes road surface according to the size of the angle.
3. according to the method described in claim 2, it is characterized in that, determining doubtful road surface region according to the row mean value column vector
It specifically includes：
By in the depth map under the world coordinate system, row mean value column vector intermediate value be 0 row set to 0；And by the world coordinates
The difference of the depth value and the row mean value column vector analog value of each pixel in depth map under system, is more than or equal to preset
The value of the position of road surface fluctuating tolerance is set to 0；
It is doubtful road surface region by the location determination not for 0 in the depth map under the world coordinate system.
4. according to the method described in claim 2, it is characterized in that, judging the doubtful road surface area according to the size of the angle
Whether domain specifically includes including road surface：
When the absolute value of the angle is greater than the set value, it is road surface to judge the principal plane not, and the doubtful road surface region is not
Including road surface；
When the angle absolute value is less than or equal to setting value, judge that the principal plane is road surface, the doubtful road surface region packet
Include road surface.
5. according to the method described in claim 4, it is characterized in that, when the angle absolute value be less than or equal to setting value when,
Judge that the principal plane is road surface, after the doubtful road surface region includes road surface, further includes：
When the angle is more than the first value, and is less than the setting value, judge the road surface for upward trend；
When the angle is less than second value, and is more than the negative setting value, judge the road surface for downhill path；Wherein, institute
It is negative value to state second value.
6. according to the method described in claim 1, it is characterized in that, whether being wrapped in determining described image according to the row mean value
After including road surface, further include：
If described image includes road surface, the road surface is removed from the depth map under the world coordinate system；And according to
The depth map behind road surface is removed, obstacle detection is executed；
If not including that road surface executes obstacle detection according to the depth map under the world coordinate system in described image.
7. a kind of road surface checking device, which is characterized in that including：
Receiving module, the image for receiving depth transducer acquisition, wherein described image is under depth transducer coordinate system
Depth map；
Coordinate transferring, for the depth map under depth transducer coordinate system to be converted to the depth map under world coordinate system；
Wherein, the value of each pixel includes water of each pixel to the depth transducer in the depth map under the world coordinate system
Flat distance；
Row mean value determining module, the row mean value for determining the depth map under the world coordinate system；
Pavement detection module, for determining in described image whether include road surface according to the row mean value.
8. device according to claim 7, which is characterized in that pavement detection module specifically includes：
Computing unit, for being worth to the row mean value column vector according to the row；
First determination unit, for determining doubtful road surface region according to the row mean value column vector；
Second determination unit, for determining that the corresponding master in the doubtful road surface region is flat according to preset principal plane position threshold
Face；
Angle determination unit, for being fitted to each point on the principal plane, the normal direction and the world for obtaining the principal plane are sat
The angle of Y-axis forward direction side of mark system；
Judging unit, for judging whether the doubtful road surface region includes road surface according to the size of the angle.
9. device according to claim 8, which is characterized in that the first determination unit is specifically used for：
By in the depth map under the world coordinate system, row mean value column vector intermediate value be 0 row set to 0；And by the world coordinates
The difference of the depth value and the row mean value column vector analog value of each pixel in depth map under system, is more than or equal to preset
The value of the position of road surface fluctuating tolerance is set to 0；
It is doubtful road surface region by the location determination not for 0 in the depth map under the world coordinate system.
10. device according to claim 8, which is characterized in that judging unit is specifically used for：
When the absolute value of the angle is greater than the set value, it is road surface to judge the principal plane not, and the doubtful road surface region is not
Including road surface；
When the angle absolute value is less than or equal to setting value, judge that the principal plane is road surface, the doubtful road surface region packet
Include road surface.
11. device according to claim 10, which is characterized in that judging unit is additionally operable to：
When the angle is more than the first value, and is less than the setting value, judge the road surface for upward trend；
When the angle is less than second value, and is more than the negative setting value, judge the road surface for downhill path；Wherein, institute
It is negative value to state second value.
12. device according to claim 7, which is characterized in that further include：
Obstacle detection module, for when described image includes road surface, being removed from the depth map under the world coordinate system
The road surface；And according to the depth map behind removal road surface, execute obstacle detection；And
When not including road surface in described image, according to the depth map under the world coordinate system, obstacle detection is executed.
13. a kind of cloud server, which is characterized in that the cloud server includes：Display, memory are one or more
Processor；And one or more modules, one or more of modules are stored in the memory, and be configured to by
One or more of processors execute, and one or more of modules include for any described in perform claim requirement 1-6
The instruction of each step in method.
14. a kind of computer program product, the computer program product to being encoded for executing a kind of instruction of process,
The process includes the method according to any one of claim 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711446547.4A CN108280401B (en) | 2017-12-27 | 2017-12-27 | Pavement detection method and device, cloud server and computer program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711446547.4A CN108280401B (en) | 2017-12-27 | 2017-12-27 | Pavement detection method and device, cloud server and computer program product |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108280401A true CN108280401A (en) | 2018-07-13 |
| CN108280401B CN108280401B (en) | 2020-04-07 |
Family
ID=62802436
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711446547.4A Active CN108280401B (en) | 2017-12-27 | 2017-12-27 | Pavement detection method and device, cloud server and computer program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108280401B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109074668A (en) * | 2018-08-02 | 2018-12-21 | 深圳前海达闼云端智能科技有限公司 | Method for path navigation, relevant apparatus and computer readable storage medium |
| CN109188459A (en) * | 2018-08-29 | 2019-01-11 | 东南大学 | A kind of small obstacle recognition method in ramp based on multi-line laser radar |
| CN109495641A (en) * | 2018-10-24 | 2019-03-19 | 维沃移动通信有限公司 | A kind of based reminding method and mobile terminal |
| CN110216661A (en) * | 2019-04-29 | 2019-09-10 | 北京云迹科技有限公司 | Fall the method and device of region recognition |
| CN111316119A (en) * | 2018-12-28 | 2020-06-19 | 深圳市大疆创新科技有限公司 | Radar simulation method and device |
| CN111337948A (en) * | 2020-02-25 | 2020-06-26 | 达闼科技成都有限公司 | Obstacle detection method, radar data generation device, and storage medium |
| CN111428622A (en) * | 2020-03-20 | 2020-07-17 | 上海健麾信息技术股份有限公司 | Image positioning method based on segmentation algorithm and application thereof |
| CN112184792A (en) * | 2020-08-28 | 2021-01-05 | 辽宁石油化工大学 | Road slope calculation method and device based on vision |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102135617A (en) * | 2011-01-06 | 2011-07-27 | 哈尔滨工程大学 | Multi-target positioning method of bistatic multi-input multi-output radar |
| CN103630496A (en) * | 2013-12-12 | 2014-03-12 | 南京大学 | Traffic video visibility detecting method based on road surface brightness and least square approach |
| EP3109796A1 (en) * | 2015-06-25 | 2016-12-28 | Ricoh Company, Ltd. | Method and device for recognizing road scene as well as relevant program and non-transitory computer-readable medium |
| CN106681353A (en) * | 2016-11-29 | 2017-05-17 | 南京航空航天大学 | Unmanned aerial vehicle (UAV) obstacle avoidance method and system based on binocular vision and optical flow fusion |
| CN106937910A (en) * | 2017-03-20 | 2017-07-11 | 杭州视氪科技有限公司 | A kind of barrier and ramp detecting system and method |
| CN107220632A (en) * | 2017-06-12 | 2017-09-29 | 山东大学 | A kind of pavement image dividing method based on normal direction feature |
-
2017
- 2017-12-27 CN CN201711446547.4A patent/CN108280401B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102135617A (en) * | 2011-01-06 | 2011-07-27 | 哈尔滨工程大学 | Multi-target positioning method of bistatic multi-input multi-output radar |
| CN103630496A (en) * | 2013-12-12 | 2014-03-12 | 南京大学 | Traffic video visibility detecting method based on road surface brightness and least square approach |
| EP3109796A1 (en) * | 2015-06-25 | 2016-12-28 | Ricoh Company, Ltd. | Method and device for recognizing road scene as well as relevant program and non-transitory computer-readable medium |
| CN106681353A (en) * | 2016-11-29 | 2017-05-17 | 南京航空航天大学 | Unmanned aerial vehicle (UAV) obstacle avoidance method and system based on binocular vision and optical flow fusion |
| CN106937910A (en) * | 2017-03-20 | 2017-07-11 | 杭州视氪科技有限公司 | A kind of barrier and ramp detecting system and method |
| CN107220632A (en) * | 2017-06-12 | 2017-09-29 | 山东大学 | A kind of pavement image dividing method based on normal direction feature |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109074668A (en) * | 2018-08-02 | 2018-12-21 | 深圳前海达闼云端智能科技有限公司 | Method for path navigation, relevant apparatus and computer readable storage medium |
| WO2020024234A1 (en) * | 2018-08-02 | 2020-02-06 | 深圳前海达闼云端智能科技有限公司 | Route navigation method, related device, and computer readable storage medium |
| CN109188459A (en) * | 2018-08-29 | 2019-01-11 | 东南大学 | A kind of small obstacle recognition method in ramp based on multi-line laser radar |
| CN109188459B (en) * | 2018-08-29 | 2022-04-15 | 东南大学 | Ramp small obstacle identification method based on multi-line laser radar |
| CN109495641A (en) * | 2018-10-24 | 2019-03-19 | 维沃移动通信有限公司 | A kind of based reminding method and mobile terminal |
| CN111316119A (en) * | 2018-12-28 | 2020-06-19 | 深圳市大疆创新科技有限公司 | Radar simulation method and device |
| CN110216661A (en) * | 2019-04-29 | 2019-09-10 | 北京云迹科技有限公司 | Fall the method and device of region recognition |
| CN111337948A (en) * | 2020-02-25 | 2020-06-26 | 达闼科技成都有限公司 | Obstacle detection method, radar data generation device, and storage medium |
| CN111428622A (en) * | 2020-03-20 | 2020-07-17 | 上海健麾信息技术股份有限公司 | Image positioning method based on segmentation algorithm and application thereof |
| CN111428622B (en) * | 2020-03-20 | 2023-05-09 | 上海健麾信息技术股份有限公司 | Image positioning method based on segmentation algorithm and application thereof |
| CN112184792A (en) * | 2020-08-28 | 2021-01-05 | 辽宁石油化工大学 | Road slope calculation method and device based on vision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108280401B (en) | 2020-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108280401A (en) | A kind of pavement detection method, apparatus, cloud server and computer program product | |
| CN104916163B (en) | Parking space detection method | |
| US10956756B2 (en) | Hazard detection from a camera in a scene with moving shadows | |
| CN106681353B (en) | The unmanned plane barrier-avoiding method and system merged based on binocular vision with light stream | |
| CN105346706B (en) | Flight instruments, flight control system and method | |
| CN106156723B (en) | A kind of crossing fine positioning method of view-based access control model | |
| CN105261020B (en) | A kind of express lane line detecting method | |
| JP5926228B2 (en) | Depth detection method and system for autonomous vehicles | |
| CN103718213B (en) | Automatic scene is calibrated | |
| CN108230392A (en) | A kind of dysopia analyte detection false-alarm elimination method based on IMU | |
| CN109472831A (en) | Obstacle recognition range-measurement system and method towards road roller work progress | |
| CN107907048A (en) | A kind of binocular stereo vision method for three-dimensional measurement based on line-structured light scanning | |
| CN109308718B (en) | Space personnel positioning device and method based on multiple depth cameras | |
| CN103679120B (en) | The detection method of rough road and system | |
| CN105447853A (en) | Flight device, flight control system and flight control method | |
| CN110334678A (en) | A kind of pedestrian detection method of view-based access control model fusion | |
| CN109410264A (en) | A kind of front vehicles distance measurement method based on laser point cloud and image co-registration | |
| CN105286871A (en) | Video processing-based body height measurement method | |
| CN103852060A (en) | Visible light image distance measuring method based on monocular vision | |
| CN109410234A (en) | A kind of control method and control system based on binocular vision avoidance | |
| KR20130053980A (en) | Obstacle detection method using image data fusion and apparatus | |
| JP5310027B2 (en) | Lane recognition device and lane recognition method | |
| JP2018025906A (en) | Image processing device, imaging device, mobile equipment control system, image processing method, and program | |
| CN106709432B (en) | Human head detection counting method based on binocular stereo vision | |
| JP5501084B2 (en) | Planar area detection apparatus and stereo camera system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |