CN110132278A - A kind of instant method and device for positioning and building figure - Google Patents
A kind of instant method and device for positioning and building figure Download PDFInfo
- Publication number
- CN110132278A CN110132278A CN201910400102.5A CN201910400102A CN110132278A CN 110132278 A CN110132278 A CN 110132278A CN 201910400102 A CN201910400102 A CN 201910400102A CN 110132278 A CN110132278 A CN 110132278A
- Authority
- CN
- China
- Prior art keywords
- warehouse compartment
- angle point
- building
- map
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
Abstract
This application involves a kind of method and apparatus for positioning and building figure immediately.The instant positioning and the method for building figure include: acquisition overhead view image；Determine the image coordinate of the warehouse compartment angle point and the warehouse compartment angle point in the overhead view image, wherein each warehouse compartment angle point corresponds to an anchor point of warehouse compartment；The default warehouse compartment width of image coordinate and first based on the warehouse compartment angle point, determines effective warehouse compartment；Effective warehouse compartment is matched with the warehouse compartment in map；The pose for building figure equipment is determined based on matching result；The map is updated based on the pose for building figure equipment.
Description
Technical field
The present invention relates to computer vision field more particularly to vision positioning and build figure field.
Background technique
Immediately positioning is a kind of logical with figure (Simultaneous LocalizationAndMapping, abbreviation SLAM) is built
It crosses real-time tracking robot motion and in the process while establishing ambient enviroment map to reach the skill of the targets such as location navigation
Art.
Applying at present in the SLAM in automatic parking field mainly includes SLAM and view-based access control model based on laser radar
SLAM.Both schemes can realize positioning based on characteristic point and build figure function, however the location information that the two provides is limited, together
Shi Buneng it is manifestly intended that warehouse compartment global or local position.
Therefore it provides it is a kind of new apply automatic parking field positioning and build drawing method and device be very it is necessary to
's.
Summary of the invention
The application's is designed to provide a kind of positioning immediately and the method for building figure.This method can be during building figure
The semantic information (for example, the width information of warehouse compartment, warehouse compartment line position relationship, warehouse compartment corner location relationship) of warehouse compartment is directly utilized,
Thus the map established can not only provide the location information for building figure equipment, moreover it is possible to store the location information of warehouse compartment.
On the one hand the application provides a kind of positioning immediately and the method for building figure.The described method includes: obtaining overhead view image；Really
The image coordinate of warehouse compartment angle point and the warehouse compartment angle point in the fixed overhead view image, wherein each warehouse compartment angle point corresponds to warehouse compartment
An anchor point；The default warehouse compartment width of image coordinate and first based on the warehouse compartment angle point, determines effective warehouse compartment；It will be described
Effective warehouse compartment is matched with the warehouse compartment in map；The pose for building figure equipment is determined based on matching result；Based on the figure equipment of building
Pose updates the map.
In some embodiments, the acquisition overhead view image includes: to obtain an at least visual pattern, is become by inverse perspective
An at least visual pattern of changing commanders is converted to an at least sub- overhead view image, and an at least sub- overhead view image is spelled
It is connected into the overhead view image.
In some embodiments, the warehouse compartment angle point in the determination overhead view image and its image coordinate include: to be based on
Deep neural network determines that the warehouse compartment angle point in the overhead view image, the warehouse compartment angle point are warehouse compartment entrance angle point.
In some embodiments, the image coordinate and the first default warehouse compartment width based on the warehouse compartment angle point determines
Effective warehouse compartment, comprising: image coordinate and the first default warehouse compartment width based on the warehouse compartment angle point determine candidate warehouse compartment, and
Effective warehouse compartment is determined in the candidate warehouse compartment.
In some embodiments, described to determine effective warehouse compartment in the candidate warehouse compartment, comprising: to determine the candidate warehouse compartment
Area-of-interest；Based on the area-of-interest, determining institute of classifying is carried out to the candidate warehouse compartment by deep neural network
State effective warehouse compartment.
In some embodiments, described match effective warehouse compartment with the warehouse compartment in map includes: by effective library
The warehouse compartment angle point of position is matched with the warehouse compartment angle point of the warehouse compartment in the map；Determine at least two pairs warehouse compartment angles being mutually matched
Point；Based on the pose for building figure equipment described in the described at least two pairs warehouse compartment angle points being mutually matched determinations.
In some embodiments, the warehouse compartment angle of the warehouse compartment angle point by effective warehouse compartment and the warehouse compartment in the map
Point is matched, comprising: for each warehouse compartment angle point of effective warehouse compartment, determines the library in the warehouse compartment angle point and the map
The distance between the warehouse compartment angle point of position；Judge whether the distance meets preset condition, is the warehouse compartment angle of then effective warehouse compartment
The warehouse compartment angle point of point and the warehouse compartment in the map is mutually matched.The preset condition is the distance in preset threshold range
It is interior, and the distance is minimum in the preset threshold range.
In some embodiments, described to build figure equipment based on described in the described at least two pairs warehouse compartment angle points being mutually matched determinations
Pose, comprising: determine the confidence level of each pair of warehouse compartment angle point being mutually matched, the confidence level and the warehouse compartment being mutually matched
Angle point is built at a distance from figure equipment and/or the warehouse compartment angle point being mutually matched is observed in the map by history with described
Number is related；Based on the pose for building figure equipment described in confidence level determination.
In some embodiments, the method further includes: determine in effective warehouse compartment of the overhead view image with it is described
Effective warehouse compartment that warehouse compartment in map unmatches；It will be inserted into the map with the described effective warehouse compartment unmatched.
In some embodiments, the method further includes optimizing to the map.The optimization includes following
At least one of: at least partly warehouse compartment angle point in the map is fitted according to preset positional relationship；To described
Warehouse compartment angle point of the mutual alignment difference in preset threshold range carries out weight merging in map；It is straight to the warehouse compartment in the map
The direction vector of line optimizes, wherein a side of each warehouse compartment line correspondences warehouse compartment；Based on the second default warehouse compartment width
The warehouse compartment angle point of warehouse compartment in the map is optimized.
On the one hand the application provides a kind of device for positioning immediately and building figure, described device includes that at least one image obtains
Device port, at least one storage equipment, the storage equipment includes one group of instruction；And with it is described at least one storage equipment
At least one processor of communication.Wherein, when executing one group of instruction, at least one described processor for make it is described i.e.
Shi Dingwei executes positioning immediately and the method for building figure with the device for building figure.
Other feature will be set forth in part in the description in the application.By the elaboration, make the following drawings and
The content of embodiment narration becomes apparent for those of ordinary skills.Inventive point in the application can pass through
Practice is sufficiently illustrated using method described in detailed example discussed below, means and combinations thereof.
Detailed description of the invention
Exemplary embodiment disclosed in this application is described in detail in the following drawings.Wherein identical appended drawing reference is in attached drawing
Several views in indicate similar structure.Those of ordinary skill in the art will be understood that these embodiments be non-limiting,
Exemplary embodiment, the purpose that attached drawing is merely to illustrate and describes, it is no intended to it limits the scope of the present disclosure, other modes
Embodiment may also similarly complete the intention of the invention in the application.It should be appreciated that the drawings are not drawn to scale.Wherein:
Fig. 1 shows the system for positioning and building immediately figure according to shown in some embodiments of the present application；
Fig. 2 shows the flow charts for the method for positioning and building immediately figure according to shown in some embodiments of the present application；
Fig. 3 shows the schematic diagram in the parking lot according to shown in some embodiments of the present application.
Specific embodiment
Following description provides the specific application scene of the application and requirements, it is therefore an objective to those skilled in the art be enable to make
It makes and using the content in the application.To those skilled in the art, to the various partial modifications of the disclosed embodiments
Be it will be apparent that and without departing from the spirit and scope of the disclosure, the General Principle that will can be defined here
Applied to other embodiments and application.Therefore, the embodiment the present disclosure is not limited to shown in, but it is consistent most wide with claim
Range.
Term used herein is only used for the purpose of description specific example embodiments, rather than restrictive.For example, unless
Context is expressly stated otherwise, used herein above, singular " one ", "one" and "the" also may include plural form.
When used in this manual, the terms "include", "comprise" and/or " containing " are meant that associated integer, step, behaviour
Make, element and/or component exist, but be not excluded for other one or more features, integer, step, operation, element, component and/or
Other features, integer, step, operation, element, component can be added in the system/method.
In view of being described below, the operation of the related elements of these features of the disclosure and other features and structure and
The economy of combination and the manufacture of function and component may be significantly raising.With reference to attached drawing, all these formation disclosure
A part.It is to be expressly understood, however, that the purpose that attached drawing is merely to illustrate and describes, it is no intended to limit the disclosure
Range.
Process used in the disclosure shows the operation realized according to the system of some embodiments in the disclosure.It answers
This is expressly understood, and the operation of flow chart can be realized out of order.On the contrary, operation can be realized with reversal order or simultaneously.
Furthermore, it is possible to other one or more operations of flow chart addition.One or more operations can be removed from flow chart.
The one aspect of the application is related to a kind of positioning immediately and the method for building figure.Specifically, this method includes obtaining spy
The overhead view image for determining region (for example, parking lot, warehouse), the figure of warehouse compartment angle point and the warehouse compartment angle point is determined according to the overhead view image
As coordinate；Then in conjunction with preset warehouse compartment width, effective warehouse compartment in the overhead view image is determined；It will be in effective warehouse compartment and map
Existing warehouse compartment is matched, and the pose for building figure equipment is determined according to matching result, and update according to the pose for building figure equipment
The map.
Fig. 1 shows the system for positioning and building immediately figure according to shown in some embodiments of the present application.
Immediately positioning and the available visual pattern of system 100 and the execution positioning immediately for building figure and the method for building figure.Institute
The description of Fig. 2 can be referred to the method for building figure by stating positioning immediately.As shown, the system 100 for positioning and building figure simultaneously can be with
Including image acquisition equipment 101 and build figure equipment 102 (also known as position immediately in the application and build map device).
Image acquisition equipment 101 is used to obtain the visual pattern of ambient enviroment.Image acquisition equipment 101 can be camera,
Such as fisheye camera, reflected refraction camera, panoramic imagery camera.Image acquisition equipment 101, which may be mounted at, builds figure equipment 102
On.
Building figure equipment 102 is the exemplary computer device that can execute while position and building the method for figure.As an example, building
Figure equipment 102 can be vehicle.When image acquisition equipment 101 is installed on vehicle, image acquisition equipment 101 be may be mounted at
Front part of vehicle, rear portion, at least one position in left and right side.Correspondingly, the quantity of image acquisition equipment 101 can be
It is one or more.
In some embodiments, building figure equipment 102 may include communication port 150, in order to data communication.Build figure equipment
102 can also include processor 120, and processor 120 is used for computer instructions in the form of one or more processors.
Computer instruction may include the routine for for example executing specific function described herein, program, object, component, data structure, mistake
Journey, module and function.For example, processor 120 can determine effective warehouse compartment in overhead view image in conjunction with preset warehouse compartment width.
In another example processor 120 can determine the matching result between effective warehouse compartment in overhead view image and the warehouse compartment in map, and base
The pose for building figure equipment is determined in the matching result, and map is then updated based on the pose for building figure equipment again.
In some embodiments, processor 120 may include one or more hardware processors, such as microcontroller, micro-
Processor, Reduced Instruction Set Computer (RISC), specific integrated circuit (ASIC), specific to instruction-set processor of application
(ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), micro controller unit, number
Word signal processor (DSP), field programmable gate array (FPGA), Advance RISC Machine (ARM), programmable logic device
(PLD), it is able to carry out any circuit or the processor etc. of one or more functions, or any combination thereof.
In some embodiments, building figure equipment 102 may include internal communication bus 110, program storage and different form
Data store (for example, disk 170, read-only memory (ROM) 130 or random access memory (RAM) 140).Build figure equipment
102 can also include being stored in ROM 130, RAM 140 and/or the other kinds of non-transitory that will be executed by processor 120
Program instruction in storage medium.The present processes and/or process can be used as program instruction realization.Build figure equipment 102 also
Including I/O component 160, the input/output between computer and other assemblies (for example, user interface elements) is supported.Figure is built to set
Standby 102 can also receive programming and data by network communication.
Just to describe the problem, is built in figure equipment 102 in this application and only describe a processor.However, should
Note that building figure equipment 102 in the application can also include multiple processors, therefore, operation disclosed in this application and/or method
Step can be executed by a processor as described in the present disclosure, can also be combined by multiple processors and be executed.For example, if
The processor 120 that figure equipment 102 is built in the application executes step A and step B, then it should be understood that step A and step B can also be with
It is jointly or separately executed by two different processors in information processing (for example, first processor executes step A, second processing
Device executes step B or the first and second processors execute step A and B jointly).
Fig. 2 shows the flow charts for the method for positioning and building immediately figure according to shown in some embodiments of the present application.Stream
Journey 200 may be embodied as build in the non-transitory storage medium in figure equipment 102 one group of instruction.Building figure equipment 102 can hold
Row and can correspondingly execute the step in process 200 at one group of instruction.
The operation of shown process 200 presented below, it is intended to be illustrative and be not restrictive.In some embodiments
In, process 200 can add one or more operation bidirectionals not described when realizing, and/or delete one or more herein
Described operation.In addition, shown in Fig. 2 and operations described below sequence limits not to this.
In 210, the available overhead view image of figure equipment 102 is built.
Overhead view image can be directly acquired by building figure equipment 102, can also be with indirect gain overhead view image.The indirect gain is bowed
Visible image comprising the following three steps:
First step builds the available at least visual pattern of figure equipment 102.At least visual pattern can be with
It is that one or more image acquisition equipments 101 are obtained in synchronization, every visual pattern can correspond to same or different
The scenery scenery of different regional areas (for example, in parking lot).
As an example, building 102 available four visual patterns of figure equipment.Four visual patterns build figure by being mounted on
102 front of equipment, rear portion, left and right side four image acquisition equipments 101 synchronization obtain.
Second step, build figure equipment 102 can by inverse perspective mapping by an at least visual pattern be converted to
A few sub- overhead view image.In conjunction with the example in first step, this four can be regarded by inverse perspective mapping by building figure equipment 102
Feel that image is converted to four sub- overhead view images.Sub- overhead view image and visual pattern correspond.
Third step, the overhead view image can be spliced into for an at least sub- overhead view image by building figure equipment 102.Knot
The example in first step and second step is closed, figure equipment 102 is built and can use four image acquisition equipments 101 and set with figure is built
Positional relationship between standby 102, which is transformed under same image coordinate system, then again by same image
Sub- overhead view image under coordinate system is spliced into final overhead view image.It is to be understood that compared to sub- overhead view image, institute
The overhead view image for stating splicing has the bigger visual field.
In 220, the figure of warehouse compartment angle point and the warehouse compartment angle point that figure equipment 102 can determine in the overhead view image is built
As coordinate.Wherein, each warehouse compartment angle point corresponds to an anchor point of warehouse compartment.
Warehouse compartment in the application can be parking stall, be also possible to other regions for placing article.For example, warehouse compartment can be
The different zones divided in warehouse to place article.Therefore, herein disclosed technical solution can also be used for bulk storage plant intelligence
It can handling goods.It is merely for convenience of explanation, following application scenarios are by taking parking lot as an example, and following warehouse compartment is for parking stall.Stop
Parking stall includes oblique line parking stall, "-" type parking stall and non-font parking stall, below by taking non-font parking stall as an example.
The anchor point of warehouse compartment can be the vertex on warehouse compartment boundary line in the application.For example, with reference to Fig. 3, the boundary of warehouse compartment A
Line includes line segment 301, line segment 302, line segment 305 and line segment 308, and the anchor point of warehouse compartment includes point 311, point 312, point 315 and point
316.In the application, warehouse compartment boundary line is referred to as warehouse compartment line.For non-font parking stall, warehouse compartment line is straight line.
Warehouse compartment angle point may include warehouse compartment entrance angle point and non-warehouse compartment entrance angle point.The warehouse compartment entrance angle point refers in library
Anchor point on the boundary line of position inlet.The non-warehouse compartment entrance angle point refers to other warehouse compartments in addition to warehouse compartment entrance angle point
Angle point.For example, for warehouse compartment A as non-font parking stall, warehouse compartment entrance angle point is point 315 and point 316, non-warehouse compartment with reference to Fig. 3
Entrance angle point is point 311 and point 312.
In some embodiments, warehouse compartment angle in overhead view image can be determined based on deep neural network by building figure equipment 102
Point, and further determine that the image coordinate of each warehouse compartment angle point.The deep neural network can be based on marked warehouse compartment angle point
Overhead view image to being obtained after initial depth neural metwork training.Herein, the warehouse compartment angle point for building the determination of figure equipment 102, which can be, bows
All warehouse compartment angle points in visible image, the part libraries parallactic angle point being also possible in overhead view image.The part libraries parallactic angle point can be with
Only include warehouse compartment entrance angle point, can also only include non-warehouse compartment entrance angle point, can also include part warehouse compartment inlet angle point and portion
Divide non-warehouse compartment entrance angle point.For example, to the overhead view image for only including warehouse compartment A, building the part libraries that figure equipment 102 determines with reference to Fig. 3
Parallactic angle point can only include point 315 and point 316, can also only include point 311 and point 312, can also only include point 311 and point
315, or point 312 and point 316.
In 230, build figure equipment 102 can image coordinate and the first default warehouse compartment width based on the warehouse compartment angle point,
Determine effective warehouse compartment.
First predetermined width can according to national standard, professional standard, the experience of life or actual library bit width into
Row determines.Effective warehouse compartment refers to the warehouse compartment of necessary being in specific region (for example, parking lot, warehouse), which can be with
In available mode or unusable state (for example, in by state of vehicle parking, the state occupied by cargo).
Specifically, build figure equipment 102 can image coordinate based on the warehouse compartment angle point and the first default warehouse compartment it is wide
Degree determines candidate warehouse compartment, and determines effective warehouse compartment in the candidate warehouse compartment.It can be true more specifically, building figure equipment 102
The area-of-interest of the fixed candidate warehouse compartment；Based on the area-of-interest, by deep neural network to the candidate warehouse compartment
It carries out classification and determines effective warehouse compartment.The deep neural network can be based on the overhead view image after label (for example, being marked with warehouse compartment
Overhead view image) to being obtained after initial neural metwork training.It is to be understood that deep neural network herein with above
The deep neural network of middle detection warehouse compartment angle point is different.The area-of-interest refers to the peripheral region of warehouse compartment angle point, the week
Enclosing region is the region around warehouse compartment angle point in certain distance.
Certainly, it builds figure equipment 102 and is also based on the deep neural network and the other information of candidate warehouse compartment is divided
Class, such as depth direction, whether be occupied.
Just to illustrate, warehouse compartment angle point only refers to warehouse compartment entrance angle point herein.Building figure equipment 102 can be according to warehouse compartment entrance
The image coordinate of angle point matches warehouse compartment inlet angle point two-by-two, and determines candidate using the described first default warehouse compartment width
Warehouse compartment.For example, for only including the overhead view image of warehouse compartment entrance angle point 315,316 and 317, building figure equipment 102 with reference to Fig. 3
Warehouse compartment entrance angle point 315,316 and 317 can be matched two-by-two.102 available three matching results of figure equipment are built, point
Not Wei warehouse compartment entrance angle point 315 and 316 matching result, the matching result and warehouse compartment of warehouse compartment entrance angle point 315 and 317 enter
The matching result of bicker point 316 and 317.Then the first predetermined width is utilized, warehouse compartment entrance angle point can be excluded by building figure equipment 102
315 and 317 matching result obtains 316 He of candidate warehouse compartment and warehouse compartment entrance angle point that warehouse compartment entrance angle point 315 and 316 determines
The 317 candidate warehouse compartments determined.
It is to be understood that when above-mentioned warehouse compartment angle point is non-warehouse compartment entrance angle point or warehouse compartment entrance angle point and Fei Ku
When the combination of position entrance angle point, corresponding variation occurs for the matching of warehouse compartment angle point, and the first default warehouse compartment width occurs corresponding
Variation.But above-mentioned variation still the application protect within the scope of.
In 240, building figure equipment 102 can be matched effective warehouse compartment with the warehouse compartment in map.Building figure equipment 102 can
To execute following two step:
First step, building figure equipment 102 can be by the warehouse compartment in the warehouse compartment angle point of effective warehouse compartment and the map
Warehouse compartment angle point is matched.
Specifically, for each warehouse compartment angle point of effective warehouse compartment, build figure equipment 102 can determine the warehouse compartment angle point with
The distance between warehouse compartment angle point of warehouse compartment in the map.The distance between two warehouse compartment angle points can be Euclidean distance.
When having one or more warehouse compartment angle points in map, when being matched to each warehouse compartment angle point of effective warehouse compartment,
Build the available one or more distances of figure equipment 102.At this point, building the distance that figure equipment 102 further judges aforementioned determination
Whether meet preset condition, is warehouse compartment angle point mutual of the then warehouse compartment in the warehouse compartment angle point of effective warehouse compartment and the map
Match.The preset condition refers to, distance in preset threshold range, and the distance in aforementioned one or more distances most
It is small.
When not having warehouse compartment angle point in map, such as build figure initial time, to each warehouse compartment angle point of effective warehouse compartment into
When row matching, the distance between two warehouse compartment angle points cannot be obtained by building figure equipment 102.So, it builds figure equipment 102 and determines map
In there is no warehouse compartment angle points with the warehouse compartment corners Matching of effective warehouse compartment.It does not need to execute following second at this point, building figure equipment 102
Step.
Second step, at least two pairs warehouse compartment angle points being mutually matched can be determined by building figure equipment 102.It should be appreciated that
It is that at least two warehouse compartment angle points can determine a warehouse compartment.If existed in overhead view image and the warehouse compartment in map is matched has
Warehouse compartment is imitated, at least two pairs warehouse compartment angle points being mutually matched can be determined by building figure equipment 102.
In 250, the pose for building figure equipment 102 can be determined based on matching result by building figure equipment 102.Build figure equipment 102
The pose of figure equipment is built described in can determining based on the described at least two pairs warehouse compartment angle points being mutually matched.
Specifically, the confidence level of each pair of warehouse compartment angle point being mutually matched can be determined by building figure equipment 102, be based on the confidence
Degree determines the pose for building figure equipment 102.It confidence level and the warehouse compartment angle point that is mutually matched and builds at a distance from figure equipment 102 and/or phase
The number that mutual matched warehouse compartment angle point is observed in map by history is related.The warehouse compartment angle point being mutually matched is (in such as map
Warehouse compartment angle point) with build bigger at a distance from figure equipment 102, confidence level is smaller；Warehouse compartment point (the warehouse compartment angle in such as map being mutually matched
Point) number that is observed by history is more, and confidence level is bigger.
In some embodiments, each pair of warehouse compartment angle point being mutually matched can be determined by formula (1) by building figure equipment 102
Confidence level.Formula (1) is as follows:
Ck=Detkf(Obk)g(dk) formula (1)
Wherein, Ck is confidence level of the kth to the warehouse compartment angle point being mutually matched, and is kth to the library being mutually matched for Detk
Confidence level of the parallactic angle point in detection network (that is, deep neural network for detecting warehouse compartment angle point)；Obk is kth to mutual
The number that matched warehouse compartment angle point is observed by history；Dk be kth to the warehouse compartment angle point being mutually matched with build at a distance from figure equipment.
In some embodiments, the pose for building figure equipment 102 can be determined by formula (2) by building figure equipment 102.Formula
(2) as follows:
Wherein, TwvIt is variable to be optimized to build module and carriage transformation matrix of the figure equipment under global map coordinate system；TviFor
Image coordinate is to the transformation matrix for building figure device coordinate system；Pk_iSeat for kth to the warehouse compartment angle point being mutually matched on the image
Mark, PkCoordinate for kth to the warehouse compartment angle point being mutually matched under map.
In 260, the map can be updated based on the pose for building figure equipment 102 by building figure equipment 102.It is set for example, building figure
Standby 102 can calculate determining new point map by trigonometric ratio is inserted into the map.
In some embodiments, the instant positioning may further include with the method for building figure: building figure equipment 102 and determines
The effective warehouse compartment unmatched in effective warehouse compartment of the overhead view image with the warehouse compartment in the map；It will be unmatched with described
Effective warehouse compartment be inserted into the map.
As an example, warehouse compartment is not present in map building figure initial time, effective warehouse compartment in the overhead view image is inevitable
It is unmatched with the warehouse compartment in map.At this point, map can be inserted into for effective warehouse compartment in the overhead view image by building figure equipment 102
In.
In some embodiments, the instant positioning may further include with the method for building figure: build 102 pairs of institutes of figure equipment
Map is stated to optimize.The optimization includes at least one of the following:
First, at least partly warehouse compartment angle point in the map is fitted according to preset positional relationship.For non-
Font parking stall, the predeterminated position relationship are that all warehouse compartment angle points are distributed on a plurality of straight line.Therefore, building figure equipment 102 can
Straight line fitting is carried out at least partly warehouse compartment angle point in map.
As an example, warehouse compartment angle point 311,312,313 and 314 is located in a straight line with reference to Fig. 3, warehouse compartment angle point 315,
316, it 317 and 318 is located in a straight line, warehouse compartment angle point 311 and 315 is located in a straight line, warehouse compartment angle point 312 and 316
In on straight line, warehouse compartment angle point 315 and 317 is located in a straight line, and warehouse compartment angle point 314 and 318 is located in a straight line.It builds
Figure equipment 102 can carry out straight line fitting to warehouse compartment angle point 311 to 318 according to above-mentioned positional relationship.
In some embodiments, straight line fitting can be carried out to warehouse compartment angle point according to formula (3) by building figure equipment 102.Formula
(3) as follows:
Wherein,And diThe direction vector of respectively i-th straight line and biasing, PijFor j-th of warehouse compartment angle on i-th straight line
Point, CijFor the confidence level of j-th of warehouse compartment angle point on i-th straight line.
Second, weight merging is carried out to warehouse compartment angle point of the mutual alignment difference in the map in preset threshold range.
When warehouse compartment corner location differences more than two in map are smaller, building figure equipment 102 can be to more than two warehouse compartment angle
Point carries out weight merging.
In some embodiments, weight merging can be carried out to warehouse compartment angle point according to formula (4) by building figure equipment 102.Formula
(4) as follows:
Wherein, PmergeFor the warehouse compartment angular coordinate after merging, PiFor i-th of warehouse compartment angular coordinate, Ci_normFor i-th of library
The weight of parallactic angle point, after being normalized for the warehouse compartment angle point confidence level relative to all (that is, I) warehouse compartment angle point confidence level to be combined
Value.
Third optimizes the direction vector of the warehouse compartment line in the map.In some embodiments, between warehouse compartment line
In parallel or overlapping relation.In some embodiments, building figure equipment 102 can be based on the parallel or overlapping relation (for example, intersection
Certain degree) direction vector of warehouse compartment line in map optimizes.In some embodiments, it builds figure equipment 102 and works as library
When parallel and vertical between bit line, the method that is optimized based on the parallel and/or vertical relationship of warehouse compartment line to warehouse compartment line can be with
It is described with reference to Fig. 3 as example.
It with reference to Fig. 3, is parallel to each other between warehouse compartment line 301,302,303 and 304, forms the first warehouse compartment line set；Warehouse compartment line
It is parallel to each other between 305 to 310, forms the second warehouse compartment line set.Any warehouse compartment line and institute in the first warehouse compartment line set
Any warehouse compartment line stated in the second warehouse compartment line set is mutually perpendicular to.Figure equipment 102 is built according to above-mentioned parallel and/or vertical relation
The direction vector of warehouse compartment line in map optimizes.
In some embodiments, building figure equipment 102 can be according in formula (5), formula (6) and formula (7) to map
The direction vector of warehouse compartment line optimizes.Formula (5) is as follows to formula (7):
Wherein,ForDesired unit direction vector；For withThere may be all warehouse compartment lines of parallel relation
Unit direction vector；For withThere may be the unit direction vectors of all warehouse compartment lines of vertical relation；ForList
Position orthogonal vectors,WithIt can be obtained by the method clustered；Cj_lineWith Ck_lineRespectivelyWithWeight, with presence
It is related in the confidence level of all warehouse compartment angle points on the warehouse compartment line.BecauseFor withThere may be vertical relations, soIt is rightContribution by orthogonal to thatIt generates.
4th, the warehouse compartment angle point of the warehouse compartment in the map is optimized based on the second default warehouse compartment width.
Second predetermined width can according to national standard, professional standard, the experience of life or actual library bit width into
Row determines.Second predetermined width and the first predetermined width above can be identical or different.
Specifically, building figure equipment 102 can be according in the second default warehouse compartment width adjustment map between two warehouse compartment angle points
Distance, and then optimize the position of two warehouse compartment angle points.
In some embodiments, global optimization can be carried out in conjunction with above-mentioned four kinds of optimization method to maps by building figure equipment 102.
As an example, global optimization can be carried out according to formula (8) to map by building figure equipment 102.
Wherein,For the set of all warehouse compartment line direction vectors；dopFor the set of all library bit line bias；PopIt is all
The set of warehouse compartment angle point；Twv_opFor the set for building all pose key frames of figure equipment.
Above four set are variable to be optimized.In some embodiments, building figure equipment 102 can be according to formula (9)
Aforementioned four set is optimized to (14).
Formula (9) to (11) can guarantee the warehouse compartment line after optimization, warehouse compartment angle point and pose and the optimization of building figure equipment
Preceding numerical value is close, will not generate mutation.Formula (12) is the wide constraint of warehouse compartment, Pr1_opAnd Pr2_opIt is belonging respectively to same
Two warehouse compartment angle points (for example, warehouse compartment entrance angle point) of warehouse compartment, Lot_W are default warehouse compartment width information.Formula (13) is to build figure
The projection error for the warehouse compartment angle point that equipment pose is observed with it constrains, the pose and map office for building figure equipment after guaranteeing optimization
Parallactic angle point, which meets, sees projection relation, and formula (14) guarantees the warehouse compartment angle point P belonged on i-th warehouse compartment lineij_opStill position after optimization
In on warehouse compartment line.
With reference to above description, when carrying out building figure and optimization, whole warehouse compartment angle points can be grasped by building figure equipment 102
Make, can also the warehouse compartment angle point to part operate.It can be only to whole warehouse compartment entrance angle points for example, building figure equipment 102
It is operated.It should be understood that figure and optimization are built when building the completion of figure equipment 102, for each effective warehouse compartment, map
On there is only in two warehouse compartment entrance angle points, rather than complete four warehouse compartment angle points.At this point, building figure equipment 102 can use
It is mutually perpendicular between warehouse compartment depth direction, warehouse compartment line or parallel relationship and default warehouse compartment depth value restores complete four
A warehouse compartment angle point.
Above using non-font parking stall as example, in conjunction with Fig. 3, detailed retouch has been carried out with the method for building figure to instant positioning
It states.Certainly, warehouse compartment may be other warehouse compartments other than non-font parking stall.According to the variation of warehouse compartment type, library can be combined
The actual conditions of position are to above-mentioned while positioning and build drawing method and make some changes.It is to be understood that above-mentioned change is simultaneously
Do not pay creative labor, above-mentioned change still this application claims within the scope of.
For example, vertex is not present on warehouse compartment boundary line when the boundary line of warehouse compartment is round and smooth curve (for example, round).
Building figure equipment 102 can determine that the point on warehouse compartment boundary line on specific direction is anchor point, and then determine warehouse compartment angle point.Meanwhile
The positional relationship that building figure equipment 102 can be respectively positioned on circle according to all warehouse compartment angle points is fitted warehouse compartment angle point.
In another example the entrance of warehouse compartment is different from the entrance on non-font parking stall, right when warehouse compartment is "-" type parking stall
The default warehouse compartment width answered also needs to modify accordingly.
For another example vertical relationship is not present between the warehouse compartment boundary line of intersection when warehouse compartment is oblique line parking stall.Build figure
Equipment 102 can carry out excellent according to the direction vector of the warehouse compartment line in the corner dimension to map between the warehouse compartment boundary line of intersection
Change.
In conclusion after reading this detailed disclosures, it will be understood by those skilled in the art that aforementioned detailed disclosure
Content can be only presented in an illustrative manner, and can not be restrictive.Although not explicitly described or shown herein, this field skill
Art personnel are understood that improve and modify it is intended to include the various reasonable changes to embodiment.These change, improve and
It modifies and is intended to be proposed by the disclosure, and in the spirit and scope of the exemplary embodiment of the disclosure.
In addition, certain terms in the application have been used for describing implementation of the disclosure example.For example, " one embodiment ",
" embodiment " and/or " some embodiments " means to combine the special characteristic of embodiment description, and structure or characteristic may include
In at least one embodiment of the disclosure.Therefore, it can emphasize and it is to be understood that right in the various pieces of this specification
Two or more references of " embodiment " or " one embodiment " or " alternate embodiment " are not necessarily all referring to identical implementation
Example.In addition, special characteristic, structure or characteristic can be appropriately combined in one or more other embodiments of the present disclosure.
It should be appreciated that in the foregoing description of embodiment of the disclosure, in order to help to understand a feature, originally for simplification
Disclosed purpose, the application sometimes combine various features in single embodiment, attached drawing or its description.Alternatively, the application is again
Be by various characteristic dispersions in multiple the embodiment of the present invention.However, this be not to say that the combination of these features be it is necessary,
Those skilled in the art are entirely possible to come out a portion feature extraction as individual when reading the application
Embodiment understands.That is, embodiment in the application it can be appreciated that multiple secondary embodiments integration.And it is each
The content of secondary embodiment is also to set up when being less than individually all features of aforementioned open embodiment.
In some embodiments, the quantity or property for certain embodiments of the application to be described and claimed as are expressed
The number of matter is interpreted as in some cases through term " about ", " approximation " or " substantially " modification.For example, unless otherwise saying
Bright, otherwise " about ", " approximation " or " substantially " can indicate ± 20% variation of the value of its description.Therefore, in some embodiments
In, the numerical parameter listed in written description and the appended claims is approximation, can be tried according to specific embodiment
Scheme the required property obtained and changes.In some embodiments, numerical parameter should be according to the quantity of the effective digital of report simultaneously
It is explained by the common rounding-off technology of application.Although illustrating that some embodiments of the application list broad range of numerical value
Range and parameter are approximations, but numerical value reported as precisely as possible is all listed in specific embodiment.
Herein cited each patent, patent application, the publication and other materials of patent application, such as article, books,
Specification, publication, file, article etc. can be incorporated herein by reference.Full content for all purposes, in addition to
Its relevant any prosecution file history, may or conflicting any identical or any possibility inconsistent with this document
On any identical prosecution file history of the restrictive influence of the widest range of claim.Now or later and this document
It is associated.For example, if in description, definition and/or the use of term associated with any included material and this
The relevant term of document, description, definition and/or between there are it is any inconsistent or conflict when, be using the term in this document
It is quasi-.
Finally, it is to be understood that the embodiment of application disclosed herein is the explanation to the principle of the embodiment of the application.
Other modified embodiments are also within the scope of application.Therefore, herein disclosed embodiment it is merely exemplary rather than
Limitation.Those skilled in the art can take alternative configuration according to the embodiment in the application to realize the invention in the application.
Therefore, embodiments herein is not limited to which embodiment accurately described in application.
Claims (12)
1. a kind of positioning immediately and the method for building figure, which is characterized in that the described method includes:
Obtain overhead view image；
Determine the image coordinate of the warehouse compartment angle point and the warehouse compartment angle point in the overhead view image, wherein each warehouse compartment angle point pair
Answer an anchor point of warehouse compartment；
The default warehouse compartment width of image coordinate and first based on the warehouse compartment angle point, determines effective warehouse compartment；
Effective warehouse compartment is matched with the warehouse compartment in map；
The pose for building figure equipment is determined based on matching result；
The map is updated based on the pose for building figure equipment.
2. positioning immediately as described in claim 1 and the method for building figure, which is characterized in that the acquisition overhead view image includes:
An at least visual pattern is obtained,
An at least visual pattern is converted into an at least sub- overhead view image by inverse perspective mapping, and
An at least sub- overhead view image is spliced into the overhead view image.
3. positioning immediately as described in claim 1 and the method for building figure, which is characterized in that in the determination overhead view image
Warehouse compartment angle point and its image coordinate, comprising:
Determine that the warehouse compartment angle point in the overhead view image, the warehouse compartment angle point are warehouse compartment entrance angle point based on deep neural network.
4. positioning immediately as described in claim 1 and the method for building figure, which is characterized in that described based on the warehouse compartment angle point
Image coordinate and the first default warehouse compartment width, determine effective warehouse compartment, comprising:
Image coordinate and the first default warehouse compartment width based on the warehouse compartment angle point determine candidate warehouse compartment, and in the candidate
Effective warehouse compartment is determined in warehouse compartment.
5. positioning immediately and the method for building figure as claimed in claim 4, which is characterized in that described true in the candidate warehouse compartment
Fixed effective warehouse compartment, comprising:
Determine the area-of-interest of the candidate warehouse compartment；
Based on the area-of-interest, classification is carried out to the candidate warehouse compartment by deep neural network and determines effective library
Position.
6. positioning immediately as described in claim 1 and the method for building figure, which is characterized in that
Described match effective warehouse compartment with the warehouse compartment in map include:
The warehouse compartment angle point of effective warehouse compartment is matched with the warehouse compartment angle point of the warehouse compartment in the map；
Determine at least two pairs warehouse compartment angle points being mutually matched；
It is described to determine that the pose for building figure equipment includes: based on matching result
Based on the pose for building figure equipment described in the described at least two pairs warehouse compartment angle points being mutually matched determinations.
7. positioning immediately as claimed in claim 6 and the method for building figure, which is characterized in that the library by effective warehouse compartment
Parallactic angle point is matched with the warehouse compartment angle point of the warehouse compartment in the map, comprising:
For each warehouse compartment angle point of effective warehouse compartment,
Determine the distance between the warehouse compartment angle point of warehouse compartment in the warehouse compartment angle point and the map；
Judge whether the distance meets preset condition, is the warehouse compartment then in the warehouse compartment angle point of effective warehouse compartment and the map
Warehouse compartment angle point be mutually matched.
8. positioning immediately as claimed in claim 7 and the method for building figure, which is characterized in that the preset condition is the distance
In preset threshold range, and the distance is minimum in the preset threshold range.
9. positioning immediately as claimed in claim 6 and the method for building figure, which is characterized in that described based at least two pairs of phases
The pose of figure equipment is built described in mutual matched warehouse compartment angle point is determining, comprising:
Determine the confidence level of each pair of warehouse compartment angle point being mutually matched, the confidence level and the warehouse compartment angle point being mutually matched and institute
It states the distance for building figure equipment and/or number that the warehouse compartment angle point being mutually matched is observed in the map by history has
It closes；
Based on the pose for building figure equipment described in confidence level determination.
10. positioning immediately as described in claim 1 and the method for building figure, which is characterized in that the method further includes:
Determine the effective warehouse compartment unmatched in effective warehouse compartment of the overhead view image with the warehouse compartment in the map；
It will be inserted into the map with the described effective warehouse compartment unmatched.
11. positioning immediately as described in claim 1 and the method for building figure, which is characterized in that the method further includes right
The map optimizes, and the optimization includes at least one of the following:
At least partly warehouse compartment angle point in the map is fitted according to preset positional relationship；
Weight merging is carried out to warehouse compartment angle point of the mutual alignment difference in the map in preset threshold range；
The direction vector of warehouse compartment line in the map is optimized；
The warehouse compartment angle point of the warehouse compartment in the map is optimized based on the second default warehouse compartment width.
12. a kind of position immediately and build map device, comprising:
At least one image acquisition equipment port；
At least one storage equipment, the storage equipment include one group of instruction；And
At least one processor communicated at least one described storage equipment, wherein described when executing one group of instruction
At least one processor makes the instant positioning require any method in 1-11 with map device perform claim is built.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910400102.5A CN110132278B (en) | 2019-05-14 | 2019-05-14 | Method and device for instant positioning and mapping |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910400102.5A CN110132278B (en) | 2019-05-14 | 2019-05-14 | Method and device for instant positioning and mapping |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110132278A true CN110132278A (en) | 2019-08-16 |
| CN110132278B CN110132278B (en) | 2021-07-02 |
Family
ID=67573934
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910400102.5A Active CN110132278B (en) | 2019-05-14 | 2019-05-14 | Method and device for instant positioning and mapping |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110132278B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111337947A (en) * | 2020-05-18 | 2020-06-26 | 深圳市智绘科技有限公司 | Instant mapping and positioning method, device, system and storage medium |
| CN111862673A (en) * | 2020-06-24 | 2020-10-30 | 北京易航远智科技有限公司 | Parking lot vehicle self-positioning and map construction method based on top view |
| CN115407355A (en) * | 2022-11-01 | 2022-11-29 | 小米汽车科技有限公司 | Library position map verification method and device and terminal equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469405A (en) * | 2015-11-26 | 2016-04-06 | 清华大学 | Visual ranging-based simultaneous localization and map construction method |
| CN107856667A (en) * | 2017-11-08 | 2018-03-30 | 科大讯飞股份有限公司 | Auxiliary parking system and method |
| CN108564814A (en) * | 2018-06-06 | 2018-09-21 | 清华大学苏州汽车研究院（吴江） | A kind of parking position detection method and device based on image |
| CN108682027A (en) * | 2018-05-11 | 2018-10-19 | 北京华捷艾米科技有限公司 | VSLAM realization method and systems based on point, line Fusion Features |
| CN108875911A (en) * | 2018-05-25 | 2018-11-23 | 同济大学 | One kind is parked position detecting method |
| CN109243289A (en) * | 2018-09-05 | 2019-01-18 | 武汉中海庭数据技术有限公司 | Underground garage parking stall extracting method and system in high-precision cartography |
| CN109389013A (en) * | 2017-08-10 | 2019-02-26 | 纵目科技（上海）股份有限公司 | Parking stall combinational algorithm and medium based on parking stall principal direction and template response point |
-
2019
- 2019-05-14 CN CN201910400102.5A patent/CN110132278B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469405A (en) * | 2015-11-26 | 2016-04-06 | 清华大学 | Visual ranging-based simultaneous localization and map construction method |
| CN109389013A (en) * | 2017-08-10 | 2019-02-26 | 纵目科技（上海）股份有限公司 | Parking stall combinational algorithm and medium based on parking stall principal direction and template response point |
| CN107856667A (en) * | 2017-11-08 | 2018-03-30 | 科大讯飞股份有限公司 | Auxiliary parking system and method |
| CN108682027A (en) * | 2018-05-11 | 2018-10-19 | 北京华捷艾米科技有限公司 | VSLAM realization method and systems based on point, line Fusion Features |
| CN108875911A (en) * | 2018-05-25 | 2018-11-23 | 同济大学 | One kind is parked position detecting method |
| CN108564814A (en) * | 2018-06-06 | 2018-09-21 | 清华大学苏州汽车研究院（吴江） | A kind of parking position detection method and device based on image |
| CN109243289A (en) * | 2018-09-05 | 2019-01-18 | 武汉中海庭数据技术有限公司 | Underground garage parking stall extracting method and system in high-precision cartography |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111337947A (en) * | 2020-05-18 | 2020-06-26 | 深圳市智绘科技有限公司 | Instant mapping and positioning method, device, system and storage medium |
| CN111337947B (en) * | 2020-05-18 | 2020-09-22 | 深圳市智绘科技有限公司 | Instant mapping and positioning method, device, system and storage medium |
| CN111862673A (en) * | 2020-06-24 | 2020-10-30 | 北京易航远智科技有限公司 | Parking lot vehicle self-positioning and map construction method based on top view |
| CN111862673B (en) * | 2020-06-24 | 2021-10-15 | 北京易航远智科技有限公司 | Parking lot vehicle self-positioning and map construction method based on top view |
| CN115407355A (en) * | 2022-11-01 | 2022-11-29 | 小米汽车科技有限公司 | Library position map verification method and device and terminal equipment |
| CN115407355B (en) * | 2022-11-01 | 2023-01-10 | 小米汽车科技有限公司 | Library position map verification method and device and terminal equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110132278B (en) | 2021-07-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10373369B2 (en) | Three-dimensional pose estimation of symmetrical objects | |
| US20210209797A1 (en) | Point-based object localization from images | |
| CN110132278A (en) | A kind of instant method and device for positioning and building figure | |
| Krull et al. | 6-dof model based tracking via object coordinate regression | |
| CN110264502B (en) | Point cloud registration method and device | |
| Costante et al. | Exploiting photometric information for planning under uncertainty | |
| Narayanan et al. | Perch: Perception via search for multi-object recognition and localization | |
| CN110097064A (en) | One kind building drawing method and device | |
| Acton et al. | Biomedical image analysis: Segmentation | |
| CN109523579A (en) | A kind of matching process and device of UAV Video image and three-dimensional map | |
| Masson et al. | Robust real time tracking of 3d objects | |
| US20040165776A1 (en) | Means of matching 2D motion vector fields in a render, match, and refine iterative 3D scene model refinement system so as to attain directed hierarchical convergence and insensitivity to color, lighting, and textures | |
| CN111488783A (en) | Method and device for detecting pseudo-3D bounding box based on CNN | |
| Barath et al. | Space-partitioning RANSAC | |
| JPS62229380A (en) | Three-dimensional object displaying device | |
| CN110211190A (en) | Training method, device and the storage medium of camera self moving parameter estimation model | |
| Rother et al. | Seeing 3D objects in a single 2D image | |
| CN113570713B (en) | Semantic map construction method and device for dynamic environment | |
| CN115511970A (en) | Visual positioning method for autonomous parking | |
| Taiana et al. | Sample-Based 3D Tracking of Colored Objects: A Flexible Architecture. | |
| Zieliński et al. | 3d dense mapping with the graph of keyframe-based and view-dependent local maps | |
| Renaudeau et al. | Air-ground cooperative topometric mapping of traversable ground | |
| Frommberger | A qualitative representation of structural spatial knowledge for robot navigation with reinforcement learning | |
| Krzysztof et al. | 3D Dense Mapping with the Graph of Keyframe-Based and View-Dependent Local Maps | |
| Zielinski et al. | 3D Dense Mapping with the Graph of Keyframe-Based and View-Dependent Local Maps. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |