CN109974733A - POI display methods, device, terminal and medium for AR navigation - Google Patents
POI display methods, device, terminal and medium for AR navigation Download PDFInfo
- Publication number
- CN109974733A CN109974733A CN201910261153.4A CN201910261153A CN109974733A CN 109974733 A CN109974733 A CN 109974733A CN 201910261153 A CN201910261153 A CN 201910261153A CN 109974733 A CN109974733 A CN 109974733A
- Authority
- CN
- China
- Prior art keywords
- target construction
- vehicle
- latitude
- longitude
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
- G01C21/3453—Special cost functions, i.e. other than distance or default speed limit of road segments
- G01C21/3476—Special cost functions, i.e. other than distance or default speed limit of road segments using point of interest [POI] information, e.g. a route passing visible POIs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
Abstract
The embodiment of the invention discloses a kind of POI display methods, device, terminal and media for AR navigation, wherein, this method comprises: obtaining vehicle periphery image in real time by the camera configured on vehicle, and building analyte detection is carried out to vehicle periphery image, obtains target construction；The objective plane and its posture information in target construction are determined using Plane Detecting Technique；According to the longitude and latitude of the relative positional relationship and vehicle of target construction and vehicle, the longitude and latitude of target construction is determined；In POI information of the engine map end according to the longitude and latitude inquiry target construction of target construction；According to posture information, which is shown on the objective plane of target construction.The embodiment of the present invention solve the problems, such as in existing AR navigation mode can not display environment POI information, realize using in AR navigation procedure on real scene image convenient display environment POI information effect, check providing convenience property of environment POI information for user.
Description
Technical field
The present embodiments relate to augmented reality display technology more particularly to it is a kind of for AR navigation POI display methods,
Device, terminal and medium.
Background technique
With the fast development of computer technology, AR (Augmented Reality, augmented reality) technology is answered extensively
With realizing the information for casting out AR and true by the way that digital picture to be covered in the real world that people can see
Real environment combines together.AR, which navigates, also progresses into the visual field of people, provides more true, accurate navigation clothes for people
Business.
Under normal navigation mode, the title or retail shop's title of building are usually shown in the building block figure on road periphery
Equal POI (Point of Interest) information；And under AR navigation mode, POI information can not be still shown at present, if user thinks
It checks POI information, normal navigation mode can only be switched to and check that operating process is cumbersome, be not convenient to use.
Summary of the invention
The embodiment of the present invention provides a kind of POI display methods, device, terminal and medium for AR navigation, to realize
The effect of the convenient display environment POI information of AR navigation procedure checks that environment POI information provides convenience for user.
In a first aspect, the embodiment of the invention provides a kind of POI display methods for AR navigation, this method comprises:
Vehicle periphery image is obtained in real time by the camera configured on vehicle, and the vehicle periphery image is built
Analyte detection is built, target construction is obtained；
The objective plane and its posture information in target construction are determined using Plane Detecting Technique；
According to target construction and the relative positional relationship of the vehicle and the longitude and latitude of the vehicle, determine that target is built
Build the longitude and latitude of object；
In POI information of the engine map end according to the longitude and latitude inquiry target construction of the target construction；
According to the posture information, the POI information is shown on the objective plane of target construction.
Second aspect, the embodiment of the invention also provides a kind of POI display device for AR navigation, which includes:
Target construction determining module obtains vehicle periphery image for the camera by configuring on vehicle in real time, and
Building analyte detection is carried out to the vehicle periphery image, obtains target construction；
Objective plane and pose determining module, for determining the objective plane in target construction using Plane Detecting Technique
And its posture information；
Longitude and latitude determining module, for according to target construction and the vehicle relative positional relationship and the vehicle
Longitude and latitude, determine the longitude and latitude of target construction；
POI information enquiry module, for being built at engine map end according to the longitude and latitude inquiry target of the target construction
Build the POI information of object；
POI information display module, for according to the posture information, the POI information to be shown to target construction
On objective plane.
The third aspect, the embodiment of the invention also provides a kind of terminals, comprising:
One or more processors；
Storage device, for storing one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processing
Device realizes the POI display methods for AR navigation as described in any embodiment of the present invention.
Fourth aspect, the embodiment of the invention also provides a kind of computer readable storage mediums, are stored thereon with computer
Program realizes the POI display methods for AR navigation as described in any embodiment of the present invention when the program is executed by processor.
The embodiment of the present invention is by carrying out building quality testing to the vehicle periphery image of captured in real-time in AR navigation procedure
It surveys, determines objective plane and its position of the target construction relative to vehicle current driving direction on image using Plane Detecting Technique
The POI information of the target construction inquired is shown on objective plane solve existing according to the posture information by appearance information
Have in AR navigation mode can not display environment POI information the problem of, realize using in AR navigation procedure on real scene image just
The effect of prompt display environment POI information checks providing convenience property of environment POI information for user.
Detailed description of the invention
Fig. 1 is the flow chart for the POI display methods for AR navigation that the embodiment of the present invention one provides；
Fig. 2 is the flow chart of the POI display methods provided by Embodiment 2 of the present invention for AR navigation；
Fig. 3 is the structural schematic diagram for the POI display device for AR navigation that the embodiment of the present invention three provides；
Fig. 4 is a kind of structural schematic diagram for terminal that the embodiment of the present invention four provides.
Specific embodiment
The present invention is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining the present invention rather than limiting the invention.It also should be noted that in order to just
Only the parts related to the present invention are shown in description, attached drawing rather than entire infrastructure.
Embodiment one
Fig. 1 is the flow chart for the POI display methods for AR navigation that the embodiment of the present invention one provides, and the present embodiment can fit
For under AR navigation mode, environment POI information to be shown that the situation on real scene image, this method can be by navigating for AR
POI display device execute, which can be realized by the way of software and/or hardware, and can be integrated in and lead with AR
In the terminal for function of navigating, such as intelligent mobile terminal and mobile unit etc..
As shown in Figure 1, the POI display methods provided in this embodiment for AR navigation may include:
S110, it obtains vehicle periphery image in real time by the camera configured on vehicle, and vehicle periphery image is carried out
Building analyte detection obtains target construction.
In AR navigation procedure, it can use at least one camera configured on vehicle and acquire current driving environment in real time
Real scene image, i.e. vehicle periphery image.Due to may include in addition to building on the vehicle periphery image that currently acquires
Other kinds of environmental information, for example, therefore pedestrian, crossing, barrier, traffic sign, road can use arbitrary figure
As identification technology, the building in current driving environment, i.e. target construction are identified.Wherein, image recognition technology include but
Image recognition, the image recognition based on fractal characteristic or the image based on small echo moment characteristics neural network based is not limited to know
Not etc..
Can be illustrative, vehicle periphery image is obtained by the camera configured on vehicle in real time, and to vehicle periphery figure
As carrying out building analyte detection, target construction is obtained, comprising:
Vehicle periphery image is carried out using algorithm of target detection SSD (Single Shot Multibox Detector)
Building analyte detection obtains target construction；Wherein, algorithm of target detection SSD is detected based on neural network, neural network
It is to be obtained using the environmental samples image training with different buildings mark.In addition, YOLO also can be used in SSD detection algorithm
(You only look once) detection algorithm substitution, the present embodiment are not especially limited this.
S120, objective plane and its posture information in target construction are determined using Plane Detecting Technique.
Under AR navigation mode, user can see true three-dimension object, and from different visual angles, it is seen that building
Object plane difference.Mesh in the vehicle periphery image currently acquired is determined using the Plane Detecting Technique in visual display field
Building is marked relative to the objective plane of vehicle heading and the posture information of objective plane, it can be ensured that subsequent mark should
The accuracy of the POI information of target construction, so that the POI information of mark is adapted with the position of target structures.Wherein, pose
Information includes the tilt angle of objective plane, the information relative to the orientation of vehicle and at a distance from vehicle etc..Plane monitoring-network skill
Art can be any available method, the present embodiment in the prior art and be not especially limited.
S130, the longitude and latitude according to the relative positional relationship and vehicle of target construction and vehicle, determine target structures
The longitude and latitude of object.
Based on the real-time detection and tracking to target construction, the opposite pass of target construction and current vehicle position is determined
System, such as relative distance and relative bearing etc., are combined the vehicle latitude and longitude information that vehicle positioning technology obtains, can determine
The longitude and latitude of target construction realizes the positioning to target construction.
S140, the POI information at engine map end according to the longitude and latitude inquiry target construction of target construction.
Engine map end may include arbitrary map datum provider.According to the positioning result of target construction, from ground
POI information relevant to target construction is extracted in the database of figure engine end.
S150, foundation posture information, which is shown on the objective plane of target construction.
Illustratively, image rendering and display technology be can use, according to the posture information of objective plane, by POI information
It is shown on objective plane with arbitrary independent form of icons, the image that user sees during the navigation process is to show POI
The real scene image of information.Also, as the distance of vehicle-to-target building gradually shortens, the independent icon of POI information can be with
The Special display effect of dynamic change is carried out, to increase the authenticity for showing POI information on real scene image.
The technical solution of the present embodiment is by building the vehicle periphery image of captured in real-time in AR navigation procedure
Analyte detection, using Plane Detecting Technique determine target construction on image relative to the objective plane in vehicle current driving direction and
Its posture information shows the POI information of the target construction inquired on objective plane according to the posture information, solves
In existing AR navigation mode can not display environment POI information the problem of, realize using in AR navigation procedure in real scene image
The effect of upper convenient display environment POI information checks providing convenience property of environment POI information for user；Also, the present embodiment side
Case has the characteristics that be easily achieved, when user checks POI information during the navigation process, without being navigated mode by frequent switching
Demand is checked to meet current POI information, and applicability is higher, is also easier to promote.
Embodiment two
Fig. 2 be it is provided by Embodiment 2 of the present invention for AR navigation POI display methods flow chart, the present embodiment be
Further progress optimizes on the basis of above-described embodiment.As shown in Fig. 2, this method may include:
S210, it obtains vehicle periphery image in real time by the camera configured on vehicle, and vehicle periphery image is carried out
Building analyte detection obtains target construction.
S220, building analyte detection operation the current detection period in, to what is detected in each frame vehicle periphery image
Target construction carries out Corner Detection, obtains multiple angle points.
It, can be according to preset detection cycle to the detection of building on image with the real-time acquisition of vehicle periphery image
It carries out, wherein the being adapted to property of length of detection cycle is arranged.In the present embodiment, the angle point of target construction refers to that target is built
Aspect ratio more prominent features point on object is built, for example, target construction image region brightness changes violent point or target structures
The point of curvature maximum on object image boundary curve.Illustratively, it can be determined according to the geometry feature of target construction
Its angle point, if target construction can simplify for rule stereochemical structure, the vertex of the stereochemical structure can be used as target
The angle point of building.
Wherein, angular-point detection method includes but is not limited to Harris angular-point detection method or the Corner Detection side Shi-Tomasi
Method etc..
S230, corners Matching is carried out to multiple angle points, obtains multiple matching angle points.
By the Corner Detection in the current detection period, multiple angle points of target construction on available every frame image.
For the characteristic point at same position on same building object, there are corresponding relationships on different frame image, i.e., at same position
Characteristic point belongs to matching angle point.Specifically, binary descriptor can be used, such as BRIEF (Binary Robust
Independent Elementary Features) description is sub or FREAK (FastRetinaKeypoint) description is sub, and it is right
Angle point of the target construction on multiple image is matched.
Wherein, it is that the characteristic point having detected that is described that BRIEF, which describes son, has abandoned and has utilized area grayscale histogram
The conventional method of Expressive Features point, accelerates the speed of feature descriptor foundation, while also greatly reducing characteristic matching
Time is a kind of very quick and potential algorithm.
S240, the processing of multiframe trigonometric ratio is carried out to multiple matching angle points, obtains target construction under camera coordinate system
Multiple 3D points and its coordinate.
It is handled by multiframe trigonometric ratio, it is estimated that depth value of each matching angle point on corresponding picture frame.Tool
Body, can coordinate conversion relation between image coordinate system and camera coordinate system based on each frame image, determine every frame
Coordinate relational expression of each matching angle point in image coordinate system and camera coordinate system on image, by solving by adjacent two frame
The equation group that the coordinate relational expression composition of angle point is matched on image, it is right under camera coordinate system to can be obtained each matching angle point
The 3D coordinate answered.
S250, plane monitoring-network is carried out using multiple 3D points and its coordinate, obtains objective plane and its posture information.
Plane monitoring-network is carried out using multiple 3D points and its coordinate, can determine target construction under camera coordinate system
Posture information under camera coordinate system of objective plane and objective plane.It is sat again using camera coordinate system and image
Coordinate conversion relation between mark system, by the position of objective plane and objective plane of the target construction under camera coordinate system
Appearance information MAP is to image coordinate system, to realize the POI information mark under image coordinate system.
S260, the longitude and latitude according to the relative positional relationship and vehicle of target construction and vehicle, determine target structures
The longitude and latitude of object.
S270, the POI information at engine map end according to the longitude and latitude inquiry target construction of target construction.
S280, foundation posture information, which is shown on the objective plane of target construction.
Specifically, can use image rendering and display technology, POI information is shown on objective plane, user is leading
The image seen during boat is the real scene image for showing POI information.
Based on the above technical solution, optionally, according to the relative positional relationship of target construction and vehicle and
The longitude and latitude of vehicle determines the longitude and latitude of target construction, comprising:
The longitude and latitude of vehicle is obtained in real time；
Obtain the 2D coordinate of at least one characteristic point in the image-region of target construction；
2D coordinate is converted to the 3D coordinate of camera coordinate system；
The distance and bearing of each characteristic point and camera is calculated according to 3D coordinate；
According to the longitude and latitude of vehicle, distance and bearing, the longitude and latitude of target construction is calculated.
Wherein, according to the coordinate conversion relation between image coordinate system and camera coordinate system, by the spy of target construction
The 2D coordinate transformation of sign point is the 3D coordinate under camera coordinate system.In the camera coordinate system, camera lens focus, that is, coordinate origin,
Can use the distance between each 3D coordinate and coordinate origin indicates target construction and camera in the camera coordinate system
Distance, using each 3D coordinate relative to coordinate origin orientation references in the camera coordinate system target construction relative to
The orientation of camera；And then using the coordinate conversion relation between vehicle axis system and camera coordinate system, camera is determined
Distance and bearing relative to vehicle；Finally based on above-mentioned determining target construction relative to camera distance and bearing, take the photograph
As head is relative to the distance and bearing of vehicle and the latitude and longitude information of vehicle, the longitude and latitude letter of target construction can be obtained
Breath.
Optionally, the vehicle longitude and latitude obtained in real time, comprising: according to speed, vehicle inertia navigation and the GPS obtained in real time
Information and the image information of camera shooting, are obtained using Extended Kalman filter (Extended Kalman Filter, EKF)
To vehicle longitude and latitude.Meanwhile the information such as the course angle in vehicle travel process also can be obtained.
The technical solution of the present embodiment is by building the vehicle periphery image of captured in real-time in AR navigation procedure
Analyte detection, and determine multiple matching angle points in target construction, and then utilize the processing of multiframe trigonometric ratio and Plane Detecting Technique
Determine the objective plane of target structures and the posture information of objective plane, according to the posture information, the target that will be inquired
The POI information of building is shown on the objective plane of target construction, and ring can not be shown by solving in existing AR navigation mode
The problem of border POI information, realize using in AR navigation procedure on real scene image convenient display environment POI information effect,
Providing convenience property of environment POI information is checked for user.
Embodiment three
Fig. 3 is the structural schematic diagram for the POI display device for AR navigation that the embodiment of the present invention three provides, the present embodiment
It is applicable under AR navigation mode, environment POI information is shown to the situation on real scene image.The device can use software
And/or the mode of hardware is realized, and can be integrated in the terminal with AR navigation feature, such as intelligent mobile terminal and vehicle-mounted is set
It is standby etc..
As shown in figure 3, the POI display device provided in this embodiment for AR navigation may include that target construction determines
Module 310, objective plane and pose determining module 320, longitude and latitude determining module 330, POI information enquiry module 340 and POI letter
Cease display module 350, in which:
Target construction determining module 310 obtains vehicle periphery image for the camera by configuring on vehicle in real time,
And building analyte detection is carried out to vehicle periphery image, obtain target construction；
Objective plane and pose determining module 320, for determining the target in target construction using Plane Detecting Technique
Plane and its posture information；
Longitude and latitude determining module 330, for according to the relative positional relationship of target construction and vehicle and the warp of vehicle
Latitude determines the longitude and latitude of target construction；
POI information enquiry module 340, for inquiring target structures according to the longitude and latitude of target construction at engine map end
The POI information of object；
POI information display module 350, for according to posture information, which to be shown to the target of target construction
In plane.
Optionally, target construction determining module 310 is specifically used for:
Building analyte detection is carried out to vehicle periphery image using algorithm of target detection SSD, obtains target construction；
Wherein, algorithm of target detection SSD is detected based on neural network, and neural network is using with different buildings
The environmental samples image training of object mark obtains.
Optionally, objective plane and pose determining module 320 include:
Corner Detection unit, within the current detection period that building analyte detection operates, to each frame vehicle periphery figure
The target construction detected as in carries out Corner Detection, obtains multiple angle points；
Corners Matching unit obtains multiple matching angle points for carrying out corners Matching to multiple angle points；
Trigonometric ratio processing unit obtains target construction and exists for carrying out the processing of multiframe trigonometric ratio to multiple matching angle points
Multiple 3D points and its coordinate under camera coordinate system；
Plane monitoring-network unit obtains objective plane and its position for carrying out plane monitoring-network using multiple 3D points and its coordinate
Appearance information.
Optionally, longitude and latitude determining module 330 includes:
The real-time acquiring unit of vehicle longitude and latitude, for obtaining the longitude and latitude of vehicle in real time；
Characteristic point 2D coordinate acquiring unit, the 2D of at least one characteristic point in the image-region for obtaining target construction
Coordinate；
Coordinate transformation unit, for 2D coordinate to be converted to the 3D coordinate of camera coordinate system；
Distance and bearing determination unit, for calculating the distance and bearing of each characteristic point and camera according to 3D coordinate；
Target construction longitude and latitude determination unit calculates target structures for longitude and latitude, the distance and bearing according to vehicle
The longitude and latitude of object.
Optionally, the real-time acquiring unit of vehicle longitude and latitude is specifically used for:
According to speed, vehicle inertia navigation and the GPS information and the image information of camera shooting obtained in real time, utilize
Extended Kalman filter obtains vehicle longitude and latitude.
POI display device provided by the embodiment of the present invention for AR navigation can be performed any embodiment of that present invention and be mentioned
The POI display methods for AR navigation supplied, has the corresponding functional module of execution method and beneficial effect.In the present embodiment not
The content of detailed description can be with reference to the description in any means embodiment of the present invention.
Example IV
Fig. 4 is a kind of structural schematic diagram for terminal that the embodiment of the present invention four provides.Fig. 4, which is shown, to be suitable for being used to realizing this
The block diagram of the exemplary terminal 412 of invention embodiment.The terminal 412 that Fig. 4 is shown is only an example, should not be to the present invention
The function and use scope of embodiment bring any restrictions.
As shown in figure 4, terminal 412 is showed in the form of general purpose terminal.The component of terminal 412 can include but is not limited to:
One or more processor 416, storage device 428 connect different system components (including storage device 428 and processor
416) bus 418.
Bus 418 indicates one of a few class bus structures or a variety of, including storage device bus or storage device control
Device processed, peripheral bus, graphics acceleration port, processor or total using the local of any bus structures in a variety of bus structures
Line.For example, these architectures include but is not limited to industry standard architecture (Industry Subversive
Alliance, ISA) bus, microchannel architecture (Micro Channel Architecture, MAC) bus is enhanced
Isa bus, Video Electronics Standards Association (Video Electronics Standards Association, VESA) local are total
Line and peripheral component interconnection (Peripheral Component Interconnect, PCI) bus.
Terminal 412 typically comprises a variety of computer system readable media.These media can be it is any can be by terminal
The usable medium of 412 access, including volatile and non-volatile media, moveable and immovable medium.
Storage device 428 may include the computer system readable media of form of volatile memory, such as arbitrary access
Memory (Random Access Memory, RAM) 430 and/or cache memory 432.Terminal 412 can be wrapped further
Include other removable/nonremovable, volatile/non-volatile computer system storage mediums.Only as an example, storage system
434 can be used for reading and writing immovable, non-volatile magnetic media (Fig. 4 do not show, commonly referred to as " hard disk drive ").Although
It is not shown in Fig. 4, the disc driver for reading and writing to removable non-volatile magnetic disk (such as " floppy disk ") can be provided, and
To removable anonvolatile optical disk, such as CD-ROM (Compact Disc Read-Only Memory, CD-ROM), number
Optic disk (Digital Video Disc-Read Only Memory, DVD-ROM) or other optical mediums) read-write CD drive
Dynamic device.In these cases, each driver can be connected by one or more data media interfaces with bus 418.It deposits
Storage device 428 may include at least one program product, which has one group of (for example, at least one) program module, this
A little program modules are configured to perform the function of various embodiments of the present invention.
Program/utility 440 with one group of (at least one) program module 442 can store in such as storage dress
It sets in 428, such program module 442 includes but is not limited to operating system, one or more application program, other program moulds
It may include the realization of network environment in block and program data, each of these examples or certain combination.Program module
442 usually execute function and/or method in embodiment described in the invention.
Terminal 412 can also be logical with one or more external equipments 414 (such as keyboard, direction terminal, display 424 etc.)
Letter, can also be enabled a user to one or more terminal interact with the terminal 412 communicate, and/or with make the terminal 412
Any terminal (such as network interface card, modem etc.) communication that can be communicated with one or more of the other computing terminal.This
Kind communication can be carried out by input/output (I/O) interface 422.Also, terminal 412 can also by network adapter 420 with
One or more network (such as local area network (Local Area Network, LAN), wide area network (Wide Area Network,
WAN) and/or public network, for example, internet) communication.As shown in figure 4, network adapter 420 passes through bus 418 and terminal 412
Other modules communication.It should be understood that although not shown in the drawings, other hardware and/or software mould can be used in conjunction with terminal 412
Block, including but not limited to: microcode, terminal driver, redundant processor, external disk drive array, disk array
(Redundant Arrays of Independent Disks, RAID) system, tape drive and data backup storage system
System etc..
The program that processor 416 is stored in storage device 428 by operation, thereby executing various function application and number
According to processing, such as realize the POI display methods provided by any embodiment of the invention for AR navigation, this method can wrap
It includes:
Vehicle periphery image is obtained in real time by the camera configured on vehicle, and the vehicle periphery image is built
Analyte detection is built, target construction is obtained；
The objective plane and its posture information in target construction are determined using Plane Detecting Technique；
According to target construction and the relative positional relationship of the vehicle and the longitude and latitude of the vehicle, determine that target is built
Build the longitude and latitude of object；
In POI information of the engine map end according to the longitude and latitude inquiry target construction of the target construction；
According to the posture information, the POI information is shown on the objective plane of target construction.
Embodiment five
The embodiment of the present invention five additionally provides a kind of computer readable storage medium, is stored thereon with computer program, should
It is realized when program is executed by processor such as the POI display methods provided by any embodiment of the invention for AR navigation, the party
Method may include:
Vehicle periphery image is obtained in real time by the camera configured on vehicle, and the vehicle periphery image is built
Analyte detection is built, target construction is obtained；
The objective plane and its posture information in target construction are determined using Plane Detecting Technique；
According to target construction and the relative positional relationship of the vehicle and the longitude and latitude of the vehicle, determine that target is built
Build the longitude and latitude of object；
In POI information of the engine map end according to the longitude and latitude inquiry target construction of the target construction；
According to the posture information, the POI information is shown on the objective plane of target construction.
The computer storage medium of the embodiment of the present invention, can be using any of one or more computer-readable media
Combination.Computer-readable medium can be computer-readable signal media or computer readable storage medium.It is computer-readable
Storage medium for example may be-but not limited to-the system of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, device or
Device, or any above combination.The more specific example (non exhaustive list) of computer readable storage medium includes: tool
There are electrical connection, the portable computer diskette, hard disk, random access memory (RAM), read-only memory of one or more conducting wires
(ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-
ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.In this document, computer-readable storage
Medium can be any tangible medium for including or store program, which can be commanded execution system, device or device
Using or it is in connection.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.
The program code for including on computer-readable medium can transmit with any suitable medium, including --- but it is unlimited
In wireless, electric wire, optical cable, RF etc. or above-mentioned any appropriate combination.
The computer for executing operation of the present invention can be write with one or more programming languages or combinations thereof
Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++,
It further include conventional procedural programming language-such as " C " language or similar programming language.Program code can be with
It fully executes, partly execute on the user computer on the user computer, being executed as an independent software package, portion
Divide and partially executes or executed on remote computer or terminal completely on the remote computer on the user computer.It is relating to
And in the situation of remote computer, remote computer can pass through the network of any kind --- including local area network (LAN) or extensively
Domain net (WAN)-be connected to subscriber computer, or, it may be connected to outer computer (such as provided using Internet service
Quotient is connected by internet).
Note that the above is only a better embodiment of the present invention and the applied technical principle.It will be appreciated by those skilled in the art that
The invention is not limited to the specific embodiments described herein, be able to carry out for a person skilled in the art it is various it is apparent variation,
It readjusts and substitutes without departing from protection scope of the present invention.Therefore, although being carried out by above embodiments to the present invention
It is described in further detail, but the present invention is not limited to the above embodiments only, without departing from the inventive concept, also
It may include more other equivalent embodiments, and the scope of the invention is determined by the scope of the appended claims.
Claims (12)
1. a kind of POI display methods for AR navigation characterized by comprising
Vehicle periphery image is obtained in real time by the camera configured on vehicle, and building is carried out to the vehicle periphery image
Detection, obtains target construction；
The objective plane and its posture information in target construction are determined using Plane Detecting Technique；
According to target construction and the relative positional relationship of the vehicle and the longitude and latitude of the vehicle, target construction is determined
Longitude and latitude；
In POI information of the engine map end according to the longitude and latitude inquiry target construction of the target construction；
According to the posture information, the POI information is shown on the objective plane of target construction.
2. the method according to claim 1, wherein the camera by configuring on vehicle obtains vehicle in real time
Surrounding image, and building analyte detection is carried out to the vehicle periphery image, obtain target construction, comprising:
Building analyte detection is carried out to the vehicle periphery image using algorithm of target detection SSD, obtains target construction；
Wherein, the algorithm of target detection SSD is detected based on neural network, and the neural network is using with difference
The environmental samples image training of building mark obtains.
3. the method according to claim 1, wherein described determined in target construction using Plane Detecting Technique
Objective plane and its posture information, comprising:
Within the current detection period of building analyte detection operation, the target detected in each frame vehicle periphery image is built
It builds object and carries out Corner Detection, obtain multiple angle points；
Corners Matching is carried out to the multiple angle point, obtains multiple matching angle points；
The processing of multiframe trigonometric ratio is carried out to the multiple matching angle point, obtains target construction under the camera coordinate system
Multiple 3D points and its coordinate；
Plane monitoring-network is carried out using the multiple 3D point and its coordinate, obtains the objective plane and its posture information.
4. the method according to claim 1, wherein the opposite position according to target construction and the vehicle
The longitude and latitude for setting relationship and the vehicle determines the longitude and latitude of target construction, comprising:
The longitude and latitude of vehicle is obtained in real time；
Obtain the 2D coordinate of at least one characteristic point in the image-region of the target construction；
The 2D coordinate is converted to the 3D coordinate of the camera coordinate system；
The distance and bearing of each characteristic point Yu the camera is calculated according to the 3D coordinate；
According to the longitude and latitude of the vehicle, the distance and bearing, the longitude and latitude of target construction is calculated.
5. according to the method described in claim 4, it is characterized in that, the vehicle longitude and latitude obtained in real time, comprising:
According to speed, vehicle inertia navigation and the GPS information and the image information of camera shooting obtained in real time, utilize
Extended Kalman filter obtains the vehicle longitude and latitude.
6. a kind of POI display device for AR navigation characterized by comprising
Target construction determining module obtains vehicle periphery image for the camera by configuring on vehicle in real time, and to institute
It states vehicle periphery image and carries out building analyte detection, obtain target construction；
Objective plane and pose determining module, for determined using Plane Detecting Technique objective plane in target construction and its
Posture information；
Longitude and latitude determining module, for according to target construction and the relative positional relationship of the vehicle and the warp of the vehicle
Latitude determines the longitude and latitude of target construction；
POI information enquiry module, for inquiring target construction according to the longitude and latitude of the target construction at engine map end
POI information；
POI information display module, for according to the posture information, the POI information to be shown to the target of target construction
In plane.
7. device according to claim 6, which is characterized in that the target construction determining module is specifically used for:
Building analyte detection is carried out to the vehicle periphery image using algorithm of target detection SSD, obtains target construction；
Wherein, the algorithm of target detection SSD is detected based on neural network, and the neural network is using with difference
The environmental samples image training of building mark obtains.
8. device according to claim 6, which is characterized in that the objective plane and pose determining module include:
Corner Detection unit, within the current detection period that the building analyte detection operates, to each frame vehicle periphery figure
The target construction detected as in carries out Corner Detection, obtains multiple angle points；
Corners Matching unit obtains multiple matching angle points for carrying out corners Matching to the multiple angle point；
Trigonometric ratio processing unit obtains target construction and exists for carrying out the processing of multiframe trigonometric ratio to the multiple matching angle point
Multiple 3D points and its coordinate under the camera coordinate system；
Plane monitoring-network unit, for using the multiple 3D point and its coordinate progress plane monitoring-network, obtain the objective plane and
Its posture information.
9. device according to claim 6, which is characterized in that the longitude and latitude determining module includes:
The real-time acquiring unit of vehicle longitude and latitude, for obtaining the longitude and latitude of vehicle in real time；
Characteristic point 2D coordinate acquiring unit, the 2D of at least one characteristic point in the image-region for obtaining the target construction
Coordinate；
Coordinate transformation unit, for the 2D coordinate to be converted to the 3D coordinate of the camera coordinate system；
Distance and bearing determination unit, for calculating each characteristic point at a distance from the camera and side according to the 3D coordinate
Position；
Target construction longitude and latitude determination unit calculates target for longitude and latitude, the distance and bearing according to the vehicle
The longitude and latitude of building.
10. device according to claim 9, which is characterized in that the real-time acquiring unit of vehicle longitude and latitude is specifically used for:
According to speed, vehicle inertia navigation and the GPS information and the image information of camera shooting obtained in real time, utilize
Extended Kalman filter obtains the vehicle longitude and latitude.
11. a kind of terminal characterized by comprising
One or more processors；
Storage device, for storing one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors are real
Now such as the POI display methods as claimed in any one of claims 1 to 5 for AR navigation.
12. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
It is realized when execution such as the POI display methods as claimed in any one of claims 1 to 5 for AR navigation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910261153.4A CN109974733A (en) | 2019-04-02 | 2019-04-02 | POI display methods, device, terminal and medium for AR navigation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910261153.4A CN109974733A (en) | 2019-04-02 | 2019-04-02 | POI display methods, device, terminal and medium for AR navigation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109974733A true CN109974733A (en) | 2019-07-05 |
Family
ID=67082469
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910261153.4A Pending CN109974733A (en) | 2019-04-02 | 2019-04-02 | POI display methods, device, terminal and medium for AR navigation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109974733A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110595494A (en) * | 2019-09-17 | 2019-12-20 | 百度在线网络技术（北京）有限公司 | Map error determination method and device |
| CN110807431A (en) * | 2019-11-06 | 2020-02-18 | 上海眼控科技股份有限公司 | Object positioning method and device, electronic equipment and storage medium |
| CN111698646A (en) * | 2020-06-08 | 2020-09-22 | 浙江商汤科技开发有限公司 | Positioning method and device |
| CN112146656A (en) * | 2020-09-03 | 2020-12-29 | 武汉大学 | Indoor navigation visualization method based on augmented reality |
| CN112541467A (en) * | 2020-12-21 | 2021-03-23 | 杭州光粒科技有限公司 | Information display method, device and equipment and storage medium |
| CN112559675A (en) * | 2019-09-25 | 2021-03-26 | 上海博泰悦臻电子设备制造有限公司 | Method, apparatus, and storage medium for determining results to a user query in a vehicle |
| CN113137970A (en) * | 2020-01-20 | 2021-07-20 | 北京智慧图科技有限责任公司 | Applet AR navigation system |
| CN113448322A (en) * | 2020-03-26 | 2021-09-28 | 宝马股份公司 | Remote operation method and system for vehicle, storage medium, and electronic device |
| CN113779184A (en) * | 2020-06-09 | 2021-12-10 | 大众问问(北京)信息科技有限公司 | Information interaction method and device and electronic equipment |
| CN115164929A (en) * | 2022-08-02 | 2022-10-11 | 上海有个机器人有限公司 | Method and system for creating floor POI point on robot map and related product |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102980570A (en) * | 2011-09-06 | 2013-03-20 | 上海博路信息技术有限公司 | Live-scene augmented reality navigation system |
| CN105917329A (en) * | 2014-02-18 | 2016-08-31 | 日立麦克赛尔株式会社 | Information display device and information display program |
| CN107515888A (en) * | 2017-06-30 | 2017-12-26 | 百度在线网络技术（北京）有限公司 | Hooking method, device and the computer-readable recording medium of POI in D Urban model |
| CN107665507A (en) * | 2016-07-29 | 2018-02-06 | 成都理想境界科技有限公司 | The method and device of augmented reality is realized based on plane monitoring-network |
| CN108615260A (en) * | 2018-03-26 | 2018-10-02 | 湖北大学 | The method and device that shows of augmented reality digital culture content is carried out under a kind of exception actual environment |
| CN108898604A (en) * | 2018-06-28 | 2018-11-27 | 上海连尚网络科技有限公司 | Method and apparatus for handling image |
| CN109143305A (en) * | 2018-09-30 | 2019-01-04 | 百度在线网络技术（北京）有限公司 | Automobile navigation method and device |
-
2019
- 2019-04-02 CN CN201910261153.4A patent/CN109974733A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102980570A (en) * | 2011-09-06 | 2013-03-20 | 上海博路信息技术有限公司 | Live-scene augmented reality navigation system |
| CN105917329A (en) * | 2014-02-18 | 2016-08-31 | 日立麦克赛尔株式会社 | Information display device and information display program |
| CN107665507A (en) * | 2016-07-29 | 2018-02-06 | 成都理想境界科技有限公司 | The method and device of augmented reality is realized based on plane monitoring-network |
| CN107515888A (en) * | 2017-06-30 | 2017-12-26 | 百度在线网络技术（北京）有限公司 | Hooking method, device and the computer-readable recording medium of POI in D Urban model |
| CN108615260A (en) * | 2018-03-26 | 2018-10-02 | 湖北大学 | The method and device that shows of augmented reality digital culture content is carried out under a kind of exception actual environment |
| CN108898604A (en) * | 2018-06-28 | 2018-11-27 | 上海连尚网络科技有限公司 | Method and apparatus for handling image |
| CN109143305A (en) * | 2018-09-30 | 2019-01-04 | 百度在线网络技术（北京）有限公司 | Automobile navigation method and device |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110595494B (en) * | 2019-09-17 | 2021-06-22 | 百度在线网络技术（北京）有限公司 | Map error determination method and device |
| CN110595494A (en) * | 2019-09-17 | 2019-12-20 | 百度在线网络技术（北京）有限公司 | Map error determination method and device |
| CN112559675B (en) * | 2019-09-25 | 2023-08-25 | 博泰车联网科技(上海)股份有限公司 | Method, apparatus and storage medium for determining results of user queries in a vehicle |
| CN112559675A (en) * | 2019-09-25 | 2021-03-26 | 上海博泰悦臻电子设备制造有限公司 | Method, apparatus, and storage medium for determining results to a user query in a vehicle |
| CN110807431A (en) * | 2019-11-06 | 2020-02-18 | 上海眼控科技股份有限公司 | Object positioning method and device, electronic equipment and storage medium |
| CN113137970B (en) * | 2020-01-20 | 2022-11-15 | 北京智慧图科技有限责任公司 | Small program AR navigation system |
| CN113137970A (en) * | 2020-01-20 | 2021-07-20 | 北京智慧图科技有限责任公司 | Applet AR navigation system |
| CN113448322A (en) * | 2020-03-26 | 2021-09-28 | 宝马股份公司 | Remote operation method and system for vehicle, storage medium, and electronic device |
| CN111698646B (en) * | 2020-06-08 | 2022-10-18 | 浙江商汤科技开发有限公司 | Positioning method and device |
| CN111698646A (en) * | 2020-06-08 | 2020-09-22 | 浙江商汤科技开发有限公司 | Positioning method and device |
| CN113779184A (en) * | 2020-06-09 | 2021-12-10 | 大众问问(北京)信息科技有限公司 | Information interaction method and device and electronic equipment |
| CN112146656B (en) * | 2020-09-03 | 2023-02-17 | 武汉大学 | Indoor navigation visualization method based on augmented reality |
| CN112146656A (en) * | 2020-09-03 | 2020-12-29 | 武汉大学 | Indoor navigation visualization method based on augmented reality |
| CN112541467A (en) * | 2020-12-21 | 2021-03-23 | 杭州光粒科技有限公司 | Information display method, device and equipment and storage medium |
| CN115164929A (en) * | 2022-08-02 | 2022-10-11 | 上海有个机器人有限公司 | Method and system for creating floor POI point on robot map and related product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109974733A (en) | POI display methods, device, terminal and medium for AR navigation | |
| US10134196B2 (en) | Mobile augmented reality system | |
| US10643103B2 (en) | Method and apparatus for representing a map element and method and apparatus for locating a vehicle/robot | |
| JP2019109245A (en) | Visual reinforcement navigation | |
| CN109947886B (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| US20150371440A1 (en) | Zero-baseline 3d map initialization | |
| US10949999B2 (en) | Location determination using street view images | |
| CN109961522B (en) | Image projection method, device, equipment and storage medium | |
| CN112101339B (en) | Map interest point information acquisition method and device, electronic equipment and storage medium | |
| CN111028358B (en) | Indoor environment augmented reality display method and device and terminal equipment | |
| CN109582880A (en) | Interest point information processing method, device, terminal and storage medium | |
| CN109931945A (en) | AR air navigation aid, device, equipment and storage medium | |
| CN111062981A (en) | Image processing method, device and storage medium | |
| CN111094895A (en) | System and method for robust self-repositioning in pre-constructed visual maps | |
| US8639023B2 (en) | Method and system for hierarchically matching images of buildings, and computer-readable recording medium | |
| CN109300143A (en) | Determination method, apparatus, equipment, storage medium and the vehicle of motion vector field | |
| WO2019095735A1 (en) | Information output method and device for vehicle | |
| CN112652062B (en) | Point cloud map construction method, device, equipment and storage medium | |
| CN109345567A (en) | Movement locus of object recognition methods, device, equipment and storage medium | |
| KR102249381B1 (en) | System for generating spatial information of mobile device using 3D image information and method therefor | |
| CN116642490A (en) | Visual positioning navigation method based on hybrid map, robot and storage medium | |
| CN114332648B (en) | Position identification method and electronic equipment | |
| CN116129087A (en) | Positioning method, visual map generation method and device thereof | |
| Park et al. | Digital map based pose improvement for outdoor Augmented Reality | |
| Porzi et al. | An automatic image-to-DEM alignment approach for annotating mountains pictures on a smartphone |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211020 Address after: 100176 101, floor 1, building 1, yard 7, Ruihe West 2nd Road, Beijing Economic and Technological Development Zone, Daxing District, Beijing Applicant after: Apollo Zhilian (Beijing) Technology Co.,Ltd. Address before: 100085 Baidu Building, 10 Shangdi Tenth Street, Haidian District, Beijing Applicant before: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) Co.,Ltd. |
|
| TA01 | Transfer of patent application right |