CN104897165A - Shot scenery-based navigation method and system thereof - Google Patents

Abstract

The present invention provides a navigation method and system based on photographed scenes, by presetting the scene navigation database in a mobile terminal, and storing identification scene image sets of each indoor environment that needs to be navigated. When the user inputs the starting position and the destination position of the navigation, firstly, according to the starting position, the identification scene image set of the corresponding indoor environment that the current user needs to use is identified. Then the user takes a real-time scene image near the starting position through a mobile terminal such as a camera device of a smart phone, and matches the real-time scene image with each logo scene image in the database through a program, thereby identifying the corresponding logo scene, and then determining from the place. Direction information from the identified scene to the target location, and output the navigation direction. Therefore, there is no need to rely on the GPS system anymore, and when the user is in an indoor environment with poor GPS signals, the method of the present invention can be used to realize the navigation of the indoor environment.

Description

Navigation method and system based on shooting scene

technical field

The invention relates to the technical field of navigation, in particular to a navigation method based on photographed scenery and a navigation system based on photographed scenery.

Background technique

In recent years, electronic digital technology has been continuously progressing and developing. Modern people use many multifunctional electronic products and devices in their daily life to make life much more convenient. One of them is the smartphone, which is becoming more and more popular due to its diverse functions and shapes. One of the functions of the multi-functional smartphone is to have a global positioning system (Global Positioning System, referred to as GPS). The basic principle of GPS is to measure the distance between the user receiver and the satellites near a certain moving distance, and then integrate the data of multiple satellites to obtain the specific position. The smart phone uses the GPS signal to allow the user to carry out mobile navigation system through the smart phone application software.

The mobile navigation system allows the user to easily display the current location and moving direction. The location can be clearly displayed on the electronic map to let the user know the location in more detail. There are currently many navigation system application software on the market, and the functions include allowing users to find the location of a certain address, and providing users with the most suitable driving route and method, such as: walking, taking a bus or subway , or by car for the convenience of the user.

However, the current GPS technology is only limited to outdoor use. For indoor spaces, such as buildings, shopping malls, subway stations, etc., the signal reception of GPS will become weak or even unacceptable. One of the reasons is that the mobile phone cannot accurately receive GPS signals due to the existence of many electronic interferences in the indoor space. This limitation of GPS makes it impossible for the user to effectively find the desired exit in the building, especially in the subway station. When the user gets off the subway, due to the large number of exits, the user cannot find the desired exit without a GPS signal. You will need to look up landmarks or ask passers-by. It will waste a lot of time for the user to find the correct exit to reach the destination.

Contents of the invention

Aiming at the problem that the existing navigation technology relies on GPS and cannot navigate when the GPS signal receives interference, the present invention proposes a navigation method based on shooting scenes, which does not depend on GPS technology and can be used when the GPS signal is received. In the event of interference, the user is provided with indoor navigation stored in the database.

A navigation method based on shooting scenes, comprising the following steps:

Obtain the starting position and destination position;

According to the starting position, the identification scene image set corresponding to the starting position is obtained from the preset scene navigation database, wherein the identification scene image set contains corresponding identification scene images, and each of the identification scene images location information;

Acquire the real-time scene image captured by the mobile terminal, and match the real-time scene image with each identified scene image in the set of identified scene images. If the matching is successful, then according to the position information of the identified scene and the target location, obtain from Direction information from the identified scene to the target location;

Outputting the navigation direction according to the direction information from the identified scene to the target position.

Aiming at the problem that the existing navigation technology relies on GPS and cannot navigate when the GPS signal is interfered, the present invention proposes a navigation system based on photographed scenery corresponding to the above method.

A navigation system based on shooting scenes, comprising:

input module, used to obtain the starting position and the destination position;

A search module, configured to acquire a set of identified scene images corresponding to the initial position from a preset scene navigation database according to the initial position, wherein the set of identified scene images contains corresponding identified scene images, and The position information of each said identified scene;

The direction calculation module is used to acquire the real-time scene image captured by the mobile terminal, and match the real-time scene image with each identified scene image in the set of identified scene images. If the matching is successful, then according to the position information of the identified scene and the Describe the target position, and obtain the direction information from the identified scene to the target position;

An output module, configured to output the navigation direction according to the direction information from the identified scene to the target location.

In the navigation method and system based on photographed scenes of the present invention, the scene navigation database is preset in the mobile terminal to save the identification scene image sets of each indoor environment that needs to be navigated. When the user inputs the starting position and the destination position of the navigation, firstly, according to the starting position, the identification scene image set of the corresponding indoor environment that the current user needs to use is identified. Then the user takes a real-time scene image near the starting position through a camera of a mobile terminal such as a smart phone. After receiving the real-time scene image, the program identifies the corresponding landmark scene by matching the real-time scene image with each logo scene image, and determines the location from the logo scene according to the position information of the logo scene and the target location. Identify the direction information from the scene to the target position, and output the navigation direction. Therefore, there is no need to rely on the GPS system anymore, and when the user is in an indoor environment with poor GPS signals, the method of the present invention can be used to realize the navigation of the indoor environment. The present invention realizes indoor navigation by identifying natural features, can be applied to the exit of a destination in a subway station, can be used on a smart phone, so that users can carry it with them, and when getting off a subway train, it can be used in indoor buildings with weak GPS signals. Wuli manufactures indoor positional navigation systems to allow users to reach their destinations smoothly and in a time-saving manner.

Description of drawings

Fig. 1 is a schematic flow chart of a navigation method based on shooting a scene in the present invention;

Fig. 2 is a schematic diagram of data in the scene navigation database of the present invention based on the navigation method of photographing the scene;

3 is a schematic diagram of a real-time scene image captured by a mobile terminal in the navigation method based on shooting a scene in the present invention;

Fig. 4 is a schematic diagram of a mobile terminal displaying a direction indicator in the navigation method based on shooting a scene in the present invention;

Fig. 5 is a schematic structural diagram of the navigation system based on photographed scenes of the present invention.

Detailed ways

Please refer to FIG. 1 . FIG. 1 is a schematic flow chart of the navigation method based on shooting scenes in the present invention.

The navigation method based on the photographed scene comprises the following steps:

S101, acquiring a starting position and a target position;

S102. Acquire a set of identified scene images corresponding to the initial position from a preset scene navigation database according to the initial position, wherein the set of identified scene images contains corresponding identified scene images, and each of the Identify the location information of the scene;

S103. Acquire a real-time scene image captured by the mobile terminal, and match the real-time scene image with each identified scene image in the set of identified scene images. If the matching is successful, according to the location information of the identified scene and the target location, Obtaining direction information from the identified scene to the target location;

S104. Output a navigation direction according to the direction information from the identified scene to the target location.

The navigation method based on photographed scenery of the present invention stores the identification scenery image sets of each indoor environment that needs to be navigated by presetting the scenery navigation database in the mobile terminal. When the user inputs the starting position and the destination position of the navigation, firstly, according to the starting position, the identification scene image set of the corresponding indoor environment that the current user needs to use is identified. Then the user takes a real-time scene image near the starting position through a camera of a mobile terminal such as a smart phone. After receiving the real-time scene image, the program identifies the corresponding landmark scene by matching the real-time scene image with each logo scene image, and determines the location from the logo scene according to the position information of the logo scene and the target location. Identify the direction information from the scene to the target position, and output the navigation direction. Therefore, there is no need to rely on the GPS system anymore, and when the user is in an indoor environment with poor GPS signals, the method of the present invention can be used to realize the navigation of the indoor environment. The present invention realizes indoor navigation by identifying natural features, can be applied to the exit of a destination in a subway station, can be used on a smart phone, so that users can carry it with them, and when getting off a subway train, it can be used in indoor buildings with weak GPS signals. Wuli manufactures indoor positional navigation systems to allow users to reach their destinations smoothly and in a time-saving manner.

Wherein, for step S101, a start location and a destination location are acquired. The starting location refers to the location where the user currently needs indoor navigation, such as a certain station, a certain subway station, or a certain shopping mall or building. The destination location usually refers to an address near the starting location, for example, a hotel near a certain station.

In step S102, according to the starting position, a set of identified scene images corresponding to the starting position is acquired from a preset scene navigation database.

The scene navigation database is a database for indoor navigation stored locally in the mobile terminal, which stores each starting position that may need to be navigated, a set of identification scene images of the starting position, and buildings or addresses near the starting position name for the user to select as the destination.

For example, for a station, the name of the station (starting position) is generally saved, a set of images of the logo scene in the station, and commonly used addresses such as hotels and shopping malls near the station, or the location information of each exit of the station, and all Describe the direction information of each exit to the possible destination location.

As shown in FIG. 2 , the scene navigation database includes a comparison table of exits and destination locations in the direction of subway stations and related exits. The exit and destination position comparison table includes corresponding subway station name data; building data near the subway station; data at the exit of the subway station corresponding to the nearby building; and the azimuth position of each exit to the nearby building data. For example, the user enters the name of the subway station, Orchard, and the destination, Four Seasons Hotel, and the data of Exit B with an orientation of 200° can be found by using the comparison table between the exit and the destination.

The starting position and the target position input by the user must be related data saved in the scene navigation database before navigation can be performed, if the starting position and the target position do not exist in the scene navigation database , you may need to download an update to get the relevant data.

For example, the user can update the scene navigation database on the mobile terminal in advance according to his own itinerary, and obtain the identification scene image sets corresponding to the stations requiring indoor navigation to the local scene navigation database. That is, before obtaining the starting position and the destination position, the following steps are further included: according to the input position information, search the scene navigation database preset in the server, and download the identification scene image set corresponding to the position information from the server to the mobile terminal.

The set of identified scene images includes the identified scene image of the starting position, and position information of each of the identified scenes. The identification scene can be various indoor scenes of the indoor environment where the starting position is located. For example, the starting position is a subway station A, then the identification scene image can be an escalator or ladder image in the subway station A, Images of subway name signs, images of entry or exit checkpoints, and more.

In step S103, the real-time scene image captured by the mobile terminal is acquired, and the real-time scene image is matched with each identified scene image in the set of identified scene images.

When navigating, the user can use the camera of the mobile terminal to shoot the real-time scene near the starting position, acquire the real-time scene image, and store it in the mobile storage. After receiving the real-time scene image, the program matches the real-time scene image with each identified scene image in the set of identified scene images.

As shown in FIG. 3 , the real-time scene image captured by the mobile terminal 200 can also be various indoor scenes of the indoor environment where the starting position is located, such as an escalator or ladder image 202 in a subway station, a subway name signboard Image 204, and image 206 at the entry or exit level, and so on.

During matching, it is preferable to extract features of the real-time scene image and the identified scene image for comparison. The comparison technology can use computer vision technology, for example: take out the key points (key points) by in-depth analysis of _Speeded Up Robust Feature (SURF), and then use the fast nearest neighbor approximation search function library (Fast Approximate Nearest Search Library ( _FLANN for short) method compares the real-time scene image with the identified scene image. According to the comparison result, if the matching is successful, the location information of the identified scene is obtained, or the name of the identified scene is further obtained. According to the position information of the marked scene and the target position, the direction information from the marked scene to the target position can be obtained.

For the application scenario of the station, the direction information from the identified scene to the destination location may include two parts, one part is the direction information from the identified scene to the corresponding station exit, and the other part is the direction information from the station exit to the station exit. State the direction information of the target location.

The step of obtaining direction information from the identified scene to the target location includes:

According to the destination location, acquiring station exit information corresponding to the destination location, and direction information from the station exit to the destination location;

According to the location information of the marked scene and the station exit information, direction information from the marked scene to the station exit is acquired.

Wherein, the step of obtaining the direction information from the station exit to the destination location can be realized in the following manner: according to the location information of the station exit, search for a pre-saved comparison table between the exit and the destination location, and obtain the direction information from the station exit to the destination location. State the direction information of the target location.

The step of obtaining the direction information from the marked scene to the station exit can be realized in the following manner: according to the position information of the marked scene and the position information of the station exit, calculate the distance from the marked scene to the station exit direction information.

Wherein, because the direction information from the identified scene to the station exit is indoors, it can be calculated based on the position information of the marked scene, the position information of the station exit, and the station electronic map of the station. A navigation route from the identified scene to the station exit; then, according to the navigation route, obtain direction information from the identified scene to the station exit.

It is also possible to pre-save the direction information of each identified scene relative to each station exit in the scene navigation database, and then find the direction information from the marked scene to the corresponding station exit according to the station exit information obtained in step S102 .

In this step, the program can estimate and calculate the starting position based on the orientation at the exit of the station. The calculation uses the accelerometer and gyroscope included in the smartphone to calculate the orientation.

In step S104, a navigation direction is output according to the direction information from the identified scene to the target location.

Usually, the navigation direction is output through the display screen or voice of the mobile terminal. If it is displayed, a corresponding direction indicator can be generated according to the direction information from the identified scene to the destination position; The terminal displays the direction indication sign.

For example, the user's corresponding azimuth position is provided through the rotating arrow on the display screen of the smart phone. As shown in FIG. Provide the user's corresponding indoor orientation position.

If voice prompting is used, the direction information can be broadcast by voice.

The invention provides a method for indoor navigation by identifying the natural features of the scenery in a subway station. The invention can be used on a smart phone without a GPS signal, allowing the user to conveniently find the exit after getting off the subway indoors to reach the destination. The method uses a camera in a smart phone to record a real picture, and then compares it with a database image in the smart phone to provide the user's current location for indoor navigation.

The system includes a program that uses the method mentioned above to update the user's location indoors in real time. In the database, each image file stores unique position data, and the destination data input by the user also stores unique direction data.

In a word, the present invention provides an innovative and convenient way for the indoor navigation system to allow smart phone users to reach their destination smoothly and time-savingly after getting off the subway train under the weak GPS signal.

Please refer to FIG. 5 . FIG. 5 is a schematic structural diagram of a navigation system based on photographed scenes of the present invention.

The navigation system based on the shooting scene includes:

The input module 11 is used to obtain the starting position and the target position;

A search module 12, configured to acquire a set of identified scene images corresponding to the initial position from a preset scene navigation database according to the initial position, wherein the set of identified scene images contains corresponding identified scene images, and the location information of each of the identified sceneries;

The direction calculation module 13 is used to obtain the real-time scene image taken by the mobile terminal, and match the real-time scene image with each identification scene image in the identification scene image set. If the matching is successful, then according to the position information of the identification scene and For the target location, obtain direction information from the identified scene to the target location;

The output module 14 is configured to output the navigation direction according to the direction information from the identified scene to the target location.

In the navigation system based on photographed scenes of the present invention, by presetting the scene navigation database in the mobile terminal, the set of identification scene images of each indoor environment that needs to be navigated is stored. When the user inputs the starting position and the destination position of the navigation, firstly, according to the starting position, the identification scene image set of the corresponding indoor environment that the current user needs to use is identified. Then the user takes a real-time scene image near the starting position through a camera of a mobile terminal such as a smart phone. After receiving the real-time scene image, the program identifies the corresponding landmark scene by matching the real-time scene image with each logo scene image, and determines the location from the logo scene according to the position information of the logo scene and the target location. Identify the direction information from the scene to the target position, and output the navigation direction. Therefore, there is no need to rely on the GPS system anymore, and when the user is in an indoor environment with poor GPS signals, the method of the present invention can be used to realize the navigation of the indoor environment. The present invention realizes indoor navigation by identifying natural features, can be applied to the exit of a destination in a subway station, can be used on a smart phone, so that users can carry it with them, and when getting off a subway train, it can be used in indoor buildings with weak GPS signals. Wuli manufactures indoor positional navigation systems to allow users to reach their destinations smoothly and in a time-saving manner.

Wherein, the input module 11 acquires a starting location and a destination location. The starting location refers to the location where the user currently needs indoor navigation, such as a certain station, a certain subway station, or a certain shopping mall or building. The destination location usually refers to an address near the starting location, for example, a hotel near a certain station.

The searching module 12 obtains the set of identified scene images corresponding to the starting position from a preset scene navigation database according to the starting position.

The scene navigation database is a database for indoor navigation stored locally in the mobile terminal, which stores each starting position that may need to be navigated, a set of identification scene images of the starting position, and buildings or addresses near the starting position name for the user to select as the destination.

For example, for a station, the name of the station (starting position) is generally saved, a set of images of the logo scene in the station, and commonly used addresses such as hotels and shopping malls near the station, or the location information of each exit of the station, and all Describe the direction information of each exit to the possible destination position.

The scene navigation database includes a comparison table of exits and destination locations in the direction of subway stations and relevant exits. The exit and destination position comparison table includes corresponding subway station name data; building data near the subway station; data at the exit of the subway station corresponding to the nearby building; and the azimuth position of each exit to the nearby building data.

The starting position and the target position input by the user must be related data saved in the scene navigation database before navigation can be performed, if the starting position and the target position do not exist in the scene navigation database , you may need to download an update to get the relevant data.

Users can update the scenery navigation database on the mobile terminal according to their own itinerary in advance when there is a network, and obtain the identification scenery image sets corresponding to the stations that need indoor navigation to the local scenery navigation database. Therefore, the navigation system based on photographed scenery also includes: a download module, configured to search a preset scenery navigation database in the server according to the input location information, and download the identification scenery image corresponding to the location information from the server set to the mobile terminal.

The set of identified scene images includes the identified scene image of the starting position, and position information of each of the identified scenes. The identification scene can be various indoor scenes of the indoor environment where the starting position is located. For example, the starting position is a subway station A, then the identification scene image can be an escalator or ladder image in the subway station A, Images of subway name signs, images of entry or exit checkpoints, and more.

The direction calculation module 13 acquires the real-time scene image captured by the mobile terminal, and matches the real-time scene image with each identified scene image in the set of identified scene images.

When navigating, the user can use the camera of the mobile terminal to shoot the real-time scene near the starting position, acquire the real-time scene image, and store it in the mobile storage. After receiving the real-time scene image, the program matches the real-time scene image with each identified scene image in the set of identified scene images.

When matching, the direction calculation module 13 preferably extracts the features of the real-time scene image and the identified scene image for comparison. The comparison technology can use computer vision technology, for example: take out the keypoints by in-depth analysis of the Speeded Up Robust Feature (SURF), and then use the Fast Nearest Neighbor Approximation Search Function Library (Fast Approximate NearestSearch Library (FLANN for short) method compares the real-time scene image with the marked scene image. According to the comparison result, if the matching is successful, the location information of the identified scene is obtained, or the name of the identified scene is further obtained. According to the position information of the marked scene and the target position, the direction information from the marked scene to the target position can be obtained.

For the application scenario of the station, the direction information from the identified scene to the destination location may include two parts, one part is the direction information from the identified scene to the corresponding station exit, and the other part is the direction information from the station exit to the station exit. State the direction information of the target location.

Then described direction computing module 13 comprises:

A destination location direction acquisition module, configured to acquire station exit information corresponding to the destination location and direction information from the station exit to the destination location according to the destination location;

And, an exit direction acquiring module, configured to acquire direction information from the marked scene to the station exit according to the position information of the marked scene and the station exit information.

Wherein, the module for obtaining the direction of the destination location searches a pre-saved comparison table between the exit and the destination location according to the location information of the station exit, and obtains the direction information from the station exit to the destination location.

The exit direction acquiring module calculates direction information from the identified scene to the station exit according to the position information of the marked scene and the position information of the station exit.

Since the direction information from the sign to the station exit is indoors, it can be calculated from the sign according to the position information of the sign scene and the position information of the station exit, as well as the station electronic map of the station. A navigation route from the scene to the station exit; and then obtain direction information from the identified scene to the station exit according to the navigation route.

It is also possible to pre-store the direction information of each identified scene relative to each station exit in the scene navigation database, and then search for the direction information of the marked scene to the corresponding station exit according to the obtained station exit information.

The direction calculation module 13 can estimate and calculate the starting position based on the orientation at the exit of the station. The calculation uses the accelerometer and gyroscope included in the smartphone to calculate the orientation.

The output module 14 outputs the navigation direction according to the direction information from the identified scene to the target location.

The output module 14 usually outputs the navigation direction through the display screen or voice of the mobile terminal. If it is displayed, it can generate a corresponding direction indicator according to the direction information from the identified scene to the target location; Then display the direction indication sign through the mobile terminal. For example, the rotating arrows on the display screen of the smart phone can provide the user's corresponding bearing position. If voice prompting is used, the direction information can be broadcast by voice.

Those of ordinary skill in the art can understand that the implementation of all or part of the processes in the above embodiments and the corresponding system and the music player can be completed by instructing related hardware through a computer program, and the program can be stored in a computer. In the readable storage medium, when the program is executed, it may include the processes of the above-mentioned embodiments. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM) or a random access memory (Random Access Memory, RAM), etc.

The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but should not be construed as limiting the patent scope of the present invention. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention should be based on the appended claims.

Claims (10)

1. A navigation method based on photographing scenery, is characterized in that, comprises the following steps: Obtain the starting position and destination position; According to the starting position, the identification scene image set corresponding to the starting position is obtained from the preset scene navigation database, wherein the identification scene image set contains corresponding identification scene images, and each of the identification scene images location information; Acquire the real-time scene image captured by the mobile terminal, and match the real-time scene image with each identified scene image in the set of identified scene images. If the matching is successful, then according to the position information of the identified scene and the target location, obtain from Direction information from the identified scene to the target location; Outputting the navigation direction according to the direction information from the identified scene to the target position. 2. The navigation method based on photographed scenery as claimed in claim 1, characterized in that, according to the position information of the identified scenery and the destination location, the step of obtaining direction information from the identified scenery to the destination location include: According to the destination location, acquiring station exit information corresponding to the destination location, and direction information from the station exit to the destination location; According to the location information of the marked scene and the station exit information, direction information from the marked scene to the station exit is acquired. 3. The navigation method based on shooting scenery as claimed in claim 2, wherein the step of obtaining the direction information from the station exit to the destination location comprises: According to the location information of the station exit, look up a pre-saved comparison table between the exit and the destination location, and obtain the direction information from the station exit to the destination location. 4. The navigation method based on shooting scenery as claimed in claim 2, wherein the step of obtaining the direction information from the identification scenery to the exit of the station comprises: According to the position information of the marked scene and the position information of the station exit, calculate the direction information from the marked scene to the station exit. 5. The navigation method based on shooting scenery according to any one of claims 1 to 4, wherein, according to the direction information from the identified scenery to the destination position, the step of outputting the navigation direction comprises: generating a corresponding direction indicator according to the direction information from the identified scene to the target location; The direction indicator is displayed by the mobile terminal. 6. A navigation system based on shooting scenery, characterized in that, comprising: input module, used to obtain the starting position and the destination position; A search module, configured to acquire a set of identified scene images corresponding to the initial position from a preset scene navigation database according to the initial position, wherein the set of identified scene images contains corresponding identified scene images, and The position information of each said identified scene; The direction calculation module is used to acquire the real-time scene image captured by the mobile terminal, and match the real-time scene image with each identified scene image in the set of identified scene images. If the matching is successful, then according to the position information of the identified scene and the Describe the target position, and obtain the direction information from the identified scene to the target position; An output module, configured to output the navigation direction according to the direction information from the identified scene to the target location. 7. The navigation system based on the shooting scene as claimed in claim 6, wherein the direction calculation module comprises: A destination location direction acquisition module, configured to acquire station exit information corresponding to the destination location and direction information from the station exit to the destination location according to the destination location; An exit direction acquiring module, configured to acquire direction information from the marked scene to the station exit according to the position information of the marked scene and the station exit information. 8. The navigation system based on photographed scenery as claimed in claim 7, wherein said destination location direction acquisition module searches a pre-saved exit and destination location comparison table according to the location information of said station exit, and obtains said destination location. Direction information from the station exit to the destination. 9. The navigation system based on shooting scenery as claimed in claim 7, wherein the exit direction acquisition module calculates the distance between the identification scenery and the location information of the station exit according to the position information of the identification scenery and the location information of the station exit. Directions to station exits. 10. The navigation system based on the photographed scene according to any one of claims 6 to 9, wherein the output module generates a corresponding direction indicator according to the direction information from the identified scene to the destination position , using the mobile terminal to display the direction indication sign.