KR20180096191A - Geographic informatization method for image - Google Patents

Abstract

The present invention relates to a geographic informatization method for an image. The present invention provides a method of tracking a feature point for each frame of an image, correcting a matching error, analyzing a rotation point, acquiring a movement path, and performing geographic informatization on the image, and a computer-readable recording medium which stores a program for realizing the method. To this end, the present invention provides a geographic informatization method for an image applied to a geographic informatization apparatus for an image. The method may include a step of tracking a feature point for each frame of an image; a step of correcting a matching error; a step of analyzing a rotation point; and a step of acquiring an output path.

Description

Geographic informatization method for image

[0001] The present invention relates to a method of geographically informing an image, and more particularly, to a method of geographically displaying an image photographed by a personal portable terminal (e.g., a smart phone) and a computer readable by a computer To a recording medium on which the recording medium can be recorded.

In the following description, a smart phone will be described as an example of a personal portable terminal.

Currently, video map and walking navigation services are provided to users. For example, two services are available: 'Alleysguide' and 'Alleys Video Map'. 'Alice Guide' was launched in early June, 2016, collecting video map data as a user-engaging crowdsourcing method. 'Alice Video Map' is a video-based navigation service for walkers and travelers. Beta was opened on mobile and on the web at the end of July 2916.

The company, which provides the Alice Guide, plans to continue collecting video map data from many smartphone users in the future, and also aims to expand its business by providing O2O services and navigation services in the form of B2B to applications.

The company that provides 'Alice Wonder Lab' collects video map data by crowdsourcing method and uses it as road guide and travel guide data, and sells the data to various VR / AR (Virtual / Augmented Reality) I want to connect the world to the future in the future.

Meanwhile, demand for video map data is increasing.

For example, the size of the virtual / augmented reality market is increasing. According to Digital-Capital, the size of the VR / AR market is expected to grow at a very rapid pace, with the size of the related market reaching $ 15 billion by 2020.

In addition, VR / AR content is expected to be used in a wide variety of fields, including video games, real estate, and engineering (source: Goldman Sachs).

The necessity of video map data in the development of VR / AR contents is very large. Especially, in the augmented reality (AR), it is necessary to implement the 3D map for the surrounding terrain because the virtual space is applied to the real terrain.

As another example, the 3D mapping market is growing in size. According to a survey by Markets and Markets, the size of the 3D map market is expected to grow 55% annually, and by 2020 the market will reach $ 17 billion.

Since the need for video map data is imperative in 3D map implementation, the demand for video map data will increase significantly as the market grows.

As described above, the VR / AR market and the 3D map market are greatly increased in size, and demand for video map data is also increasing. As a result, companies around the world, such as Google and Nokia, are also gearing up to collect video map data.

The most essential thing in constructing the video map data is the geographical information of the image (geographical information refers to correspondence of population, feature, and other information to specific geographical points). That is, if the geographical information about where the image is photographed on the earth is not matched with the photographed image, it has no value as map data. Therefore, in order to construct the video map data, it is necessary to process the 'geographical information of the image'.

However, the current collection of map data remains at a primitive level. For example, a typical Google map data service is Street View. In order to match the photographed image with the photographed spot and provide the map data to the users, Google takes expensive GPS equipment, moves it by car, and shoots all over the world. Also, in places where you can not move by car, people shoot the equipment directly.

However, despite shooting 20kg of equipment worth $ 150,000 directly, photographing all over the world and putting a lot of manpower, we have not been able to shoot half of the Earth's land yet.

On the other hand, many people all over the world shoot footage of their travels on their smartphone for memories, records, and publicity. More than 21 million travel videos have been uploaded to Youtube, and more than 500 video map data have already been collected through the Alice Wonder Lab application launched in 2015.

Thus, crowdsourcing without having to go around the world and capture the possibility of collecting video map data based on images from many smartphone users around the world.

Also, if video map data can be collected through crowd sourcing, a considerable number of manpower resources and costs for collecting the current video map data can be greatly reduced, and map data can be updated at a high speed This enables more effective data collection.

However, in order to construct the video map data through crowd sourcing, it is necessary to match the geographical information about the photographed image to where the corresponding image is photographed.

The smartphone has a built-in GPS sensor that captures images and can receive the GPS data of the image, but it is difficult to obtain reliable information because of the error of the GPS data itself.

Therefore, in order to construct video map data through crowd sourcing, a technique of matching a map with only GPS information and video sent from a smart phone is required, and it is an object of the present invention to meet such a demand.

Accordingly, the present invention provides a method for tracking feature points for each frame of an image, correcting a matching error, analyzing a rotation point, acquiring a movement path to geo-inform the image, And a computer-readable recording medium on which the program is recorded.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention which are not mentioned can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. It will also be readily apparent that the objects and advantages of the invention may be realized and attained by means of the instrumentalities and combinations particularly pointed out in the appended claims.

According to another aspect of the present invention, there is provided a method of geographically informing an image applied to an apparatus for geographical information of an image, the method comprising the steps of: tracking minutiae of each frame of the image; Correcting a matching error; Analyzing the rotation point; And acquiring a movement path.

According to an aspect of the present invention, there is provided a geographical information system including a processor, the method comprising the steps of: tracking minutiae of each frame of an image; Correcting a matching error; Analyzing the rotation point; And a program for realizing a step of acquiring a movement path.

According to the present invention as described above, it is possible to efficiently track geographical information by tracking minutiae of each frame of an image, correcting a matching error, analyzing a rotation point, It is effective.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram showing a method of geographical information of an image according to the present invention;
2 is a diagram for explaining an image sampling and feature point tracking process according to the present invention.
FIG. 3 is a diagram showing a feature point tracking vector at the time of straight forward and rotation according to the present invention,
4 is a diagram for explaining a correction process of a matching error according to the present invention,
5 is a diagram for explaining a rotation point analysis process according to the present invention,
FIG. 6 is a view for explaining a moving route acquisition process according to the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings, It can be easily carried out. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

And throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between. Also, when a component is referred to as " comprising "or" comprising ", it does not exclude other components unless specifically stated to the contrary . In addition, in the description of the entire specification, it should be understood that the description of some elements in a singular form does not limit the present invention, and that a plurality of the constituent elements may be formed.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In the present invention, a method for map-matching an image photographed with a smartphone to a map for 'geographical informationization of a smartphone image' has been developed. The method of geographical information of the image can be applied to the geographical information apparatus of the image.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram illustrating a method of geographical information of an image according to the present invention.

As shown in FIG. 1, the method of geographical information according to the present invention includes a process of tracking minutiae of each frame of an image, a process of correcting a matching error, a process of analyzing a rotation point, And acquiring a movement path. Each of these processes will be described in detail as follows.

FIG. 2 is a view for explaining an image sampling and feature point tracking process according to the present invention. FIG. 3 is a diagram illustrating a feature point tracking vector at the time of straight forward and rotation according to the present invention. And the case of rotation.

Referring to FIG. 2, a process of tracking feature points for each frame will be described below.

The image captured by the smart phone and the corresponding image are captured, and the GPS data obtained by the GPS sensor of the smartphone is the input of the corresponding algorithm. After receiving the captured images, the images were sampled at regular intervals for image analysis and the feature points were tracked using the SURF (Speed Up Robust Features) algorithm of 'MATLAB'.

After tracing the feature points, we traced the movement of the feature points as the frame changes and extracted the motion vectors by the blob detecting method.

It can be seen from the characteristic point tracking result of FIG. 3 that the shape of the motion vector at the time of straightening and rotation at the time of imaging is significantly different. In other words, while the vectors converge to the vanishing point on the screen at the time of going straight, most of the points move in one direction, that is, the rotation direction at the time of rotation.

4 is a diagram for explaining a correction process of a matching error according to the present invention.

When the tendency of the vectors at the time of rotation is grasped, the turning point can be tracked through image analysis. However, at the time of rotation, as shown in FIG. 3, a matching error occurs across the screen, and an error correction process is further required for accurate image analysis.

In order to correct the error, a vector having a size of 10 times or more the size of the average vector is removed by specifying it as a wrong matching vector, and the data other than the data showing the greatest tendency is removed by applying the 'MATLAB LANSAC algorithm' Method, the matching error is effectively corrected as shown in FIG.

5 is a view for explaining a rotation point analysis process according to the present invention.

In order to effectively understand the rotation point through image analysis, various foot images were shot, and feature point tracking vectors for each frame were extracted, and numerical values for each frame were shown in a graph.

In the graph of FIG. 5, the blue graph represents the average vector size of the extracted vectors for each frame, and the red graph represents the integrated value of the vectors.

The minimum reference value that can include all the rotation points is derived by analyzing the images taken at various places and using the values of the graph to grasp the values of the actual points on the graph. As a result, the minimum reference value was obtained experimentally at 3.5, and a point having a value greater than or equal to the reference value was defined as the rotation point.

FIG. 6 is a view for explaining a moving route acquisition process according to the present invention.

After the information of the frame in which the rotation point appears is obtained through the image analysis, it is necessary to finally understand the movement route of the walk photographer by reflecting such information.

As shown in FIG. 6, an application called 'GPS Project' is made using 'Android' and when receiving GPS data by using a smartphone, the data can be displayed at predetermined time intervals Respectively.

Since GPS data often generates errors, we have developed an algorithm that removes outliers for points that are spaced by more than the maximum distance that can be moved on foot.

After the point of appearance of the rotation point is found through the graph analysis of FIG. 5, the GPS data corresponding to the point of time is designated as the rotation point, and the rotation points are connected by a straight line to complete the path.

The scale of the obtained path was adjusted according to the size of the actual map and matched. As a result, the agreement rate was 98% or more with the actual path.

Meanwhile, the map matching technology of the smartphone image developed in the present invention is a technology which is intended to adopt a dual camera of a smartphone 5 years later.

Currently, crowd sourcing is not used, and high-priced equipment is shot directly to all parts of the world and map data is obtained because the quality of images taken from any user drops significantly.

The dual camera has a wide viewing angle, sharper, faster autofocus. Most importantly, dual cameras measure 3D depth, making it very easy to create 3D maps.

The smartphone dual camera market is growing rapidly, and smart phones such as LG and Huawei are already equipped with dual cameras.

After the development of the smartphone dual camera technology is completed in the next five years, the 3D map can be easily realized even by the image obtained through the crowd sourcing, so that it is possible to efficiently collect the high quality video map data. Value will also rise.

In addition, in the present invention, an algorithm for matching the geographical information to the collected video data is developed, thereby developing a core technology that will be the basis for the process of collecting video map data through crowd sourcing. If the technology is further developed to more effectively perform image analysis and GPS data processing, the value of the corresponding technology and the value of the video map data collected by the corresponding technology in accordance with the change of the smartphone market, Is expected to rise sharply.

Meanwhile, the method of geographical information according to the present invention as described above can be implemented in the form of a program command which can be executed through various computer means, and can be recorded in a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. The medium may be a transmission medium such as an optical or metal line, a wave guide, or the like, including a carrier wave for transmitting a signal designating a program command, a data structure, or the like. Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, Various permutations, modifications and variations are possible without departing from the spirit of the invention.

Therefore, the scope of the present invention should not be construed as being limited to the embodiments described, but should be determined by the scope of the appended claims, as well as the appended claims.

Claims (2)

A method of geographical information of an image applied to a geographical information apparatus of an image,
Tracking feature points for each frame of an image;
Correcting a matching error;
Analyzing the rotation point; And
The step of acquiring the movement route
The method comprising the steps of:
A geographical information system for an image having a processor,
Tracking feature points for each frame of an image;
Correcting a matching error;
Analyzing the rotation point; And
The step of acquiring the movement route
A computer-readable recording medium having recorded thereon a program for realizing a computer-readable recording medium.

Images