KR100679864B1 - Cellular phone capable of displaying geographic information and a method thereof - Google Patents

Abstract

A mobile communication terminal capable of displaying geographic information and a method thereof are provided to display the geographic information on a peripheral image so that a user can easily grasp information about a strange place. A mobile communication terminal comprises the followings: a photographing unit(11) for photographing peripheral images; a location confirming unit(12) for confirming a current location by receiving GPS(Global Positioning System) information; a calculating unit(13) for calculating distance between at least one subject among the photographed peripheral images, and the current location and direction of the photographing unit(11); a search unit(14) receiving GIS(Geographical Information System) information, and searching information about the subject on GIS information based on the calculated distance and direction; and a display unit(15) for displaying the subject information searched in the photographed subject.

Description

A mobile communication terminal capable of displaying local information and a method of displaying the local information {Cellular phone capable of displaying geographic information and a method}

1 is a block diagram schematically illustrating a mobile communication terminal according to the present invention;

2 is a view for explaining the relationship between the image plane projected by the projection matrix on the plane;

3 is a flowchart illustrating an embodiment of a method for providing region information by a mobile communication terminal of FIG. 1;

4 is a flowchart illustrating another embodiment of a method for providing region information by a mobile communication terminal of FIG. 1;

FIG. 5 is a flowchart illustrating still another embodiment of a method for providing region information by the mobile communication terminal of FIG. 1.

Explanation of symbols on the main parts of the drawings

11: imaging unit 12: positioning unit

13: calculation unit 14: search unit

15: display unit 16: input unit

17: GIS information storage unit 18: voice recognition unit

The present invention relates to a mobile communication terminal capable of displaying local information and a method of displaying the local information. More particularly, the present invention relates to a mobile communication terminal capable of displaying local information on a screen using GPS information and GIS information and a local area. The information display method.

In general, the GPS (Global Positioning System) is a global radio navigation system using satellites developed by the US Department of Defense, and officially started operation in 1995. GPS can accurately measure 3D location, altitude, and time, provide 24/7 continuous service worldwide, and is resistant to weather and interference and can be used indefinitely.

The GPS is equipped with a cesium atomic clock and a rubidium clock, respectively, to maintain precise time, and by generating a unique code of PRN (Pesudo Random Noise) for each satellite, satellites can be easily distinguished according to codes.

The satellite group consists of a total of 24 satellites, including 21 operating satellites and 3 spare satellites, rotating around the Earth at an orbital period of about 12 hours. GPS uses the principle of triangulation, which determines the position of an unknown point by measuring the length of two sides between the points you want to know. The distance from the satellite to the receiver is obtained by measuring the time difference between the generation point and the reception point of the code signal generated by each satellite, and then multiplying the measured value by the speed of light.

On the other hand, the Geographic Information System (GIS) creates and manages maps and geographic information used in the past in the form of printed materials using a computer, and collects, analyzes, and processes data based on the geographic information obtained from them. Refers to a comprehensive information system designed for application in all areas of being. Such GIS is divided into land information system, urban information system, drawing automation, and facility management, but it is used in a wide range of fields such as land, resources, environment, city, ocean, fisheries, military, transportation, communication, water and sewage.

Recently, GPS has been widely commercialized as a navigation system that is combined with a Geographic Information System (GIS) to detect and confirm its location and to easily find the distance and the shortest distance from the identified location to the desired location.

Such a navigation system is generally implemented in a form in which the GIS information is displayed on the screen in the form of a map, and the user is led to the shortest distance to the destination while displaying the current location of the user on the screen. However, the implementation of such a navigation system is not easy to recognize the destination for the elderly or children who are not used to seeing the map, and when the user's current location is the first visit, the surrounding landscape and the map easily match each other. The difficulty is further increased.

The present invention has been made to solve the above problems, the mobile communication terminal and the local information display method for the user to easily recognize the destination by combining and displaying the local information in the surrounding landscape of the place where the user is located The purpose is to provide.

Embodiments of the mobile communication terminal according to the present invention for achieving the above object, the imaging unit for imaging the surrounding image; A positioning unit configured to receive GPS (Global Positioning System) information and confirm a current position; A calculation unit that calculates a distance between at least one subject in the surrounding image to be captured and the current position, and a direction of the imaging unit; A search unit for receiving Geographic Information System (GIS) information and searching for information of the subject on the GIS information based on the calculated distance and direction; And a display unit which displays information on the subject found on the subject to be photographed.

Preferably, the mobile communication terminal further includes an input unit for inputting a destination to be searched, wherein the searcher searches for GIS information corresponding to the destination from the GIS information, and the display unit is configured to display information of the subject. If the GIS information coincides with the GIS information, an identification mark is displayed on the subject to be photographed. The mobile communication terminal may further include a voice recognition unit that analyzes and recognizes a voice corresponding to the destination input through the input unit.

Preferably, the mobile communication terminal further comprises a GIS information storage unit for storing the received GIS information.

On the other hand, the mobile communication terminal according to the present invention, the step of receiving the GPS information to determine the current position; Imaging the surrounding image; Calculating a distance between at least one of the subjects in the peripheral image to be photographed and the current position, and a direction of an image capturing unit that picks up the peripheral image; Receiving GIS information and retrieving information of the subject on the GIS information based on the calculated distance and direction; And displaying information on the subject found on the photographed subject.

Preferably, the method for displaying local information includes inputting a destination to be searched, retrieving GIS information corresponding to the destination from the GIS information, and matching information of the subject with GIS information of the destination. And identifying the subject to be photographed in the case.

Another embodiment of a mobile communication terminal according to the present invention for achieving the above object, the image pickup unit for imaging the surrounding image; An input unit for inputting a destination to search for; A location checking unit for receiving GPS information and checking a current location; A calculation unit that calculates a distance between at least one subject in the surrounding image to be captured and the current position, and a direction of the imaging unit; A search unit for receiving GIS information, searching for information of the subject on the GIS information based on the calculated distance and direction, and searching for GIS information corresponding to the destination from the GIS information; And a display unit for identifying the subject to be imaged when the information of the subject matches the GIS information of the destination.

In addition, the mobile communication terminal according to another embodiment of the present invention comprises the steps of: inputting a destination to search; Receiving a GPS information and checking a current position; Imaging the surrounding image; Calculating a distance between at least one of the subjects in the peripheral image to be photographed and the current position, and a direction of an image capturing unit that picks up the peripheral image; Receiving GIS information and retrieving GIS information corresponding to the destination from the GIS information; Retrieving information of the subject on the GIS information based on the calculated distance and the direction; And displaying identification marks on the photographed subjects when the subject information matches the GIS information of the destination.

Accordingly, the mobile communication terminal according to the present invention combines and displays local information on the surrounding image displayed on the screen, so that the user can accurately recognize the current geographical location and easily find the destination.

Hereinafter, a mobile communication terminal and a method of displaying local information thereof according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically illustrating a mobile communication terminal according to the present invention. Referring to the drawings, a mobile communication terminal according to the present invention includes an imaging unit 11, a positioning unit 12, a calculating unit 13, a searching unit 14, a display unit 15, an input unit 16, and GIS information. A storage unit 17, and a speech recognition unit 18.

The imaging unit 11 is for capturing an image of a surrounding, and includes a lens (not shown), a CCD (Charged Coupled Device) (not shown) that converts an optical image of a subject photographed through the lens into an electrical signal, and the like. Is implemented. That is, the mobile communication terminal according to the present invention is based on the technology of augmented reality that superimposes information on a real world image. Here, augmented reality technology refers to a technology derived from virtual reality technology that improves awareness of the real world, and excludes interaction with the real world, and only interacts in a virtual space that is built in advance. Unlike the virtual reality technology that processes, real-world information (called 'augmentation information') built in advance on the basis of real-time processing is superimposed on the real world that is actually observed through a special interface, and provides interaction with the real world. This allows the observer to make quick and convenient situational judgment on the real world observation area. The surrounding image picked up by the imaging unit 11 becomes an image for the real world observation area.

The positioning unit 12 includes a GPS receiving module (not shown) that receives GPS information from a GPS satellite, and confirms a current position based on the received GPS information. By the positioning unit 12, the longitude position and the latitude position of the place where the mobile communication terminal is located are confirmed.

The calculation unit 13 calculates the distance between at least one subject in the surrounding image picked up by the imaging unit 11 and the current position, and the direction in which the imaging unit 11 is directed toward the subject. Here, the current position refers to the position of the mobile communication terminal, and the distance between the subject and the current position is a distance between the lens and the subject of the mobile communication terminal and may be assumed as the distance between the mobile terminal and the subject. In addition, the method of calculating the distance between the subject and the current position will be described based on "accurate focal length measurement method based on the projected geometric relationship between images" (Kim Hwa-sung, Bang Ki-cheon, Kim Chang-heon). This method calculates the distance between the subject and the current position by estimating the angle of rotation of the two image planes and extracting the focal length based on the projection geometry of the image plane. However, the method of measuring the distance between the subject and the current position is not limited to the described method and may be calculated by another method.

The positions of the two images captured at the same time point are determined by the 2D projective transformation matrix M. FIG.

Here, (x, y) is arbitrary pixel coordinates of the image and pixel coordinates in the image corresponding to (x ', y'). M is a two-dimensional projective transformation matrix. The three-parameter rotation model can be expressed as follows using two viewing matrices and a camera rotation matrix R between two images.

Szeliski calculates the projection transformation matrix M in a manner that minimizes the difference in image brightness. Here, to estimate the focal length, a limited camera matrix considering only the focal length is used, and a coordinate system having the origin as the center of the image is applied. In this case, the projective transformation matrix is as follows.

The first two rows and the first two columns of the rotation matrix have the same norm and the orthogonal property is used to find the focal length. However, the focal length estimated by this method is not accurate and should be improved by using a minimization technique.

McMillan defines a camera matrix that contains all of the camera's internal camera parameters, and uses a rotation matrix that only considers the y-axis rotation. The focal length f is estimated by using the following equation to obtain the two-dimensional movement amount t ₁ of the 360-degree rotation images N required to obtain these internal and external parameters.

This method is capable of having a set of images that produce a 360 degree panorama, but not a panorama with a limited viewing angle.

Next, IIP (Image Intersection Point) is defined to identify the geometric relationship between image centers and focal length. The IIP can determine how accurate the estimated focal length is and is used for accurate focal length estimation.

FIG. 2 is a diagram for explaining the relationship of the image plane projected by the projection matrix on the plane. The figure shows a three-dimensional plane and two-dimensional planes of the time axis of two images when a camera with a fixed focal length f and a fixed projection point (COP) is rotated by θ in the plane of COP, C ₁ , C ₂ on three dimensions. Two-dimensional representation of the intersection with the image planes. Here, f ₀ , f ₁ are estimated values of f for each image.

Through the projection transformation matrix, image I ₁ is projected as I ₁ 'on the image plane of I ₂ . C ₁ , C ₂ means the center of each image, and C ₁ ′ is the point projected on image I ₂ .

The intersection between the intersection of the two image planes and the plane by COP, C ₁ , C ₂ is defined as IIP. As shown in the drawing, when two images have the same focal length, the lengths of the line segment C ₁ IIP and the line segment C ₂ IIP are the same, and the position of the focal length is located near the center of the line segment C ₁ 'C ₂ . Here, the two image joints by the minimization technique do not always show good results. In other words, the conversion matrix is incorrectly calculated. In this case, the focal lengths between the two images are different. If the estimated focal lengths of the two images are f ₀ and f ₁ , respectively, the position of the IIP depends on the following three cases.

1) If f ₀ = f ₁ , the IIP is located near the center of the line segment C ₁ 'C ₂ .

2) If f ₀ > f ₁ , the IIP is close to C ₁ '.

3) If f ₀ <f ₁ , the IIP is close to C ₂ .

These three cases can be used to measure the reliability of image bonding. The closer the IIP is to the center of the line segment C ₁ 'C _2, the more successful the bonding.

That in the drawing the line segment C ₁ IIP the projection plane line segment in I ₂ C ₁ 'is the IIP. Dividing this line segment C ₁ IIP into n unit pieces, the projection length of each unit piece is equal to the length of the unit piece at the IIP position. The position of the IIP is determined by the ratio of the length of the unit piece in the projected plane to the length of the unit piece. In the IIP the two length ratios are one.

In addition, it can be seen that the angle formed by C ₁ ′, IIP, and C ₁ is equal to θ. Therefore, the rotation angle θ and the focal length f can be obtained by the following equations.

Szeliski estimates the focal length using the elements of the projective transformation matrix. In Equation 4, m ₆ and m ₇ are close to 0, which is likely to exceed the error tolerance. Therefore, the focal length of I ₂ has a wide range of fluctuations in several images with the same actual focal length. To reduce this error, the geometric mean value of the estimated focal length was used as the estimated focal length.

Here, the distance between the center C ₂ of one image plane and the image center C ₁ ′ projected on the plane and the rotation angle θ between the centers is obtained. The difference between the measured rotation angle and the actual rotation angle is not so large, and the length between the centers is also small and the error is not large. In fact, the error of the rotation angle is about 2 to 5 degrees, the difference in length between centers is about 15 pixels, and the estimated focal length obtained is close to the actual focal length. Once the focal length is estimated, the distance between the current position and the subject can be calculated by the following equation.

Here, a denotes a distance between the subject and the lens, that is, a distance between the lens and the subject of the mobile communication terminal, and b denotes a distance between the lens and the image. Here, b may be assumed as the distance between the CCD and the lens, and this value is a value known by the manufacturing process of the mobile communication terminal.

In addition, the calculation unit 13 calculates the rotation angle of the imaging unit 11 rotated toward the subject with respect to the direction of the imaging unit 11, that is, true north. The direction of the imaging section 11 can be realized by a gyro sensor. The gyro sensor is a posture positioning device that measures a change in posture of a mobile communication terminal. Since the measurement of the posture change by the gyro sensor is a known technique, a description thereof will be omitted.

The search unit 14 includes a GIS receiving module (not shown) capable of wirelessly receiving GIS information from a GIS information provider, and includes a distance between a subject calculated by the calculator 13 and a mobile communication terminal. The information on the subject on the GIS information is retrieved based on the direction of the imaging unit 11. That is, when the current position of the mobile communication terminal is confirmed by the positioning unit 12, the search unit 14 may determine the current position on the received GIS information by the direction and distance calculated by the calculation unit 13. Search for information on distant subjects.

The display unit 15 is a liquid crystal display (LCD) on which an image picked up by the image pickup unit 11 is displayed, and displays information on a subject retrieved by the retrieval unit 14 on a subject to be picked up.

The input unit 16 is provided in the form of a button so that a user can input a destination to be searched, or in the form of a microphone that receives a voice command. In this case, when the input unit 16 is implemented in the form of a microphone, the mobile communication terminal further includes a voice recognition unit 17 capable of analyzing and recognizing a voice corresponding to a destination input through the input unit 16. desirable.

The GIS information storage unit 18 stores the GIS information received from the GIS information provider, thereby eliminating the need to connect with the GIS information provider every time.

3 is a flowchart illustrating an embodiment of a method for providing region information by the mobile communication terminal of FIG. 1. Referring to the drawings, when the user visits a place where the geography is not well known and sets the local information display mode, the positioning unit 12 receives the GPS satellite signals from the GPS satellites and checks the current position (S101). .

The user captures the surrounding image through the image pickup unit 11 of the mobile communication terminal together with the local information display mode setting (S103). In this case, the user may capture at least two or more still images, or may capture a continuous image of a moving image while rotating 360 degrees around. In addition, the captured image is not necessarily stored, it is enough to be displayed on the screen of the mobile communication terminal so that the user can easily match the actual surroundings.

The calculator 13 calculates a distance between the at least one subject to be picked up and the mobile communication terminal and the direction of the imaging unit 11 (S105).

The search unit 14 receives the GIS information, and searches for information on the subject from the received GIS information based on the distance and direction calculated by the calculation unit 13 (S107). That is, the searching unit 14 searches for information on the subject separated by the direction and distance calculated by the calculating unit 13 based on the current position identified by the positioning unit 12. The information on the subject here refers to information on the subject displayed on the GIS information, such as a building name, a street name, an old palace name, and the like.

The display unit 15 displays the information of the subject searched by the searching unit 14 on the subject appearing on the screen of the mobile communication terminal, that is, the photographed subject (S109).

As a result, when the user visits a strange place with no knowledge of the local information, the user can easily grasp the local information only by photographing the surrounding image through the mobile communication terminal.

4 is a flowchart illustrating another embodiment of a method for providing region information by the mobile communication terminal of FIG. 1. Referring to the drawing, a user who visits an unfamiliar place sets the mobile communication terminal to the local information display mode, checks the current position through the positioning unit 12 (S201), and searches through the input unit 16. A destination can be input (S203). However, the step of confirming the current position (S201) and the destination input step (S203) is not limited to that shown, the order of these may be changed.

The user captures an image of the surroundings through the image capturing unit 11 together with setting the local information display mode (S205). The imaging of the peripheral image by the imaging unit 11 is the same as described above.

The calculation unit 13 calculates the distance between the at least one subject to be imaged and the mobile communication terminal and the direction of the imaging unit 11 (S207).

The searching unit 14 receives the GIS information, and searches for the information of the subject from the GIS information based on the distance and the direction calculated by the calculating unit 13 (S209). At this time, the GIS information storage unit 18 is preferably implemented to store the GIS information received from the GIS information provider (S211). As a result, when the local information display mode is repeatedly used in the same place, the process of receiving the same information again from the GIS information provider can be omitted, thereby reducing the communication cost and reducing the search time.

Information of the subject searched by the search unit 14 is displayed on the screen of the mobile communication terminal as in the case of FIG. 3 (S213).

On the other hand, when the user inputs the destination through the input unit 16, the search unit 14 receives the GIS information corresponding to the destination from the GIS information received from the GIS information provider or the GIS information stored in the GIS information storage unit 18. Search (S215). To this end, the input of the destination through the input unit 16 is preferably input to the address, it is preferably made of the name of the main building, or the name of the main public office. In addition, when there are names of the same building and the like in a plurality of areas, it is preferable to input the name together with the area name.

When the GIS information corresponding to the destination is found, the searching unit 14 determines whether there is information of the subject that matches the information of the destination among the information of the subject displayed on the display unit 15 of the mobile communication terminal (S217).

If a subject that matches the information of the destination is found among the information of the photographed subject, the searching unit 14 displays the identification of the subject (S219). Such an identification mark may be implemented by expressing the corresponding subject in a color distinguishing it from other subjects, or by marking an arrow when indicating the corresponding subject.

As a result, when a user visits an unfamiliar area, the user may not only easily grasp local information of the surrounding area, but also easily search for a desired destination.

FIG. 5 is a flowchart illustrating still another embodiment of a method for providing region information by the mobile communication terminal of FIG. 1. 4 illustrates a case where the destination input step S301 and the current location checking step S303 are changed, and the peripheral image capturing step S305 and the distance and direction calculation step S307 are described above. Since it is the same as the one, the description thereof is omitted.

The search unit 14 receives the GIS information and searches for information corresponding to the destination from the received GIS information (S309). In addition, the searching unit 14 searches for information on the subject from the received GIS information based on the distance and direction calculated by the calculating unit 13 (S311).

When the GIS information corresponding to the subject is found, the searching unit 14 determines whether the information of the found subject matches the information of the destination (S313).

If it is determined that the information of the subject coincides with the information of the destination, the searching unit 14 displays the identification mark on the subject (S315). Such an identification mark may be implemented by expressing the corresponding subject in a color distinguishing it from other subjects, or by marking an arrow when indicating the corresponding subject. As a result, when the user visits an unfamiliar area and seeks a destination, the user can easily identify the destination simply by photographing the surrounding image through the mobile communication terminal.

Although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the present invention is not limited to the specific embodiments of the present invention without departing from the spirit of the present invention as claimed in the claims. Anyone skilled in the art can make various modifications, as well as such modifications are within the scope of the claims.

According to the present invention, the mobile communication terminal can display local information on the surrounding image, thereby allowing a user to easily grasp information of an unfamiliar area.

In addition, the mobile communication terminal according to the present invention identifies and displays a destination to be searched by the user from the surrounding images, so that the user can easily identify the destination.

In addition, since the mobile communication terminal according to the present invention combines and displays local information in an image of a surrounding, it is much easier to grasp than the local information displayed on the map.

Claims (11)

An imaging unit which picks up an image of a surrounding; A positioning unit configured to receive GPS (Global Positioning System) information and confirm a current position; A calculation unit that calculates a distance between at least one subject in the surrounding image to be captured and the current position, and a direction of the imaging unit; A search unit for receiving Geographic Information System (GIS) information and searching for information of the subject on the GIS information based on the calculated distance and direction; And And a display unit which displays the information of the subject found on the photographed subject. The method of claim 1, Further comprising an input unit for inputting a destination to search, The search unit searches for the GIS information corresponding to the destination from the GIS information, and the display unit displays an identification mark on the photographed subject when the information of the subject matches the GIS information of the destination. Communication terminal. The method of claim 1, The mobile communication terminal further comprises a GIS information storage unit for storing the received GIS information. The method of claim 2, And a voice recognition unit for analyzing and recognizing a voice corresponding to the destination input through the input unit. Receiving a GPS information and checking a current position; Imaging the surrounding image; Calculating a distance between at least one of the subjects in the peripheral image to be photographed and the current position, and a direction of an image capturing unit that picks up the peripheral image; Receiving GIS information and retrieving information of the subject on the GIS information based on the calculated distance and direction; And And displaying the information on the subject found on the photographed subject. The method of claim 5, Inputting a destination to search for; Retrieving GIS information corresponding to the destination from the GIS information; And And displaying an identification mark on the photographed subject when the subject information matches the GIS information of the destination. The method of claim 5, And storing the received GIS information. An imaging unit which picks up an image of a surrounding; An input unit for inputting a destination to search for; A location checking unit for receiving GPS information and checking a current location; A calculation unit that calculates a distance between at least one subject in the surrounding image to be captured and the current position, and a direction of the imaging unit; A search unit for receiving GIS information, searching for information of the subject on the GIS information based on the calculated distance and direction, and searching for GIS information corresponding to the destination from the GIS information; And And a display unit for displaying an identification mark on the subject to be photographed when the information of the subject coincides with the GIS information of the destination. The method of claim 8, And a voice recognition unit for analyzing and recognizing a voice corresponding to the destination input through the input unit. The method of claim 8, The mobile communication terminal further comprises a GIS information storage unit for storing the received GIS information. Inputting a destination to search for; Receiving a GPS information and checking a current position; Imaging the surrounding image; Calculating a distance between at least one of the subjects in the peripheral image to be photographed and the current position, and a direction of an image capturing unit that picks up the peripheral image; Receiving GIS information and retrieving GIS information corresponding to the destination from the GIS information; Retrieving information of the subject on the GIS information based on the calculated distance and the direction; And And displaying identification marks on the photographed subjects when the subject information matches the GIS information of the destination.

Images