KR20110004974A - Method for providing location based services by using augmented reality, system, server and portable terminal therefor - Google Patents

Abstract

PURPOSE: A method for providing location based services by using augmented reality, a system, a server and a portable terminal therefore are provided to display location based information corresponding to the directivity change of a real-time image obtained through a camera of a portable terminal. CONSTITUTION: A region determination unit(320) determines the size of a map region including a current position of a portable terminal based on the location measurement information received from the portable terminal. An environment information searching unit(340) searches the major peripheral environment information included in the determined size of the map region. A server control unit(330) transmits the location based information to the portable terminal. The location based information is created by applying the some of the major peripheral environment information to the map information corresponding to the determined size of the map region.

Description

Method for Providing Location-Based Services using Augmented Reality and System, Server, and Portable Terminal for It {Method for Providing Location Based Services by Using Augmented Reality, System, Server And Portable Terminal Therefor}

One embodiment of the present invention relates to a method for providing a location based service using augmented reality and a system, a server, and a portable terminal therefor. More specifically, the user can obtain the location information and the direction information by synthesizing and displaying the location-based information corresponding to the change in the orientation of the real-time image and the real-time image acquired through the camera by using a portable terminal equipped with a camera. The present invention relates to a method for providing location-based services using augmented reality for continuously receiving location-based information according to a real-time image that changes according to movement of a system, and a system, a server, and a portable terminal therefor.

In general, location-based services (LBS) measure the location of the terminal with a mobile phone, PDA, PC, etc., and provide route guidance, POI (Point Of Interest) information, shopping information, advertising, etc. Refers to a service that receives information services. Such methods for location measurement for location-based services include a method using a base station of a mobile communication network, a method using a GPS satellite signal, and various methods using a wireless LAN, Bluetooth, WLAN, WPAN, or AP (Access Point). It is being used.

In the above-described conventional techniques, the position of the terminal can be detected, but information on which direction the terminal user is facing is not known. That is, when providing a location-based service with only the location information, by outputting the surrounding map on the display device of the terminal, the location of the terminal user on the map, and enumerating the relevant location-based service items, the user wants to It is configured to choose. However, in this case, if the user does not provide accurate map information such as 3D map information, the user must recognize the surroundings by comparing the actual environment seen through the real eye with the map through the terminal. In addition, even if the terminal provides three-dimensional map information, since the amount of data is large, it is difficult to store in the storage device, there is a disadvantage that it is not easy to download through the wireless communication network.

In order to solve the above-described problem, an embodiment of the present invention synthesizes and displays location-based information corresponding to a change in orientation of a real-time image obtained through a camera of a portable terminal, thereby displaying a real-time image changed according to a user's movement. The main purpose is to provide a method for providing a location-based service using augmented reality to continuously receive the location-based information according to the following, and a system, a server, and a portable terminal therefor.

One embodiment of the present invention to achieve the above object, a server communication unit for interworking with a portable terminal; An area determiner configured to determine a map area size including a current location of the portable terminal based on the location information received from the portable terminal; An environment information search unit for searching main surrounding environment information included in the determined map area size; And a server controller configured to transmit location-based information to the portable terminal, wherein the location-based information is applied to the map information corresponding to the determined map area size based on some of the found main surrounding information according to the determined area size. to provide.

In addition, according to another object of the present invention, the display unit for displaying the execution screen of the running application; A terminal communication unit interworking with a location-based server; A location based measuring unit obtaining location based related information; A location-based input unit configured to generate location information, real-time image information, and direction information using the obtained location-based related information; And transmitting the location information to the location-based server, receiving location-based information corresponding to the location location information from the location-based server, and synthesizing the real-time image information, the direction information, and the received location-based information. It provides a portable terminal characterized in that it comprises a terminal controller for displaying through the display unit.

According to another object of the present invention, the size of the map area including the current location is determined based on the location information, the main surrounding environment information included in the determined map area size is searched, and according to the determined area size. A location-based server transmitting location-based information to which some information of the found main surrounding environment information is applied; A portable terminal for transmitting the location information to the location-based server, receiving the location-based information from the location-based server, and displaying the real-time image information by combining direction information and the location-based information; And a wired / wireless communication network for relaying data transmitted and received between the portable terminal and the location-based service server.

According to another object of the present invention, the step of determining the size of the map area including the current location of the terminal based on the location information received from the portable terminal; Retrieving, by the location-based server, main surrounding environment information included in the determined map area size; Transmitting, by the location-based server, location-based information to which the partial information of the found main surrounding environment information is applied to the portable terminal according to the determined area size; And receiving, by the portable terminal, the location-based information, and synthesizing and displaying the real-time image information and the direction information acquired on the received location-based information. To provide.

As described above, according to an embodiment of the present invention, by synthesizing and displaying location-based information corresponding to a change in orientation of a real-time image obtained through a camera of a portable terminal, It is effective to continuously receive location-based information.

In addition, according to an embodiment of the present invention, compared to the general map display method from the user's point of view, it is easy to determine the surrounding position through the location-based service synthesized in the real-time image. In addition, according to an embodiment of the present invention, there is an effect that can be provided with a location-based service using augmented reality using a portable terminal possessed by the user without a separate device using augmented reality.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In addition, in describing the component of this invention, terms, such as 1st, 2nd, A, B, (a), (b), can be used. These terms are only for distinguishing the components from other components, and the nature, order or order of the components are not limited by the terms. If a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected to or connected to that other component, but another component may be It is to be understood that the components may be "connected", "coupled" or "connected".

1 is a block diagram schematically illustrating a location-based service system using augmented reality according to an embodiment of the present invention.

This technology uses the location and direction information to determine the approximate location, and compares facility information such as surrounding building information with live-action image information input according to the camera movement to identify the desired service and provide related information. It is called Augmented Reality (AR) technology. More specifically, augmented reality is a field of virtual reality (VR), a computer graphics technique that synthesizes virtual objects in the real environment and makes them look like objects in the original environment. Unlike the existing virtual reality targeting only objects, it is a technology that can synthesize virtual objects on the basis of the real world and reinforce and provide additional information that is difficult to obtain only in the real world.

Location-based service system using augmented reality according to an embodiment of the present invention includes a portable terminal 110, wired and wireless communication network 120, location-based server 130 and DB server 140.

The portable terminal 110 refers to a terminal that performs normal voice call and data communication by wireless communication in conjunction with the wired / wireless communication network 120. For example, the portable terminal 110 may be a personal digital assistant (PDA), a cellular phone, a personal communication service (PCS) phone, a hand-held PC, a CDMA-2000 phone, a WCDMA phone, a portable multimedia (PMP). Player), PSP (PlayStation Portable) and MBS (Mobile Broadband System) phone and the like.

The portable terminal 110 interworks with the location-based server 130, obtains location information, real-time image information, and direction information through the location-based measurement unit 210, and acquires the location information from the location-based server 130. Transmits the location-based information corresponding to the location information from the location-based server 130, and synthesizes and displays the obtained real-time image information, the direction information, and the received location-based information. Here, the portable terminal 110 includes a camera module 212 for obtaining real-time image information, a gyro sensor 214 for detecting a motion, an acceleration sensor 216 for detecting an acceleration, and navigation included in a radio wave signal. The terminal is equipped with a GPS module 218 for acquiring navigation data. The portable terminal 110 receives an image input unit 222 that receives and transmits the acquired real-time image information, a direction measuring unit 224 that generates direction information based on the detected motion and the detected acceleration, and based on the obtained navigation data. And a position measuring unit 226 for generating positional positioning information including latitude and longitude coordinate values.

The portable terminal 110 processes the received real-time image information and the received location-based information according to the direction information, and receives the received real-time image information on the received real-time image information through pattern matching between the received real-time image information and the received location-based information. Recognizes objects of location-based information, and extracts and transmits a location value of the object when synthesizing the location-based information into real-time image information. In addition, the portable terminal 110 receives the location-based information by controlling the terminal communication unit 260, and performs a function of controlling so that only the information set by the user among the main surrounding environment information included in the location-based information appears. . Meanwhile, the portable terminal 110 performs a function of synthesizing the real-time image and the direction information according to the position value of the object and then displaying it.

The wired / wireless communication network 120 refers to a network capable of transmitting and receiving data using an internet protocol using various wired and wireless communication technologies such as an internet network, an intranet network, a mobile communication network, and a satellite communication network. The wired / wireless communication network 120 is a network connecting the location-based server 130 and the portable terminal 110, but may be a closed network such as a local area network (LAN), a wide area network (WAN), or the like. It is preferable to be open type such as Internet). The Internet has many services in the TCP / IP protocol and its upper layers: HyperText Transfer Protocol (HTTP), Telnet, File Transfer Protocol (FTP), Domain Name System (DNS), Simple Mail Transfer Protocol (SMTP), and SNMP ( The global open computer network architecture that provides the Simple Network Management Protocol (NFS), Network File Service (NFS), and Network Information Service (NIS). In addition, when the portable terminal 110 is a mobile communication terminal, the wired / wireless communication network 120 may include a mobile communication network. Here, since the technology for the wired and wireless communication network 120 is already known technology, a detailed description thereof will be omitted.

Location-based server 130 is a server that provides location-based services, receives the location information from the mobile terminal 110, extracts the current location using the location information, route guidance associated with the current location, POI ( Point Of Interest) refers to a server that provides information, shopping information, advertisement information, and the like. That is, when the location-based server 130 receives a location-based service request from the mobile terminal 110 via the wired / wireless communication network 120, the location-based server 130 receives location location information from the mobile terminal 110, and obtains directions information and tourism information. After receiving and receiving information about the surrounding location of the portable terminal 110, such as shopping information and advertising information to the DB server 140, and delivers to the portable terminal 110 via the wired and wireless communication network 120. do.

The location-based server 130 interoperates with the mobile terminal 110, determines the size of the map area including the current location of the mobile terminal 110 based on the location information received from the mobile terminal 110, and determines the determined map. It searches for the main surrounding environment information included in the area size and transmits the location-based information to which the portable terminal 110 is applied based on some of the found main surrounding information according to the determined area size. Here, the location-based server 130 performs a function of applying some of the main environment information to the location-based information based on the importance information according to the retrieved main environment information. In addition, for example, the importance information is important information for each neighboring building information included in the searched main environment information, and public institutions, hospitals, gas stations, intersection names, and access road names are given high importance. Information for advertising purposes such as restaurants and restaurant names may be given a relatively low importance.

In addition, the location-based server 130 extracts the current location information of the portable terminal 110 based on the latitude and longitude coordinate values included in the location information, and the area corresponding to the altitude information and the current location information of the current location information. The size of the map area is determined based on at least one piece of type information. In addition, the location-based server 130, in conjunction with the DB server 140, performs a function of determining the map area size using the map information received from the DB server 140. For example, the location-based server 130 may use the altitude information of the current location information of the portable terminal 110 in order to change the size of the area to deliver the information according to the location of the portable terminal 110. That is, when it is determined that the location of the portable terminal 110 is located at a very high place such as a mountain top by using the altitude information of the current location information, the area (or the user) displayed through the camera module 212 of the portable terminal 110. Desired area) is displayed as a large area (or a distant area), so it is possible to determine a large area where the map area size is accumulated, and generate location-based information including only some of the most important information of the surrounding environment. It is. Meanwhile, the location-based server 130 may use region type information corresponding to the current location information of the portable terminal 110 in order to change the size of the area to which the information is to be delivered according to the location of the portable terminal 110. That is, when the area type information corresponding to the current location information is 'Gangnam area commercial area', it may be assumed that the location of the portable terminal 110 is surrounded by a large building, and at this time, the camera module of the portable terminal 110 Since the image (or image required by the user) acquired through the 212 is displayed as a narrow region (or a region limited to a short distance), the size of the map area is determined as a narrow region, and the detailed location base including all surrounding environment information is included. Information can be generated.

The location-based server 130 performs a function of retrieving the main surrounding environment information from at least one or more of tourism information, shopping information, advertisement information, place name information, and address information. Meanwhile, the location-based server 130 interworks with the DB server 140 and performs a function of searching for main peripheral information based on the data received from the DB server 140.

The DB server 140 is a kind of database that stores information related to a map, and may include a map DB, a place name and address DB, a directions DB, a tourist information DB, a shopping DB, an advertisement DB, and other DBs. In addition, in FIG. 1, the DB server 140 is illustrated as a server implemented separately from the location-based server 130, but is not necessarily limited thereto, and may be implemented as being included in the location-based server 130. .

2 is a block diagram schematically illustrating a portable terminal according to an embodiment of the present invention.

The portable terminal 110 according to an embodiment of the present invention includes a location-based measurement unit 210, a location-based input unit 220, a terminal controller 230, a display unit 250, and a terminal communication unit 260. .

In an embodiment of the present invention, the portable terminal 110 includes only the location-based measurement unit 210, the location-based input unit 220, the terminal control unit 230, the display unit 250, and the terminal communication unit 260. Although described, these are merely illustrative of the technical idea of one embodiment of the present invention, those skilled in the art to which one embodiment of the present invention belongs will not depart from the essential characteristics of one embodiment of the present invention. It will be applicable to various modifications and variations by adding a memory unit, a key input unit, a microphone, a speaker, etc. to the components included in the portable terminal 110 in the range.

The location-based measuring unit 210 acquires navigation data included in a camera module 212 for obtaining real-time image information, a gyro sensor 214 for detecting motion, an acceleration sensor 216 for detecting acceleration, and a radio wave signal. GPS module 218 may be configured. Here, the gyro sensor 214 senses the angular velocity of the terminal movement of the yaw axis and the pitch axis. The gyro sensor 214 may be attached to capture the angular velocity of the movement in both the yaw axis and the pitch axis in order to recognize the rotation, tilt or movement of the terminal up, down, left and right. That is, the direction measuring unit 224 removes a high frequency noise component from the motion signal of the terminal captured by the gyro sensor 214, extracts only a desired band, and uses the signal processed from the gyro signal to move up, down, left, and right of the terminal. Rotation, tilt or movement of the furnace can be recognized. Here, the real-time image information, the detected motion information, the detected acceleration information, and the navigation data acquired by the location-based measurement unit 210 may be grouped with location-based related information for convenience of description.

The location-based input unit 220 performs a function of receiving location positioning information, real-time image information, and direction information through the location-based measuring unit 210. Meanwhile, the location-based input unit 220 may transmit location location information, real-time image information, and direction information to the terminal controller 230 by using the location-based related information obtained through the location-based measurement unit 210. In addition, the location-based input unit 220 is an image input unit 222 for receiving and transmitting the obtained real-time image information, the direction measuring unit 224 for generating direction information based on the detected motion and the detected acceleration, the obtained navigation And a position measuring unit 226 for generating positional positioning information including latitude and longitude coordinate values based on the data. That is, the image input unit 222 receives image information input in real time through the camera module 212 and transmits the image information to the image output controller 232 and the image processor 236. The direction measuring unit 224 is a module for finding direction information actually directed by the camera module 212, and may calculate direction information using at least one or more of the gyro sensor 214 and the acceleration sensor 216. . The position measuring unit 226 is a module for determining the current position of the portable terminal 110 and may calculate positional positioning information using the GPS module 218. However, the position measuring unit 226 may not be able to measure the location information when the portable terminal 110 is located in a sound region of the GPS, such as indoors (in-building) or between tall buildings. ID) or WLAN or WPAN triangulation.

The terminal controller 230 transmits the received location information to the location based server 130, receives location based information corresponding to the location location information from the location based server 130, and receives the received real-time image information and direction information. To the image processor 236. In addition, the terminal controller 230 includes an image output controller 232, a location information controller 234, and an image processor 236.

The image output controller 232 processes the received real-time image information and the received position-based information according to the direction information, and receives the received real-time image information on the received real-time image information through pattern matching between the received real-time image information and the received position-based information. Recognizes the object of the location-based information, and extracts and transfers the position value of the object when synthesizing the location-based information to the real-time image information. That is, the role of the image output controller 232 is to direct the real-time image information received through the image input unit 222 and the characteristic information about the building in the direction of the portable terminal 110 received through the location information control unit 234 Recognizes objects of location-based information by comparing them through pattern matching between real-time image information and location-based information, and determines the appropriate location when mixing location-based service information with real-time image information. The value is transmitted to the image processor 236.

In addition, the location information controller 234 controls the terminal communication unit 260 to receive location-based information, and performs a function of controlling only the user-set information among the main surrounding environment information included in the location-based information to appear. . That is, the location information controller 234 transfers the location information of the portable terminal 110 received from the location measuring unit 226 to the location based server 130 through the terminal communication unit 260, and transmits the terminal communication unit 260. Receives the main surrounding environment information from the location-based server 130 through, location location information, direction information received from the location measuring unit 226 and the direction measuring unit 224 and location-based received from the location-based server 130 Only information determined as information desired by the user from the main surrounding environment information of the information may be extracted and transmitted to the image output controller 232.

The image processor 236 performs a function of synthesizing the received real-time image information, the received direction information, and the received location-based information through the display unit 250. In addition, the image processor 236 synthesizes and displays the real-time image and the direction information according to the position value for the object. That is, the image processor 236 synthesizes the real-time image information received from the image input unit 222 and the additional information received from the image output control unit 232 to match the position value of the object received from the image output control unit 232 to display the display unit. And may be displayed via 250. Meanwhile, although FIG. 2 illustrates that the image processor 236 is implemented separately from the terminal controller 230, the present invention is not limited thereto, and the image processor 236 is included in the terminal controller 230. Can be implemented.

The display unit 250 displays the operation state of the terminal including at least one of at least one of power state information, radio wave reception intensity information, date information, and time information, and when information is transmitted in the form of letters, numbers, images, etc. Screen display means for outputting information. That is, the display unit 250 displays an execution screen of a running application.

The terminal communication unit 260 is a wireless communication processing means for performing voice or data communication through the wired / wireless communication network 120 or an access point (AP). The terminal communication unit 260 codes or decodes a voice signal and performs an equalizer function to remove multipath noise. Digital signal processing function to perform sound data processing function, baseband signal for transmitting and receiving signals, baseband conversion function to perform digital-to-analog conversion and analog-to-digital conversion processing, and RF (Radio Frequency) It receives the signal and converts it into an IF (Intermediate Frequency) signal, converts the IF signal into an RF signal, performs an RF signal processing function for demodulating and amplifying the RF signal, and an antenna function for transmitting and receiving a wireless signal in the air. Meanwhile, the terminal communication unit 260 may be implemented to process data packets using wireless communication technologies such as WLAN, Wi-Fi, WiMAX, and WiBro. It is not necessarily limited thereto. Meanwhile, the terminal communication unit 260 performs a function of interworking with the location based server 130.

3 is a block diagram schematically illustrating a location-based server according to an embodiment of the present invention.

Location-based server 130 according to an embodiment of the present invention is configured to include a server communication unit 310, area determination unit 320, server control unit 330, environmental information search unit 340.

In an embodiment of the present invention, the location-based server 130 is described as including only the server communication unit 310, the area determination unit 320, the server control unit 330, and the environment information retrieval unit 340. This is merely illustrative of the technical idea of one embodiment of the present invention, those skilled in the art to which one embodiment of the present invention belongs to location within the scope without departing from the essential characteristics of one embodiment of the present invention Various modifications and variations to the components included in the server 130 may be applicable.

The server communication unit 310 is a communication means for performing a function of interworking with the portable terminal 110 via the wired or wireless communication network 120, and performs a function of transmitting and receiving various data. In addition, the server communication unit 310 performs a function of interworking with the DB server 140. For example, the server communicator 310 may include a data receiver and a data transmitter, and the data receiver receives the location information of the mobile terminal 110 through the wired / wireless communication network 120 and transmits the location determination information to the area determiner 320. It serves to convey. Meanwhile, the data transmitter of the server communication unit 310 transmits the information received from the server controller 330 to the portable terminal 110 via the wired / wireless communication network 120.

The area determiner 320 determines the size of the map area including the current location of the mobile terminal 110 based on the location information received from the mobile terminal 110. In addition, the area determiner 320 extracts current location information of the mobile terminal 110 based on the latitude and longitude coordinate values included in the location information, and includes an area corresponding to the altitude information and the current location information of the current location information. The size of the map area is determined based on at least one piece of type information. In addition, the area determiner 320 performs a function of determining a map area size using the map information received from the DB server 140. For example, the area determiner 320 grasps the shape of the area corresponding to the current location of the portable terminal 110 using map DB information, and determines the size of the area for which the information is to be provided by the server controller 330. To send.

Here, the DB server 140 classifies information about a map DB, a place name and an address DB, a directions DB, a tourist information DB, a shopping DB, an advertisement DB and other DBs, and stores and manages the information in a database. May be implemented inside or outside the location-based server 130. In an embodiment of the present invention, the DB server 140 is illustrated as being implemented outside the location-based server 130, but is not necessarily limited thereto. In addition, such a database refers to a general data structure implemented in a storage system (hard disk or memory) of a computer system using a database management program (DBMS), and searches for (extracts), deletes, edits, and adds data. It is a data storage type that can be used freely, such as relational database management system (RDBMS) such as Oracle, Infomix, Sybase, DB2, Gemston, Orion ( An object-oriented database management system (OODBMS) such as Orion, O2, etc., and an XML Native Database such as Excelon, Tamino, Sekaiju, etc. It can be implemented according to the purpose and has appropriate fields or elements to achieve its function. Meanwhile, the DB server 140 transmits the map information included in the map DB among the map DB, place name and address DB, directions DB, tourism information DB, shopping DB, advertisement DB, and other DBs to the area determination unit 320. By doing so, the area determiner 320 may determine the map area size.

The server controller 330 is a control means for controlling the overall functions of the server. The server controller 330 determines some information of the main surrounding environment information searched by the environment information search unit 340 according to the area size determined by the area determiner 320. The location-based information applied to the map information corresponding to the area size is generated and transmitted to the portable terminal 110. The server controller 330 may include a memory for storing a program for controlling the components of the server and a microprocessor for controlling the components of the server by executing the corresponding program. For example, the server controller 330 requests the environment information retrieval unit 340 for the main surrounding environment information included in the size of the determined map area received from the area determination unit 320 and from the environment information retrieval unit 340. The received information and the importance of the information are extracted in consideration of the size of the information providing area, and then transferred to the data transmitter of the server communication unit 310.

The environment information retrieval unit 340 retrieves the main surrounding environment information included in the determined map area size. In addition, the environment information retrieval unit 340 transmits the importance information according to the retrieved main surrounding environment information to the server control unit 330, and causes the server control unit 330 to display the main surrounding environment based on the location-based information based on the received importance information. Apply some of the information.

The environment information retrieval unit 340 retrieves the main surrounding environment information from at least one of tourism information, shopping information, advertisement information, place name information, and address information. In addition, the environment information retrieval unit 340 retrieves the main surrounding information based on the data received from the DB server 140. For example, the environment information retrieval unit 340 searches the DB server 140 to find the corresponding information requested from the server control unit 330, and retrieves the importance of the information and the information stored in the DB server 140. Transfer to the control unit 330. Here, the DB server 140, the map information contained in the map DB, name and address DB, directions DB, tourism information DB, shopping DB, advertising DB and other DB of the map DB to the environmental information search unit 340 By transmitting the information, the environment information retrieval unit 340 retrieves the main surrounding environment information included in the determined map area size.

4 is a flowchart illustrating a location-based service method using augmented reality according to an embodiment of the present invention.

The portable terminal 110 checks whether there is a request for a location-based service by a user's manipulation or command (S410). That is, a user may drive a location based application installed in the portable terminal 110 or select a location based menu to use the location based service.

As a result of checking in step S410, if there is a request for location-based service, the mobile terminal 110 obtains the navigation data included in the radio signal through the GPS module 218 and the location measuring unit 226, and obtained navigation The position including the latitude-longitude coordinate value is determined based on the data (S421).

The portable terminal 110 transmits the location information to the location-based server 130 via the wired or wireless communication network 120 in order to use the location-based service (S422). When the location-based server 130 receives the location information from the mobile terminal 110, the location-based server 130 transmits the map terrain information request signal to the DB server 140 (S423), and receives the map terrain information response signal from the DB server 140. It receives (S424).

The location-based server 130 extracts current location information of the portable terminal 110 based on the latitude and longitude coordinate values included in the location information, and obtains altitude information of the current location information and region type information corresponding to the current location information. In operation S425, the size of the map area is determined based on at least one or more pieces of information of the map terrain information response signal from the DB server 140.

The location-based server 130 transmits the surrounding main environment information request signal for searching the main surrounding environment information included in the determined map area size to the DB server 140 (S426), and the surrounding main environment from the DB server 140. An information response signal is received (S427).

The location-based server 130 generates location-based information to which some information of the main surrounding environment information found according to the determined area size is applied (S428). In this case, the location-based server 130 applies some of the main surrounding information to the location-based information based on the importance information according to the searched main surrounding information, or the user based on the main surrounding information included in the location-based information. Only the set information can be displayed. The location-based server 130 transmits the location-based information to the mobile terminal 110 via the wired / wireless communication network 120 (S429).

Steps S421 to S429 are processes of generating location-based information from the location-based server 130 and transferring the location-based information to the mobile terminal 110. Therefore, the step is viewed as the location-based information reception step S420 at the portable terminal 110. Can be.

The portable terminal 110 receives location based information from the location based server 130, obtains real time image information through the camera module 212, and obtains direction information through the gyro sensor 214 and the acceleration sensor 216. Acquire (S432).

The portable terminal 110 synthesizes and displays the received location-based information, real-time image information and direction information (S434). That is, the portable terminal 110 processes the received real-time image information and the received location-based information according to the direction information, and on the received real-time image information through pattern matching between the received real-time image information and the received location-based information. Recognizes the object of the received location-based information, extracts the position value of the object when synthesizing the position-based information into the real-time image information, synthesizes the real-time image and direction information according to the position value of the object and display can do.

Steps S432 to S434 are processes of synthesizing and displaying the location-based information, the direction information, and the real-time image information in the portable terminal 110, and thus, the corresponding step is to provide the location-based service on the portable terminal 110 (S430). Can be seen as.

The portable terminal 110 confirms whether the area of the location-based information received from the location-based server 130 is out of operation (S440), and based on the result of the check, If it leaves the area, S420 is performed (S450). On the other hand, if the check result of step S440, the area of the location-based information received from the location-based server 130 does not leave, S430 is performed (S460).

In FIG. 4, the location-based server 130 and the portable terminal 110 sequentially execute steps S410 to S460. However, the location-based server 130 and the portable terminal 110 are merely illustrative of the technical idea of an embodiment of the present invention. Those skilled in the art to which an embodiment of the present invention belongs, the location-based server 130 and the portable terminal 110 by changing the order described in Figure 4 without departing from the essential characteristics of one embodiment of the present invention 4 may be applied by various modifications and variations by executing one or more steps of steps S410 to S460 in parallel, and therefore, FIG. 4 is not limited to the time series order.

As described above, the location-based service method using augmented reality according to an embodiment of the present invention described in FIG. 4 may be implemented in a program and recorded in a computer-readable recording medium. The computer-readable recording medium for recording a program for implementing a location-based service method using augmented reality according to an embodiment of the present invention includes all kinds of recording devices for storing data that can be read by a computer system. do.

Examples of such computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and also implemented in the form of a carrier wave (e.g., transmission over the Internet) . The computer readable recording medium may also be distributed over a networked computer system so that computer readable code is stored and executed in a distributed manner. In addition, functional programs, code, and code segments for implementing an embodiment of the present invention may be easily inferred by programmers skilled in the art to which an embodiment of the present invention belongs.

5 is an exemplary view of a location based service using augmented reality according to an embodiment of the present invention.

As illustrated in FIG. 5, the portable terminal 110 may include direction information and a location-based server obtained through the gyro sensor 214 or the acceleration sensor 216 in real time image information obtained through the camera module 212. The location-based information received from 130 may be synthesized and displayed on the display unit 250. For example, as shown in FIG. 5, when the mobile terminal 110 is fixed to a vehicle to receive location-based services, the display module 250 of the mobile terminal 110 includes a camera module 212. The real-time image information obtained through the display is overlapped with the vehicle, and the speed information of the vehicle is displayed, and the main building information such as the gas station is overlapped.

That is, the location-based service according to an exemplary embodiment of the present invention not only uses the current location information of the portable terminal 110 but also uses the direction information to determine the direction of the portable terminal 110 as well as the current location information. The region of the terminal 110 may be limited in a direction desired. That is, the portable terminal 110 can know the direction information through the gyro sensor 214 or the acceleration sensor 216, and can display a map of the direction that the portable terminal 110 is directed to, so that the user can see the actual environment. The perception rate can be improved by comparing with the map. In addition, it is possible to receive a desired service without displaying the map information displayed on the display unit 250 of the portable terminal 110. For example, in the case of a directions service, the service may be provided by displaying only a simple icon such as an arrow in the direction in which the user should go.

In addition, apart from the positional information and the direction information, since the high performance central processing unit and the high resolution camera module 212 are mounted in accordance with the development of the technology of the portable terminal 110, due diligence input through the camera module 212 By combining image information with location and direction information, a more accurate and intuitive location-based service can be provided. In other words, the existing map information is displayed using the location and direction information and the desired service is used.However, the location and direction information can be used to identify the approximate location and the facility information such as the surrounding buildings and the camera. By comparing the real-time image information input according to the movement of the user may be able to identify the service desired by the user and provide related information.

In addition, as shown in Figure 5, the location-based service according to an embodiment of the present invention should be different in the size of the area to transfer the information according to the location of the portable terminal (110). If the location of the mobile terminal is located at a considerably high place such as a mountain peak, the area (or desired area) displayed through the camera module 212 of the mobile terminal 110 is displayed in a wide area (or an area located far away). Will be On the other hand, when the location of the portable terminal 110 is surrounded by a large building, an area (or an area required by the user) displayed through the camera module 212 of the portable terminal 110 is displayed as a narrow area (or at a short distance). Limited area).

Therefore, the location-based server 130 determines the size of the map area of the area to be delivered by identifying the surrounding environment of the location of the mobile terminal 110 from the map DB based on the location information received from the mobile terminal 110 and the corresponding area. Receives information about the DB from the various portable terminal 110 to be delivered. In this case, since the image input through the camera module 212 is displayed through the portable terminal 110 having a limited size, it may be limited to the amount of information that can be displayed by being synthesized and displayed on the real-time image. However, considering that the amount of things that can be accurately identified through the camera module 212 toward the general distance is limited, the location-based server 130 is a portable terminal 110 as it is received from the DB server 140 as it is. The information to be transmitted is determined according to the importance of the information without being transmitted to it. Therefore, the environment information retrieval unit 340 of the DB server 140 (or the location-based server 130) must inform the location-based service server of the importance of the information along with the information, unlike the conventional location-based service.

In the above description, it is described that all the components constituting the embodiments of the present invention are combined or operated in one, but the present invention is not necessarily limited to these embodiments. In other words, within the scope of the present invention, all of the components may be selectively operated in combination with one or more. In addition, although all of the components may be implemented in one independent hardware, each or all of the components may be selectively combined to perform some or all functions combined in one or a plurality of hardware. It may be implemented as a computer program having a. Codes and code segments constituting the computer program may be easily inferred by those skilled in the art. Such a computer program may be stored in a computer readable storage medium and read and executed by a computer, thereby implementing embodiments of the present invention. The storage medium of the computer program may include a magnetic recording medium, an optical recording medium, a carrier wave medium, and the like.

In addition, the terms "comprise", "comprise" or "having" described above mean that the corresponding component may be included, unless otherwise stated, and thus excludes other components. It should be construed that it may further include other components instead. All terms, including technical and scientific terms, have the same meanings as commonly understood by one of ordinary skill in the art unless otherwise defined. Terms commonly used, such as terms defined in a dictionary, should be interpreted to coincide with the contextual meaning of the related art, and shall not be construed in an ideal or excessively formal sense unless explicitly defined in the present invention.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

As described above, the present invention provides a method for providing a location-based service using augmented reality to continuously receive location-based information according to a real-time image that changes according to a user's movement, and a system, a server, and a portable terminal for the same. It is a very useful invention that generates an effect that is easy to grasp the surrounding location through the location-based services synthesized in the real-time image compared to the general map display method from the user's point of view.

1 is a block diagram schematically illustrating a location based service system using augmented reality according to an embodiment of the present invention;

2 is a block diagram schematically illustrating a portable terminal according to an embodiment of the present invention;

3 is a block diagram schematically illustrating a location based server according to an embodiment of the present invention;

4 is a flowchart illustrating a location-based service method using augmented reality according to an embodiment of the present invention;

5 is an exemplary view of a location based service using augmented reality according to an embodiment of the present invention.

<Description of Symbols for Main Parts of Drawings>

110: portable terminal 120: wired and wireless communication network

130: location-based server 140: DB server

210: location-based measurement unit 220: location-based input unit

230: terminal control unit 250: display unit

260: terminal communication unit 310: server communication unit

320: region determination unit 330: server control unit

340: environmental information search unit

Claims (12)

A server communication unit interworking with the portable terminal; An area determiner configured to determine a map area size including a current location of the portable terminal based on the location information received from the portable terminal; An environment information search unit for searching main surrounding environment information included in the determined map area size; And The server controller transmits the location-based information to the portable terminal by applying some information of the found main surrounding environment information to the map information corresponding to the determined map area size according to the determined area size. Server comprising a. The method of claim 1, The environment information search unit, And transmits the importance information according to the found main surrounding environment information to the server controller, and the server controller applies some of the main surrounding information to the location-based information based on the received importance information. server. The method of claim 1, The area determiner, Based on the latitude and longitude coordinate values included in the location information, the current location information of the portable terminal is extracted, and the altitude information of the current location information and area type information corresponding to the current location information are included in at least one or more information. And determine the map area size based on the size of the map area. The method of claim 1, The environment information search unit, And the main surrounding environment information is searched for in at least one or more of tourist information, shopping information, advertisement information, place name information, and address information. The method of claim 1, The server communication unit is interlocked with the DB server, And the area determining unit determines the size of the map area by using the map information received from the DB server. The method of claim 1, The server communication unit is interlocked with the DB server, And the environment information retrieval unit retrieves the main surrounding information based on the data received from the DB server. A display unit displaying an execution screen of a running application; A terminal communication unit interworking with a location-based server; A location based measuring unit obtaining location based related information; A location-based input unit configured to generate location information, real-time image information, and direction information using the obtained location-based related information; And Transmitting the location information to the location-based server, receiving location-based information corresponding to the location location information from the location-based server, and synthesizing the real-time image information, the direction information, and the received location-based information. Terminal control unit to display through the display unit A portable terminal comprising a. The method of claim 7, wherein The location-based measuring unit, A camera module for acquiring the real time image information; A gyro sensor for detecting movement; An acceleration sensor for sensing acceleration; And GPS module to acquire navigation data included in radio signals Portable terminal comprising a. The method of claim 8, The location based input unit, An image input unit configured to receive and transfer the obtained real-time image information; A direction measuring unit configured to generate the direction information based on the detected movement and the sensed acceleration; And Position measuring unit for generating the position information including the latitude-longitude coordinate value based on the obtained navigation data A portable terminal comprising a. The method of claim 7, wherein The terminal control unit, The real-time image information and the received location-based information is processed according to the direction information, and through the pattern matching of the real-time image information and the received location-based information to the object of the received location-based information on the real-time image information An image output control unit for recognizing and extracting and transmitting a position value of the object when synthesizing the location-based information with the real-time image information; A location information control unit configured to receive the location-based information by controlling the terminal communication unit, and control to display only information preset by a user among main surrounding environment information included in the location-based information; And An image processor for synthesizing the real-time image, the direction information, and the received location-based information according to a position value of the object, and displaying the synthesized image on the display unit; A portable terminal comprising a. Determine a map area size including a current location based on the location information, retrieve main surrounding information included in the determined map area size, and retrieve some information from the found main surrounding information according to the determined area size Location-based server for transmitting the applied location-based information; A portable terminal for transmitting the location information to the location-based server, receiving the location-based information from the location-based server, and displaying the real-time image information by combining direction information and the location-based information; And Wired and wireless communication network for relaying data transmitted and received between the portable terminal and the location-based service server Location-based service providing system using augmented reality, characterized in that it comprises a. Determining, by the location-based server, a map area size including the current location of the terminal based on the location information received from the portable terminal; Retrieving, by the location-based server, main surrounding environment information included in the determined map area size; Transmitting, by the location-based server, location-based information to which the partial information of the found main surrounding environment information is applied to the portable terminal according to the determined area size; And Receiving, by the portable terminal, the location-based information, and synthesizing and displaying the real-time image information and the direction information acquired on the received location-based information; Location-based service providing method using augmented reality comprising a.

Images