KR20190087267A - Apparatus and method for automatically recognizing local information through focusing around a user - Google Patents

Abstract

The present invention relates to a method for automatically recognizing area information. The method for automatically recognizing area information comprises the steps of: (a) extracting candidate geographic feature objects corresponding to a search area set to be visually displayed on a user terminal of geographic feature objects stored in an object storage unit of an apparatus for automatically recognizing area information based on location information and direction information obtained from the user terminal; (b) selecting visible geographic feature objects which can be visually identified in the search area of the candidate geographic feature objects; and, (c) recognizing a visible geographic feature object located closed to the center of the search area of the visible geographic feature objects as an area information request target.

Description

[0001] APPARATUS AND METHOD FOR AUTOMATICALLY RECOGNIZING LOCAL INFORMATION THROUGH FOCUSING AROUND A USER [0002]

The present invention relates to an apparatus and method for automatically recognizing a local area through focusing on a user's surroundings. That is, the present invention relates to an apparatus and method for automatically recognizing local information that can automatically recognize local information about a focused object by focusing an object (a feature object) around the user by using the local information automatic recognition apparatus.

As mobile devices become more popular and communication technologies are developed, there is an increasing demand for information retrieval / providing services that can provide convenience for living by utilizing them

Generally, there is a method of searching information by inputting a keyword to be searched in a search engine such as Google or Naver. However, this method has a problem that information can not be retrieved when the keyword is not known.

In order to solve this problem, there is a service for searching and providing surrounding information based on the location of the user.

FIG. 1 is a diagram illustrating an example of providing an information search service based on an Augmented Reality (AR).

Referring to FIG. 1, an information search service based on an Augmented Reality (AR) does not require a separate keyword, and information (for example, POI information) around the user is retrieved using the location information of the user, Overlayed on top.

Conventionally known AR-based information search services use only the location of a user in displaying information. As a result, AR-based information search services not only display information in a place where the user is difficult to move but also display information superimposed on one screen without distinction of nearby information and distant information, Information) is difficult to recognize easily.

As an example, one of the AR apps, Pokemon GO, uses the user's location to show the location of a virtual Pokemon on the map. And when you encounter that Pokemon, it shows the Pokemon according to the direction of the user looking on the camera image. However, since Pokemon GO does not consider the features of real life space, Pokemon appears to float in the air regardless of objects in real life space.

In addition, one of the AR apps, Sekai Camera, displays a virtual memo left on a specific location by the user on the camera screen. However, this also shows a memo in a place that should not be seen because it is covered by buildings or obstacles in real life space.

As described above, the conventional AR-based service overlays information on the image obtained through the camera by utilizing only the location information of the user without taking the user's surrounding environment into consideration at all. In other words, the conventional AR-based service provides information to the air regardless of the position of the object, provides information on a screen not to be hidden by the building or the obstacle as well as information not covered by the building or the obstacle, There is a problem that it is difficult to easily recognize the information desired by the user because the information is provided on one screen without distinguishing the information from the far information.

In addition, AR (Mixed Reality) service technology that displays information by using the positional relationship and the shape of an actual spatial object in the visible region of the user has been studied.

MR service technology mainly utilizes depth sensor and image processing technology for object extraction and provides better matching information in real life space than AR. However, such conventional MR service technologies have good results in an indoor space such as a room or office where the sensing range is limited, but they are difficult to apply in an outdoor environment where the ambient environment changes greatly.

In addition, since the conventional MR service technology recognizes an object based on image processing, the recognition rate of the object is affected by the surrounding environment such as light, and it is difficult to recognize objects that are partially hidden by the front building have.

That is, in order to enable outdoor MR service, accurate location information of outdoor entities is required. However, since conventional MR services are mainly based on indoor services, it is difficult to recognize outdoor objects with a conventional MR service technology that uses a depth sensor or characterizes indoor objects using image minutiae .

The background technology of the present application is disclosed in Korean Patent Laid-Open Publication No. 10-2012-0075609.

The present invention has been made to solve the above-mentioned problems of the conventional art, and it is an object of the present invention to provide an image processing apparatus and method capable of automatically recognizing information desired by a user (i.e., local information on a focused object) And an apparatus and method for automatically recognizing local information through focusing on the user's surroundings.

It is to be understood, however, that the technical scope of the embodiments of the present invention is not limited to the above-described technical problems, and other technical problems may exist.

According to another aspect of the present invention, there is provided a method for automatically recognizing a local area through a user's peripheral focusing using an automatic local area information recognizing apparatus, comprising the steps of: (a) Extracting candidate feature objects corresponding to a search area visually displayed on the user terminal among the feature objects stored in the object storage unit of the automatic area recognition device based on the direction information; (b) selecting visible feature objects visually recognizable within the search area among the candidate feature objects; And (c) recognizing a visible feature object located closest to the center of the search area among the visible feature objects as an area information request object.

In the step (b), the candidate feature objects are sequentially searched in the search area in the order close to the position corresponding to the position information to select the visible feature objects, and the selected visible feature objects The width area occupied in the search area can be excluded from the search range from the next search.

The object storage unit may store information on the position and shape of the feature objects as object information, and the width area corresponding to the selected visible feature object may include the selected visible feature object object in consideration of the object information. And may be an area corresponding to an object width appearing in a state projected on the search area.

In addition, the shape information may be stored in a MBR (Minimum Bounding Rectangle) format.

In addition, the information on the shape may be stored in a MAR (Minimum Area Rectangle) form.

In addition, the MAR type is a minimum area of all cases in which an MBR form is formed based on a local axis including a line connecting two adjacent vertexes among vertexes contained in an outline of a feature object included on a digital map Lt; / RTI > may be in a corresponding form.

In the step (a), if it is determined that there are a plurality of feature objects whose entire widths are all exposed in the search area through the image analysis for the search area, A plurality of candidate feature objects corresponding to positions and directions of each of the plurality of feature objects on the basis of the feature objects stored in the object storage unit, It is possible to judge whether there is a mismatch between the set of object information-based width ratios calculated corresponding to the object width appearing in the projected state.

In the step (a), when the position information is determined to be inconsistent, the position information is changed within the error range of the position sensor of the user terminal while the direction information is kept constant, Information-based width ratio set corresponding to the object-information-based width-ratio set corresponding to the object-information-based width-ratio set, and if there is an error in the positional information, Can be corrected.

In addition, a method for automatically recognizing a local area through focusing on a user using a local area automatic recognition apparatus according to an embodiment of the present invention may include: (d) extracting a point of interest corresponding to a local information request object recognized in step (c) POI (Point of Interest) information on the area information request object.

Meanwhile, according to an embodiment of the present invention, an apparatus for automatically recognizing a local area through focusing on a user's perimeter includes a feature information storage unit for storing the feature information stored in the object storage unit of the local information automatic recognition apparatus, An extracting unit for extracting candidate feature objects corresponding to the search area set to be visually displayed on the user terminal; A sorting unit for sorting the visible feature objects visually recognizable within the search area among the candidate feature objects; And a recognition unit for recognizing a visible feature object located closest to the center of the search area among the visible feature objects as a region information request object.

In addition, the selector may select the visible feature objects sequentially by searching the candidate feature object objects sequentially in the search area in the order close to the position corresponding to the position information, The width area occupied within the search area can be excluded from the search range from the next search.

The object storage unit may store information on the position and shape of the feature objects as object information, and the width area corresponding to the selected visible feature object may include the selected visible feature object object in consideration of the object information. And may be an area corresponding to an object width appearing in a state projected on the search area.

In addition, the shape information may be stored in a MBR (Minimum Bounding Rectangle) format.

In addition, the information on the shape may be stored in a MAR (Minimum Area Rectangle) form.

In addition, the MAR type is a minimum area of all cases in which an MBR form is formed based on a local axis including a line connecting two adjacent vertexes among vertexes contained in an outline of a feature object included on a digital map Lt; / RTI > may be in a corresponding form.

If it is determined that there are a plurality of feature objects whose entire widths are all exposed in the search area through the image analysis of the search area, the extracting unit extracts the plurality of feature objects A plurality of candidate feature objects corresponding to positions and directions of each of the plurality of feature objects based on the feature objects stored in the object storage unit are projected onto the search area, Based width ratio set calculated corresponding to the object width shown in FIG.

If it is determined that the mismatch occurs, the extracting unit may change the position information within the error range of the position sensor of the user terminal while keeping the direction information constant, And if it is determined that there is another object information based width ratio set corresponding to the image analysis based width ratio set, the position information is corrected by considering that there is an error in the position information .

In addition, according to an embodiment of the present invention, an apparatus for automatically recognizing a local area through focusing on a user's perimeter includes a point-of-interest (POI) information corresponding to a region- And a display control unit for displaying the overlay on the display unit.

The above-described task solution is merely exemplary and should not be construed as limiting the present disclosure. In addition to the exemplary embodiments described above, there may be additional embodiments in the drawings and the detailed description of the invention.

According to an aspect of the present invention, there is provided an apparatus for automatically recognizing a local feature of a user, the apparatus comprising: Can be automatically recognized.

According to an aspect of the present invention, there is provided a method of searching for a local information request object (in other words, a search object ) Can be effectively recognized (identified).

However, the effects obtainable here are not limited to the effects as described above, and other effects may exist.

FIG. 1 is a diagram illustrating an example of providing an information search service based on an Augmented Reality (AR).
FIG. 2 is a diagram illustrating a schematic configuration of an apparatus for automatically recognizing a local area through focusing on a user's perimeter according to an embodiment of the present invention.
FIG. 3 is a diagram for explaining a search area that can be set in the automatic area recognition apparatus through focusing on the user's perimeter according to an embodiment of the present invention.
FIG. 4 is a diagram illustrating an example of a search area visually displayed on a user terminal in an automatic local area recognition apparatus for focusing on surroundings of a user according to an embodiment of the present invention.
FIG. 5 is a diagram for explaining an MBR type (a) and a MAR type (b) considered in an automatic local area recognition apparatus through user's peripheral focusing according to an embodiment of the present invention.
FIG. 6 is a diagram for explaining a process of selecting a visible feature object in the apparatus for automatically recognizing a local area through focusing on the user's perimeter according to an embodiment of the present invention.
FIG. 7 is a diagram schematically illustrating a configuration of an automatic local area recognition system including an automatic local area information recognizing apparatus according to an embodiment of the present invention. Referring to FIG.
8 is a diagram schematically illustrating a structure of an outdoor MR model applied to an automatic local area information recognition system according to an embodiment of the present invention.
FIG. 9 is a view for explaining an R-tree structure, which is a storage structure of object information in the automatic area recognition apparatus according to an embodiment of the present invention.
10 is a diagram illustrating an example of searching POI information about a local information requesting object in the automatic local area information recognizing apparatus according to an embodiment of the present invention.
11 is a diagram illustrating an example of search information provided by the automatic area recognition apparatus according to an embodiment of the present invention.
12 is a diagram illustrating another example of search information provided by the automatic area recognition apparatus according to an embodiment of the present invention.
FIG. 13 is a diagram illustrating another example of search information provided by the automatic area recognition apparatus according to an embodiment of the present invention.
FIG. 14 is a diagram for explaining an example in which the determination of whether or not an error exists in the location information of the user terminal, which is obtained by the automatic local area information recognizing apparatus according to an embodiment of the present invention, is manually performed by a user's judgment.
FIG. 15 is a diagram illustrating an example of manually changing the location information of a user terminal in a device for automatically recognizing local information according to an embodiment of the present invention by user input.
16 is a diagram for explaining an example in which the apparatus determines whether or not an error exists in the location information of the user terminal acquired by the automatic local area information recognizing apparatus according to an embodiment of the present invention.
17 and 18 are views for explaining correction of location information of a user terminal in an automatic local area information recognizing apparatus according to an embodiment of the present invention.
FIG. 19 is a flowchart illustrating a method of automatically recognizing a local area through focusing on a user using an automatic local area information recognizing apparatus according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. It should be understood, however, that the present invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In the drawings, the same reference numbers are used throughout the specification to refer to the same or like parts.

Throughout this specification, when an element is referred to as being "connected" to another element, it is intended to be understood that it is not only "directly connected" but also "electrically connected" or "indirectly connected" "Is included.

It will be appreciated that throughout the specification it will be understood that when a member is located on another member "top", "top", "under", "bottom" But also the case where there is another member between the two members as well as the case where they are in contact with each other.

Throughout this specification, when an element is referred to as "including " an element, it is understood that the element may include other elements as well, without departing from the other elements unless specifically stated otherwise.

FIG. 2 is a diagram illustrating a schematic configuration of an apparatus for automatically recognizing a local area through focusing on a user's perimeter according to an embodiment of the present invention. Hereinafter, an apparatus 10 for automatically recognizing a local area through focusing around the user according to an embodiment of the present invention will be referred to as the apparatus 10 for convenience of explanation.

2, the apparatus 10 may include an extracting unit 11, a sorting unit 12, and a recognizing unit 13. In addition, the present apparatus 10 may include a display control unit 14 and an object storage unit 15.

The present apparatus 10 may be a device included (or installed, embedded, or included) in the user terminal 1.

Here, the user terminal 1 is a mobile communication device that is guaranteed to be portable and mobility, for example, a Personal Communication System (PCS), a Global System for Mobile communication (GSM), a Personal Digital Cellular (PDC) System, a PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication) -2000, Code Division Multiple Access (CDMA) -2000, W-CDMA (WCode Division Multiple Access), Wibro (Wireless Broadband Internet) Smartphones, smart pads, tablet PCs, notebooks, wearable devices, and the like.

The extraction unit 11 visually extracts the feature objects stored in the object storage unit 15 of the device 10 from the user terminal 1 based on the location information and the direction information obtained from the user terminal 1. [ The candidate feature object objects corresponding to the search area set to be displayed can be extracted.

The search area can be set based on the information obtained using the plurality of sensors included in the user terminal 1. [ The search area may be set to have an area similar to the user ' s view. This can be more easily understood with reference to FIG.

FIG. 3 is a diagram for explaining a search area that can be set in the automatic area recognition apparatus through focusing on the user's perimeter according to an embodiment of the present invention.

3, the search area s includes the position information of the user terminal 1 acquired from the position sensor (position acquisition sensor) included in the user terminal 1, and the position information of the user terminal 1 acquired from the gyro sensor , The accelerometer, the magnetometer, and the compass (electronic compass). Here, the direction information may particularly refer to information about a direction in which the image sensor 1a included in the user terminal 1 is looking, and may be expressed by azimuth information. Also, the position sensor may be, for example, a GPS sensor, but is not limited thereto. Further, the image sensor 1a may be represented by a camera, an image acquiring device, a photographing device, or the like.

)를 이용하여 획득 가능한 이미지센서(1a)의 시야각(α), 및 인식거리를 더 고려하여 설정될 수 있다. 여기서, 시야각 α는 2*atan(

/2d)의 조건을 만족할 수 있다. 여기서, 인식거리는 미리 설정된 거리로 설정될 수 있다. 예시적으로, 인식거리는 사용자의 시야를 고려하여 미리 설정된 거리로서 사용자 단말(1)의 위치로부터 일예로 200m 이내의 거리로 설정될 수 있으며, 이에 한정되는 것은 아니다.In addition, the search area s may be determined based on the focal length d of the lens of the image sensor 1a included in the user terminal 1 and the width of the image sensor 1a

, The viewing angle alpha of the image sensor 1a obtainable using the angle?, And the recognition distance. Here, the viewing angle alpha is 2 * atan (

/ 2d) can be satisfied. Here, the recognition distance may be set to a predetermined distance. Illustratively, the recognition distance may be set to a predetermined distance in consideration of the user's view, for example, within a distance of 200 m from the position of the user terminal 1, but is not limited thereto.

As described above, the search area s can be set in consideration of the position information, the direction (azimuth) information, the viewing angle, and the recognition distance of the user terminal 1.

FIG. 4 is a diagram illustrating an example of a search area s visually displayed in the user terminal 1 in the automatic local area recognition device through user's peripheral focusing according to an embodiment of the present invention.

Referring to FIG. 4, the search area s may be an area that is visually displayed on the screen of the user terminal 1 and may include a photographing area, a recognition area recognizable through the image sensor of the user terminal 1, . ≪ / RTI >

Illustratively, when a user wishes to know information of a specific feature object, the user may select a particular feature object to be displayed in the search area set to be visually displayed on the user terminal 1, You can focus on a specific feature object you want to know.

The extracting unit 11 may extract candidate feature objects corresponding to the search region s set to be visually displayed on the user terminal 1 among the feature objects stored in the object storage unit 15. [ Alternatively, the extracting unit 11 may extract the feature objects located in the search region s among the feature objects stored in the object storage unit 15 as candidate feature objects.

The object storage unit 15 may store information on the location and shape of the feature objects as object information. Here, the object information may include the keyword and the phase information of the feature object. In addition, the feature object is a fixed object that does not move, and can be, for example, a structure such as a building, a building, a bridge, a bridge, a bulletin board, a large sign, a monument, and the like. Hereinafter, for convenience of description, the feature object refers to a building, for example.

For example, in the object storage unit 15, information on the shape of the feature objects may be stored in the form of a Minimum Bounding Rectangle (MBR). In another example, the object storage unit 15 may store information on the shape of the feature objects in a MAR (Minimum Area Rectangle) format. Here, the MAR form is a minimum area of all cases in which the MBR is formed based on the local axis including the line connecting two adjacent vertexes among the vertexes included in the contour of the feature object included in the digital map May refer to a form corresponding to the case. A more detailed description can be more easily understood with reference to Fig.

FIG. 5 is a diagram for explaining an MBR type (a) and a MAR type (b) considered in an automatic local area recognition apparatus through user's peripheral focusing according to an embodiment of the present invention.

Referring to FIG. 5, since the feature objects (e.g., buildings) are composed of a plurality of vertexes, the object storage unit 15 may be, for example, Information about the shape of water objects can be stored in the form of a Minimum Bounding Rectangle (MBR). Here, the MBR type may mean a rectangle (minimum bounding rectangle) having a minimum size that can include all vertexes of the feature object.

In this case, when the information about the shape of the feature objects is stored in the MBR format, a lot of empty space is generated. Therefore, when the information of the feature objects stored in the MBR format is used in extracting the candidate feature objects in the extraction unit 11, the MBR regions do not overlap with the actual building regions, May not be achieved. That is, when the information about the shape of the feature objects is stored in the form of MBR as shown in FIG. 5 (a), for example, the buildings located in the search area during the search for the building B are searched together, Can fall.

Therefore, in the object storage unit 15, it may be preferable that the information on the shape of the feature objects is stored in the MAR format, which is a modification of the MBR. The MAR type may mean a rectangle (minimum area rectangle) shape having a minimum area that can include all the vertexes of the feature object among the MBR shapes

MAR can be obtained through the following process. To obtain the MAR, the angle between the two vertexes of the vertexes included in the contour of the feature object is first obtained, and the area (width) of the MBR can be obtained by rotating the x-axis or the y-axis. Next, the MBR is obtained for all cases of adjacent vertexes among the vertexes included in the contour of the feature object irrespective of the order, and the MBR of the case having the smallest area is obtained. Next, the MAR can be obtained by rotating the MBR having the minimum area by the rotation of the MBR again.

As shown in FIG. 5 (b), when the information about the shape of the feature objects is stored in the MAR format, the empty space can be eliminated as compared with the MBR, so that the search for the feature object is performed more accurately than the MBR .

When the location information and the direction information are obtained from the user terminal 1, the apparatus 10 transmits information on the location and shape of the feature objects located within a predetermined distance range from the location of the user terminal 1, And store information about the shape of the obtained feature objects in the object storage unit 15 in the form of a MAR, for example. Here, the predetermined distance range may mean a range within 500 m from the position of the user terminal 1 by way of example, but not limited thereto, and may be variously set.

According to this, the server may store information on the position and shape of all the feature object objects as object information, and the object storage unit 15 of the present device 10 may store information Information about the position and shape of the feature objects located within a predetermined distance range from the position of the user terminal, for example) can be obtained from the server and stored.

The object storage unit 15 may store information related to the location and shape of the feature objects as object information, in association with points of interest (POI) information corresponding to the object information. Here, the point of interest information can be obtained from the server. According to this, the database of the server may have object information (information on the position and shape of the feature objects) constructed in the R-tree structure as an example and information of the corresponding point of interest (mapping). A more detailed description thereof will be described later.

The selector 12 can select the visible feature objects visually recognizable within the search area s among the candidate feature objects extracted by the extractor 11. [ This can be more easily understood with reference to FIG.

FIG. 6 is a diagram for explaining a process of selecting a visible feature object in the apparatus for automatically recognizing a local area through focusing on the user's perimeter according to an embodiment of the present invention. In other words, FIG. 6 is a diagram for explaining a process of resetting a search target range for selection of a visible feature object in the search area s (in other words, a search algorithm for a visible feature object).

Referring to FIGS. 6A and 6B, a candidate feature object corresponding to the search region s extracted by the extraction unit 11 is assigned to a first candidate feature object O1, Object 1 , A second candidate feature object O2, Object 2, and a third candidate feature object O3, Object 3 may be included.

The selector 12 can select the visible feature objects O1 and O2 visually recognizable within the search area s among the candidate feature objects O1, O2 and O3.

At this time, the selector 12 sequentially searches the candidate feature object objects in the order of the position corresponding to the position information of the user terminal 1 (i.e., the position of the user terminal, pos) in the search area s The visible feature object is selected, and the width area occupied by the selected visual feature object in the search area (s) can be excluded from the search range from the next search.

Here, the width area corresponding to the selected visible feature object is displayed by projecting the selected visible feature object into the search area in consideration of the object information stored in the object storage unit (i.e., information on the position and shape of the feature objects) Quot; area " corresponding to the object width appearing in the state.

In other words, the selector 12 may first search for the selection of the visible feature object by setting the entire search area s as the first search object range s. At this time, the selector 12 selects the candidate feature object O1 closest to the position pos of the user terminal among the candidate feature objects O1, O2, and O3 located in the first search target range s. For example, as the first visible feature object.

After the selection of the first visible feature object O1 in the first search object range s is made, the selector 12 selects the first visible feature object O1 in the search area s It is possible to perform a search for selection of a visible feature object with respect to a search target range excluding the width area occupied by the first visible feature object O1 and excluding the width area occupied by the first visible feature object O1. That is, the selector 12 can reset the search target range after the first visible feature object O1 is selected in the first search target range s. At this time, the reset search target range (i.e., the second search target range, s1) is set such that the selected first visible feature object O1 among the first search target ranges s corresponding to the entire search region s is searched And may mean an area s1 excluding a width area occupied in the area s.

Accordingly, after the selection of the first visible feature object O1 is performed, the selector 12 can perform a search for selection of the visible feature object with respect to the second search object range s1. As a result of the search for the second search target range s1, the selector 12 selects the candidate feature objects O3 located in the second search target range s1 closest to the position pos of the user terminal The candidate feature object O3 can be selected as the second visible feature object as an example.

The selection unit 12 then excludes a width area occupied by the selected second visible feature object O3 in the search area s in the second search object range s1 from the search object range, It is possible to perform a search for selection of the visible feature object for the search object range excluding the width area occupied by the water object O3. In this case, if the search area s is set as shown in (a) of FIG. 6, for example, the selector 12 can perform a search for selection of a visible feature object object three times.

6 (a), the selector 12 searches the first candidate feature object O1 and the third candidate feature object O1 among the candidate feature objects O1, O2, and O3 through three searches, The water object O3 can be selected as a visible feature object visually recognizable in the search area s. Since the second candidate feature object O2 is an invisible object hidden by the first candidate feature object O1 when viewed from the position pos of the user terminal, the selector 12 selects the second candidate feature object O2, The object O2 is not selected as the visible feature object.

The selection process of the visible feature objects by the selector 12 will be described again. FIG. 6B shows an example in which a building is located as a first candidate feature object O1 in the search area s.

6 (c), after the extraction of the candidate feature object objects by the extraction unit 11, the selector 12 selects the visible feature objects that are visually recognizable among the extracted candidate feature objects Can be selected. At this time, the selector 12 selects the visible feature objects using the following search algorithm so as not to recognize the feature objects that are hidden by the preceding feature object (e.g., building) .

The selector 12 can search for the visible feature object in the order of the candidate feature objects close to the position pos of the user terminal among the candidate feature object objects located in the search area s. If a selection is made for one visible feature object by searching, the selector 12 selects a portion of the candidate feature object closest to the pos of the user terminal (i.e., And a search process for a visible feature object in the re-search target area can be repeatedly performed.

The selecting unit 12 performs a process of resetting the search target area and a search process of the re-set search target area until all of the candidate feature objects among the candidate feature objects included in the search area s are selected Repeat can be done.

, r)로 이루어져 있다고 하자. 산출부(12)는 검색 영역(s)을 탐색 대상 영역으로 하여 가시 지형지물 객체에 대한 1차 탐색을 수행함으로써, 탐색 대상 영역 내에 위치한 제1 후보 지형지물 객체(O1)를 가시 지형지물 객체로서 선별할 수 있다.As a specific example, if the search area s is ( pos ,

, r ). The calculating unit 12 performs the first search on the visible feature object using the search area s as the search target area so that the first candidate feature object O1 located in the search target area is displayed as the visible feature object Can be selected.

At this time, it can be confirmed that the portion (a, b) corresponding to the selected first candidate feature object O1 occupies a part of the visually recognizable region in the search region s. In other words, it can be seen that the width areas ( pos , c , d ) corresponding to the selected visible feature object O1 in the search area s occupy a part. Here, the width area corresponding to the selected visible feature object O1 is displayed in a state where the selected visible feature object O1 is projected on the search area s in consideration of the object information stored in the object storage unit 15 May refer to regions ( pos , c , d ) corresponding to the appearing object widths ( a , b ).

, c) 영역과 (pos, d, r) 영역이 포함될 수 있다.The selection unit 12 performs a primary search to select one visible feature object, and then selects one of the selected visible feature objects among the search target areas corresponding to the search area s subjected to the primary search resetting the corresponding width region (pos, c, d) with the exception of the area that the O1) as a search target area for the second search, and to perform a second search for the visible feature object with respect to the reset search target area . Here, in the search target area that is the secondary search target, that is, the reset search target area, ( pos ,

, c ) region and ( pos , d , r ) region may be included.

, r)이 재귀(Recursion) 인자로 설정되고, Recursion(

, r)에서 검색 영역(s)의 좌표를 인자로 제공할 수 있다. 이를 통해, 선별부(12)는 탐색 알고리즘을 통해 검색 영역(s)에 대하여 건물들의 포함 관계를 판단할 수 있다.The search algorithm will be described as follows. The selection unit 12 may perform a search algorithm for selection of a visible feature object. In the search algorithm,

, r ) is set as the recursion factor, and Recursion (

, r ) of the search area s as a factor. Accordingly, the selector 12 can determine the inclusion relation of the buildings with respect to the search area s through the search algorithm.

,c)과 Recursion(d,r)이 수행될 수 있다. 또한, 검색 영역(s)에 대하여 건물이 포함되지 않은 경우, 일예로 본 장치(10)의 디스플레이 제어부(14)는 건물이 포함된 영역에 대하여 재촬영(재인식)이 이루어지도록 하기 위한 알림을 사용자 단말(1)로 제공할 수 있다. 또한, 검색 영역(s)에 대하여 건물이 검색 영역(s)보다 큰 경우, 해당 건물이 가시 지형지물 객체로서 탐색된 이후 탐색 알고리즘이 종료될 수 있다.At this time, in the case of the search algorithm, if the building is completely included in the search area (s) or the building is partially included,

, c ) and Recursion ( d , r ) can be performed. When the building is not included in the search area s, for example, the display control unit 14 of the present device 10 notifies the user that the area including the building should be re-photographed (re- And can provide it to the terminal 1. Also, when the building is larger than the search area s with respect to the search area s, the search algorithm may be terminated after the building is searched as a visible feature object.

Using the search algorithm, the selector 12 can select the visible feature objects visually recognizable within the search area s among the candidate feature objects.

The recognizing unit 13 searches the visible feature information object located closest to the center of the search area s among the visible feature information objects selected by the selector 12 to a local information request object (in other words, Object to be searched).

When a user wants to know information about a target of interest (for example, a specific feature object), it is common to focus the interest to be positioned at the center of the screen (the image display screen displayed on the user terminal) The recognition unit 13 can recognize the visible feature object that is located closest to the center of the search area among the visible feature objects as the area information request object. According to the present invention, the present invention can provide a convenience of allowing a user to directly select a desired building (object to be searched) through focusing of a nearby building using the user terminal 1. [

6 (a), the recognition unit 13 selects the visible feature objects O1 and O3 closest to the center of the search region S among the visible feature objects O1 and O3 selected by the selector 12 It is possible to recognize the visible feature object O3 located as a target of the local information request.

In the following description, the local information request target recognized by the recognition unit 13 may be represented by a search target object.

In the embodiment of the present invention, the recognizing unit 13 recognizes the visible feature object located closest to the center of the search area among the selected visible feature objects as a local information request object (search object) It is not.

In another example, the display control unit 14 may display a center identification mark at the center of the screen of the user terminal 1. [ Thereafter, the user is allowed to display the center identification mark displayed on the screen of the user terminal 1 on the feature object (that is, the object to be searched, for example) The feature object to be searched by the image sensor of the user terminal can be displayed. At this time, the recognition unit 13 can recognize the feature object corresponding to the center identification mark among the feature objects displayed on the user terminal as a local information request object (search object). According to another embodiment of the present invention, the object to be searched can be more easily recognized (identified) by using the center identification mark.

The display control unit 14 can overlay POI (Point of Interest) information corresponding to the local information request object recognized by the recognition unit 13 on the local information request object. The display control unit 14 can overlay the point of interest information corresponding to the request for the local information on the local information request object displayed on the screen of the user terminal 1. Here, the point of interest information may include, for example, at least one of a name (name) and an address of a point of interest. However, the present invention is not limited thereto.

The object storage unit 15 may store information on the location and shape of the feature objects as object information. In addition, the object storage unit 15 may associate POI information corresponding to the object information together. Here, the object information stored in the object storage unit 15 and the corresponding POI information may include information on some of the object information stored in the database of the server and corresponding POI information (for example, Object information located within the range and corresponding POI information). The description thereof can be more easily understood through the following description.

7 is a diagram schematically showing a configuration of an automatic local area recognition system 100 including an automatic local area recognition apparatus 10 according to an embodiment of the present invention.

Referring to FIG. 7, the automatic local area recognition system 100 according to an embodiment of the present invention may include a local area automatic recognition apparatus (apparatus 10) and a server 20.

Here, the automatic local area recognition apparatus 10 may mean the present apparatus 10 described above. Therefore, even if the following description is omitted, the description of the apparatus 10 described above can be applied to the description of the apparatus 10 (automatic local area information recognizing apparatus) to be described later.

The present apparatus 10 can transmit location information and direction information obtained from a user terminal to the server 20. [ In the database of the server 20, information on the position and shape of all the feature object objects may be stored as object information, and in addition, interest point information corresponding to the object information may be associated (mapped) and stored.

Accordingly, when the server 20 acquires the location information and the direction information of the user terminal from the apparatus 10, the server 20 sets the location information and direction information of the user terminal from the location corresponding to the location information of the user terminal Object information located within the distance range and corresponding POI information can be retrieved from the database. Thereafter, the server 20 can transmit the object information and POI information corresponding thereto, which are located within the predetermined distance range, to the present apparatus 10. [

The device 10 may then store object information and POI information corresponding to the objects within the predetermined distance range received from the server 20 in the object storage unit 15 of the device 10. [ The apparatus 10 then extracts a candidate feature object through the extraction unit 11 based on the information stored in the object storage unit 15 and selects and recognizes a visible feature object through the selection unit 12 13), and displaying the POI information through the display control unit 14.

And may be connected between the present apparatus 10 and the server 20 via the network 30. [

The network 30 may be, for example, a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, a World Interoperability for Microwave Access (WIMAX) network, (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), a Bluetooth network, an NFC (Near Field Communication) network, a satellite broadcasting network, an analog broadcasting network, a DMB (Digital Multimedia Broadcasting) But is not limited thereto.

On the other hand, the Ambient Browser can be regarded as an MR system that can obtain information of an object by illuminating an object in a real-life space desired to be searched by an image sensor of a user terminal. The core of this system is that the outdoor objects to be searched can be specified without keywords, and it is difficult to utilize the conventionally known MR research based on the sensor / image processing in constructing such a system.

In other words, as described above, the conventional MR service is mainly based on the provision of the service in the indoor space, and thus there is a limit to applying the conventional MR service in the outdoor environment where the environmental change is large.

In this embodiment, the present system 100 is capable of searching for a target object (i.e., a target object) by using the position information and the direction information of the user terminal based on the real space spatial object information of the digital map, And the local information requested by the department). In addition, we propose a real space spatial location based outdoor MR model for effective construction of outdoor MR service based on this. In the following description, a local information request object recognized by the recognition unit 13 may be referred to as a search object, as a target object to be searched by the user.

That is, the present invention provides point of interest information of a search object (that is, a local information request object recognized by the recognition unit) on the basis of the digital map and the POI information by utilizing the outdoor MR model proposed in the present application And to develop an Ambient Browser which is an outdoor MR service. In other words, the present apparatus 10 may be expressed as an ambient browser device for providing an outdoor MR service. The description of the outdoor MR model proposed here can be more easily understood with reference to Fig.

FIG. 8 is a diagram schematically illustrating the structure of an outdoor MR model applied to the automatic local area information recognition system 100 according to an embodiment of the present invention. The process of providing point of interest information based on the outdoor MR model will be described below.

Referring to FIG. 8, the outdoor MR model proposed in the present application may include a real-life spatial layer, an object storage spatial layer, an information storage spatial layer, and an information display spatial layer.

In the outdoor MR model, object information extracted from the digital map is stored in an object storage space layer, and is used to construct a search object (an outdoor object to be searched), and based on a specific search object, (I.e., POI information) related to the search target object that is constructed in advance. A more detailed description follows.

[Real space space layer]

The real-life space corresponding to the real-life space layer may mean a space in which the user lives. The user can receive the information providing service in the real space through the information display space layer. Here, the information display space layer represents a display screen on the user terminal 1, and a detailed description will be described later.

When the user who owns the user terminal 1 wants to know information about a certain building (a feature object) while moving in the real life space, the user can use the image sensor of the user terminal 1 to illuminate the building have. That is, the user can focus the corresponding building using the meaningful sensor of the user terminal 1 so that the building (object to be searched) to know the information is visually displayed on the screen of the user terminal 1.

In this case, when the feature 10 is focused on the user terminal 1 for a predetermined period of time, for example, the feature information is displayed on the screen of the user terminal 1, The image containing the object can be recognized as a search request for searching the search object (i.e., for recognizing the local information request object).

That is, when it is determined that the user terminal 1 has focused the feature object for a preset time, the apparatus 10 recognizes the feature object as a search request and acquires the position information and direction information from the user terminal 1 have. To this end, the apparatus 10 may include a position information and direction information obtaining unit (not shown).

Here, the predetermined time for determining whether to recognize the search request may be, for example, 3 seconds, but is not limited thereto.

When the search request is recognized, the present device 10 searches the location information (the latitude and longitude coordinates) of the user terminal 1 and the user terminal 1 from the GPS sensor, electronic compass, gyro sensor, Direction information (azimuth information) can be acquired. The present device 10 can transmit the obtained location information and direction information of the user terminal 1 to the server 20. In particular, the location information and the direction information of the user terminal 1 transmitted from the present apparatus 10 can be transmitted to the object storage space portion corresponding to the object storage space layer in the server 20. [ In other words, in the real-life spatial layer, the location information and direction information (position + direction) of the user terminal 1 can be transferred to the object storage space layer.

Here, the server 20 may include a database, and the database may include an object storage space corresponding to the object storage space layer and an information storage space corresponding to the information storage space layer. In the object storage space, information on the position and shape of all the feature objects is stored as object information (the object information includes the keyword and phase information of the feature object), and in the information storage space, Point of interest (POI) information may be stored. According to this, the database of the server 20 may store the object information and the corresponding point of interest corresponding to each other (mapped).

[Object storage space layer]

The object storage space corresponding to the object storage space layer may be a space for storing the information of the objects (feature objects) in the real space space based on the digital map information (geographic information) rather than simply the location information. The information of the objects (feature objects) in the real space may be stored in the database of the server 20, in particular, the object storage space. In other words, the object storage space may store the object information extracted from the digital map in which the location information of various feature objects exist in the real space. That is, the object storage space part can be constructed using the object information of the digital map.

In the object storage space, information on the position and shape of all the feature objects may be stored as object information. At this time, the object information is converted into MBR and stored in the R-tree structure in the object storage space part. However, the present invention is not limited to this, and object information may be converted into MAR and stored in an R-tree structure in the object storage space portion. The description related to the MBR and the MAR has been described in detail above, and the following description will be omitted.

FIG. 9 is a diagram for explaining an R-tree structure that is a storage structure of object information in the automatic area recognition apparatus 10 according to an embodiment of the present invention.

Referring to FIG. 9, the R-tree structure may be a structure of tree data for storing spatial information about feature objects. Here, the spatial information can be represented by object information. Accordingly, the spatial information may refer to object information including information on the position, shape, and phase of feature objects.

In the object storage space, the object information can be stored in an R-tree structure in the form of an MBR. That is, all the object information expressed by the MBR may be stored (constructed) in the server 20 in an R-tree structure. Based on this, the present inventor can more quickly perform searching for the existence of buildings around the user (i.e., searching for buildings existing in the vicinity of the location corresponding to the location information of the user terminal).

By way of example, an R-tree structure may be constructed such that the R1 region includes 1,2,3 feature objects, and the R2 includes 4,5,6 feature objects. At this time, if the search range is as shown in FIG. 9, the server 20 confirms (identifies) that 3, 4, or 6 feature objects exist in the search range through search based on the R-tree structure .

For example, the object information is stored in the object storage space of the server 20 in the MBR format, and the object information stored in the object storage unit 15 of the apparatus 10 may be stored in the MAR format. However, the present invention is not limited thereto. For example, the object information may be stored in the object storage space in the MAR format.

Using the location information and the direction information of the user terminal 1 received from the real-life spatial layer, it is possible to search for objects in the surrounding area where the user is located based on the object information stored in the R-tree structure in the object storage space Can be searched (identified) more quickly than if they existed. In the database of the server 20, the object information stored in the R-tree structure and the corresponding POI information may be mapped and stored.

Accordingly, the server 20 may use the position information and the direction information of the user terminal 1 received from the apparatus 10 to determine the position of the user terminal 1 within the predetermined distance range from the position of the user terminal It is possible to identify the object information (ie, information about the location and shape of the feature objects) using the R-tree more quickly. In other words, the present invention can provide an object storage space as an object information DB (otherwise expressed as a building information DB) that can respond more quickly to a query related to building search by using the property of Tree and the MBR relation relation.

The server 20 may then provide the apparatus 10 with the object information identified in the object storage space and the point of interest information corresponding to the object information identified in the information storage space described below.

In the object storage space layer, information (position + direction) received from the real space space layer and the identified object information can be transmitted to the information storage space layer. Here, the identified object information may include keyword and phase information (i.e., building keyword + phase information) of the feature objects (e.g., buildings). In addition, the object information may include information about the position and shape of the feature object.

[Information storage layer]

The information storage space corresponding to the information storage space layer may be a space in which POI information corresponding to the object information in the object storage space is stored based on the position based on the resin map information have. This point of interest information may be stored in the database of the server 20, particularly in the information storage space.

For example, only POI information corresponding to object information is stored in the information storage space, but the present invention is not limited thereto. In another example, the information storage space may store not only information having a public nature such as POI but also information directly written by the user on the object information.

The information (POI information) stored in the information storage space section may be indexed by location information of the feature object and keyword information. In other words, POI information may be stored in the information storage space. At this time, the POI information may be indexed with keywords of the feature object stored in the object storage space based on the position (latitude and longitude coordinates) of the feature object, and stored in the information storage space.

Therefore, according to the information storage space unit, the POI information of the corresponding feature object can be identified (retrieved) by using the object information identified in the object storage space layer. In other words, when the object information is identified using the (position + direction) information in the object storage space, the server 20 can identify (search) the POI information corresponding to the object information identified in the storage space. That is, the server 20 identifies object information located within a predetermined distance range from the position of the user terminal using the object storage space, identifies (searches) POI information corresponding to the identified object information using the information storage space, can do.

In the information storage space layer, information (position + direction) received from the object storage space layer, building keyword + phase information as identified object information, and information (i.e., POI information) corresponding to the identified object information Layer. In other words, when the server 20 acquires (receives) the positional information and directional information of the user terminal 1 from the present device 10, the server 20 determines, based on the database including the object storage space and the information storage space, (Object information and POI information corresponding thereto) from the position corresponding to the position information of the terminal, and identifies the POI information corresponding to the object information, (Transmission) to the base station.

Here, the object storage space portion is represented by a building information DB, and the information storage space portion can be represented by a POI DB. For example, in the present example, the building center coordinates in the POI DB are inquired from the building information constructed by the R-tree in the object storage space, and the building name (name) in the POI DB is mapped The database of the server 20 can be constructed.

In other words, after the R-tree is constructed, the POI DB can be constructed by querying the building center coordinates of the POI DB to the R-tree and mapping the building information (object information) and the POI as a result. Thereafter, when the server 20 receives the location information and the direction information of the user terminal through communication with the device 10, the server 20 provides the object information about the user and the corresponding POI information to the user terminal using the R-tree . The present apparatus 10 performs a search for a building desired by a user based on object information provided from the server 20 and corresponding POI information and transmits POI information about the searched building to the user terminal 1, On the screen of FIG.

The database of the server 20 will be described as follows. The database may store object information (building information) and corresponding POI information. The object information (building information) may be composed of the vertex coordinates of the building. One object information may store all the coordinates of a vertex of a building. The corner coordinates of a building may consist of building latitude / longitude information. In addition, the information about the shape of the object in relation to the object information can be formed of polygons (polygons) as the starting point and the ending point of the vertex coordinates of the building are always the same.

The POI information may be configured in the form of NID, Ncode, name_full, Name_alias, Name_field, Xpos, Ypos, Admcode_A, Admname_A, Admcode_L, admname_L, LANDMARK, Codename, TOP_POI with respect to the POI information stored in the database . Also, the POI information can be used as the main data by way of example, a building name (name), building center coordinates, and building address.

According to this, the object information (building information) constructed by using the R-tree and the corresponding POI information can be mapped and stored in the database, and the building vertex and the building name (name) are linked (connected) .

[Information display space layer]

The information display space corresponding to the information display space layer may refer to a space displayed on the screen of the user terminal 1. [ That is, the information display space layer is a layer for connecting the information that is identified (searched) in the information storage space and the real space, and may be a screen of the user terminal 1. The user terminal 1 may include all kinds of devices capable of visually displaying information such as a portable terminal, a smart device, a head mounted display (HMD), and a digital signage.

Specifically, after providing the location information and the direction information of the user terminal to the server, the apparatus 10 responds to the location information and the direction information of the user terminal in a predetermined distance range from a location corresponding to the location information of the user terminal identified by the server Information and POI information corresponding thereto can be obtained. The present apparatus 10 can store information (object information and corresponding POI information) obtained from the server 20 in the object storage unit 15 of the present apparatus 10. [ The apparatus 10 then uses the extracting unit 11, the selecting unit 12 and the recognizing unit 13 to determine whether or not the local information request target (s) of the feature objects contained in the search area displayed on the user terminal (The object to be searched). At this time, assuming that the object to be searched for by the user is the closest building to the center of the screen of the user terminal 1, the present device 10 may be provided with the feature objects included in the search area displayed on the user terminal (That is, specify the object to be searched) to which POI search the user desires. The apparatus 10 then transmits interest point information corresponding to the recognized local information request object (object to be searched) based on the information stored in the object storage unit 15 to a region displayed on the screen of the user terminal 1 It can be overlaid on the information request subject.

According to this, the present apparatus (10) to the present system (100) can provide the POI information identified based on the location information and the direction information of the user terminal (1). Particularly, the present apparatus 10 to the present system 100 allow the user in the real space to share the POI information about the search object, which the user wants to recognize, only by illuminating the search object with the user terminal 2 Can be provided through the screen of the user terminal 1 so as to be recognized.

The apparatus 10 comprehensively considers the location information and direction information of the user terminal, the phase information of the object, and the like, and displays the POI information about the search object to be searched by the user on the search object, .

The present invention can provide an outdoor MR-based service based on an outdoor MR model constructed on the basis of a real-life spatial location capable of searching a POI of a search object without a keyword.

FIG. 10 is a diagram illustrating an example of searching POI information for a local information request object (object to be retrieved) in the automatic local area information recognition apparatus 10 according to an embodiment of the present invention. 10 is a diagram showing an example of a screen that is displayed on the screen of the user terminal 1 by the present apparatus 10. In FIG.

Referring to FIG. 10, a screen corresponding to a search area s recognized (viewed) by the image sensor may be displayed in the first area S1 of the user terminal 1. [

A map in which the position on the map corresponding to the search area s is displayed may be displayed in the second area S2 of the user terminal 1. [

Illustratively, the square (L1) of the red border displayed on the map of the second area S2 represents the MAR of the POI in the real world. The red triangle L2 indicating the various POIs displayed on the map of the second area S2 can be used to intuitively identify the visually recognizable visible feature objects by the selector 12 in the search area . The blue rectangle L3 among the various POIs pointed to by the red triangle L2 represents a local information request object recognized by the recognition unit 13 (i.e., a search object).

A first UI (User Interface) icon may be displayed in the third area S3 of the user terminal 1 so that the user can select (determine) whether to display the map on the screen of the user terminal 1 have. The screen size of the first area S1 may be set to be large or small depending on whether the map is displayed by the first UI icon.

A second UI icon may be displayed in the fourth area S4 of the user terminal 1 to allow the user to select (determine) whether to start searching POIs. For example, if the user inputs a second UI icon in the form of a search start button and the POI search starts, the fourth region S4 may be changed to a search stop button icon to stop the search for the POI . For example, after the search for the POI is completed, an icon of the search start button may be displayed so that the search can be resumed. If the user input is performed, the POI can be rescanned. The POI search can identify (identify) a search object (local information request object) to be searched by the user among the in-screen feature objects recognized through the image sensor.

After the POI search is completed, the POI information corresponding to the search object may be overlaid on the search object recognized (identified) on the screen recognized by the image sensor. Here, the POI information may include at least one of a name (name) and an address.

Illustratively, when a user illuminates a particular feature object for a predetermined amount of time (e.g., 3 seconds) using the image sensor of the user terminal 1, the device 10 regards this as a search request, It is possible to recognize (identify) the object to be searched in the image recognized by the sensor and to provide corresponding POI information.

In the embodiment of the present invention, the apparatus 10 displays the POI information only for one local information request object (search request object) recognized by the recognition unit 13, but the present invention is not limited thereto. Alternatively, the apparatus 10 may display POI information corresponding to each of the visible feature objects selected by the selector 12 according to the user setting. In another example, the present apparatus 10 displays POI information corresponding to each of the object information stored in the object storage unit 15 (i.e., the feature objects located within a predetermined distance range from the position of the user terminal) can do. Whether POI information is displayed for the visible feature objects and whether POI information is displayed for the object information stored in the object storage unit 15 can be selectively made based on user input.

The display control unit 14 of the present apparatus 10 displays the point of interest information corresponding to the local information request object recognized by the recognition unit 13 and then displays a user input (e.g., a touch It is possible to display search information searched based on the displayed point of interest information (e.g., POI name, POI address, etc.) on the screen of the user terminal 1. Here, the displayed search information may be information retrieved through a search engine as search information corresponding to interest point information on which user input is made. Illustratively, the search engine may refer to a search engine such as Google, Naver, or the like, but is not limited thereto. Through the provision of such search information, the present apparatus 10 can provide more detailed information on the POI information identified and recognized through the present apparatus 10. [

11 is a diagram illustrating an example of search information provided by the automatic area recognition apparatus 10 according to an embodiment of the present invention.

Referring to FIG. 11, a user performs focusing on a building to be searched among buildings (feature objects) existing in a real life environment through an image sensor of the user terminal 1 as shown in FIG. 11 (a) . In this case, the apparatus 10 recognizes (identifies) a focused building to be searched by the user among the image buildings recognized through the image sensor, and recognizes the focused focused building (that is, The POI information corresponding to the request destination can be displayed on the screen of the user terminal 1 as shown in Fig. 11 (b). For example, when the focused building is the 'Leaning Tower of Pisa', the display control unit 15 displays the POI information of the building focused on the building (displayed on the screen of the user terminal 1) Leaning Tower of Pisa 'can be overlaid and displayed.

11 (b), when the display control unit 15 determines that the POI information displayed on the user terminal 1 is user input (for example, selection input using touch or the like), the display control unit 15 displays The search information retrieved based on the POI information can be displayed on the screen of the user terminal 1. [ As shown in FIG. 11C, when the user input is made to the 'Leaning Tower of Pisa', which is the POI information displayed on the user terminal 1, the display control unit 15 displays search information for 'Leaning Tower of Pisa' Can be displayed on the screen.

FIG. 12 is a diagram illustrating another example of search information provided by the automatic local area recognition apparatus 10 according to an embodiment of the present invention.

12, when the user focuses a building to be searched using the user terminal 1 as shown in (a) of FIG. 12, ) Can display information such as 'Aeronautical University lecture hall' as POI information of the focused building on the screen of the airline. Thereafter, when user input is made to the displayed POI information (e.g., Aerial University of Korea Aerospace University), the present apparatus 10 searches for search information (search) based on the 'Aeronautical University Lecture Information' Result information) on the screen of the user terminal 1. Fig.

FIG. 13 is a diagram illustrating another example of search information provided by the automatic area recognition apparatus 10 according to an embodiment of the present invention.

13, when a user focuses a building to be searched using a user terminal as shown in (a) of FIG. 13, the present apparatus 10 displays the screen of the user terminal 1 as shown in (b) It is possible to provide the POI information of the focused building on a pop-up window. For example, the display control unit 15 of the present apparatus 10 can provide POI information such as "the information of the searched building is the Lotte Maverization vertex (Hwajungro 66, Doyang-gu, Goyang-si, Korea)"

Next, when it is determined that the user input for the provided POI information confirmation is made and the user confirmation of the POI information is made, the display control unit 15 searches the detailed information about the POI information shown in (c) of FIG. A search engine selection menu may be provided. Illustratively, the search engine selection menu may include a selection menu such as "search by Naver", "search by next", "search by Google".

In this case, when the user input for the 'search with Naver' is performed, the display control unit 15 displays a Naver search result using the POI information as shown in FIG. 13 (d) as the search information for the displayed POI information, Can be displayed on the screen of the display device (1).

The present apparatus 10 to the present system 100 can provide a local information search capable of easily searching POIs around the user by simply focusing the surrounding POIs existing in the real-life space with the image sensor of the user terminal Technology can be provided.

In the present invention, a new geographical information infrastructure can be constructed through the object storage space and the information storage space, thereby providing more information considering the user's surroundings. The present system 100 proposed in the present application can expand the scale of the infrastructure with extensibility that can be extended if only object information (building information) and POI data exist. Also, according to the present invention, it is possible to utilize a system specialized for POI information by using the constructed geographic information infrastructure.

In addition, this paper proposes an outdoor MR model based on the actual location of real - life spatial objects, where the location of real - life spatial objects can be extracted using a digital map. In addition, the present invention can provide an Ambient Browser capable of searching for POI information only by illuminating a building in a real-life space to a user terminal without keywords, based on the outdoor MR model proposed in the present application.

According to the outdoor MR model proposed in the present application, the location and range of the real-life space that the user desires to search can be specified using the position of the user terminal and the direction information of the user terminal. Then, based on the specified position and range information, an object in the object storage space layer can be searched for the object at the corresponding position. In the information storage space layer, information (POI information) at the corresponding location, that is, POI information about the objects retrieved from the object storage space layer, can be retrieved. Thereafter, the object information retrieved from the object storage space layer and the information storage space layer and the corresponding POI information correspond to the positional relationship between the two pieces of information (object information and POI information) in the information display space and the position information and direction information of the user terminal And can be displayed on the screen. The information (for example, POI information) thus displayed can be recognized by the user in the real-life space through the user terminal.

We can provide Ambient Browser, a mobile MR system based on digital map with Ambient Intelligence concept. The present invention enables the user to receive information about a building in a real-life space in which information is to be obtained, by illuminating the building with an image sensor of the user terminal without a keyword.

This paper proposes an outdoor MR model based on real-life space location for outdoor MR-based services, considering that conventional MR research considers indoor use. In this paper, we showed that the object storage space, which is the core of the outdoor MR model, is constructed using the digital map. In addition, the present application shows an actual application example of the Ambient Browser (illustratively, see Figs. 10 to 13) as an example of the service based on the outdoor MR model.

Meanwhile, in the present apparatus 10, the extracting unit 11 may correct position information obtained from the user terminal 1 before extracting the candidate feature objects. A more detailed description of the correction of the position information is as follows.

If the location information obtained from the user terminal 1 is GPS information, the GPS has an error range of about 5 m, which may be larger or smaller depending on the surrounding environment. The error of the location information may cause an incorrect POI search to be performed on the present apparatus 10 that performs the POI search based on the location information and the direction information of the user terminal. Accordingly, the apparatus 10 corrects the error of the position information obtained from the user terminal 1 by using the object information (building information) extracted from the digital map and the image processing so that a more accurate POI search can be performed And can provide a POI search based on the corrected location information.

If the extracting unit 11 determines that there are a plurality of feature objects whose entire widths are all exposed in the search area through the image analysis on the search area, An analysis-based width ratio set and a plurality of candidate feature objects corresponding to the positions and directions of each of the plurality of feature objects based on the feature objects stored in the object storage unit, And determine whether there is a mismatch between the set of object information-based width ratios calculated.

Here, the width ratio set may mean, for example, a ratio with respect to the width of each of the buildings. As another example, a width ratio set can mean the ratio of not only the width of each building but also the width of space between buildings. In other words, a width ratio set can mean a ratio of the width of each of the buildings and the width of the space between each building.

In addition, the image analysis-based width ratio set may mean a width ratio set calculated through image analysis on an image displayed on the screen of the user terminal 1. [ Also, the object information based width ratio set may mean a width ratio set calculated based on the object information (building information) extracted from the digital map. Such a set of object information-based width ratios may include a plurality of sets of object information-based width ratios that are calculated according to the search positions for a plurality of candidate feature objects.

When it is determined that the determination result is inconsistent, the extraction unit 11 changes (shifts) the position information within the error range of the position sensor of the user terminal while keeping the direction information obtained from the user terminal constant, Width rate set and other object information based width rate sets corresponding to the set. That is, when it is determined that the determination result is inconsistent, the extracting unit 11 extracts, from the plurality of candidate information object-based width ratio sets calculated for the plurality of candidate feature information objects, You can search the rate set.

Here, the change of the position information may mean a change by a shift movement that is moved without changing the orientation information. Further, the change of the position information within the error range can be automatically performed by the present apparatus 10, but is not limited thereto, and can be performed manually by user input. In addition, the presence or absence of an error as to whether or not there is an error in the positional information obtained from the user terminal may be automatically performed by the apparatus 10, or may be manually performed by the user's judgment. This can be more easily understood through the following description.

The extracting unit 11 may correct the position information obtained from the user terminal because there is an error in the position information if it is determined that there exists the other object information based width ratio set corresponding to the image analysis based width ratio set. That is, if it is determined that there exists an object information-based width ratio set corresponding to the image analysis-based width ratio set, the extracting unit 11 determines that there is an error in the position information, Based width ratio set and other object information based width ratio sets corresponding to the set can be corrected to the calculated search position.

The description of the position information correction can be more easily understood with reference to the following drawings.

FIG. 14 is a diagram for explaining an example in which the determination of whether or not an error exists in the location information of the user terminal, which is obtained by the automatic local area information recognition apparatus 10 according to an embodiment of the present invention, is manually performed by the user . In other words, FIG. 14 shows a case in which a user (for example, a building A) that the user actually desires to search for and a building (for example, a building B) For example.

14, pos 1 in FIG. 14 indicates a position corresponding to the position information of the user terminal 1 acquired by the present apparatus 10, that is, an example of the position coordinates of the user terminal 1 recognized by the present apparatus 10 . pos 2 represents an example of the positional coordinates of the actual user terminal 1. Also, pos 1-f (focus) represents a central line of sight recognized in pos 1 (i.e., a line of sight corresponding to the center of the screen), and pos 2-f represents a central line of sight recognized by pos 2.

Referring to FIG. 14, for example, when a user desires to perform a POI search (search) for building A, the user may select a POI from the user terminal 1 so that the building A is located at the central portion on the screen of the user terminal 1 The image sensor can be focused toward the A building. Accordingly, in the entire screen area S1 of the user terminal 1, the A building can be visually displayed so as to be located at the center.

At this time, one area S2 of the screen of the user terminal 1 may be provided with a map showing the position information and the direction information of the user terminal recognized by the apparatus 10. [

At this time, if there is an error in the position information of the user terminal recognized by the present apparatus 10, the position of the user terminal appearing on the map may be displayed as a position having an error different from the position (pos 2) of the actual user. That is, in the position of the user with an error appearing on the map, it can be recognized that the building B is located in the central line of sight.

According to this, when accurate retrieval is performed, the present device 10 should display POI information of A building as POI information about the object to be retrieved on the screen of the user terminal 1. However, in the example shown in FIG. 14, since the object to be searched is erroneously recognized due to the existence of the error of the acquired position information of the user terminal, the apparatus 10 can display the POI information of the B building in error.

As described above, in the present invention, the user can search the user terminal 1 by using a map provided in one area S2 of the user terminal 1 and an image recognized through the image sensor provided in the full screen area S1 of the user terminal 1 (1) whether or not there is an error with respect to the positional information acquired from the positional information acquiring unit.

FIG. 15 is a diagram illustrating an example of manually changing the location information of the user terminal in the automatic local area information recognition apparatus 10 according to an embodiment of the present invention by user input.

Referring to FIG. 15, (a) of FIG. 15 shows a case where a position of a user terminal displayed on a map is recognized through an image sensor of the user terminal due to an error of the position information of the user terminal Indicates a case where the position of the actual user terminal corresponding to the video is different from the position of the actual user terminal corresponding to the video.

In this case, the user performs a user input such as dragging the position coordinate (pos 1) corresponding to the position information displayed on the map of the user terminal 1 through a screen touch, for example, Similarly, the position information can be corrected.

In another example, the user can correct the position information through user input to the position information adjustment icon (Q) displayed in one area of the screen of the user terminal (1). At this time, the position information displayed on the map can be changed in real time by user input to the position information adjustment icon (Q). By the position information adjustment icon Q, the position information can be moved upward or downward, for example, with reference to the drawing. However, the present invention is not limited thereto, and the positional information can be moved in any one of the 360-degree directions by the positional information adjustment icon (Q).

By the user input, manually corrected position coordinates (pos 1 ', i.e., the position coordinates of the actual user terminal) can be displayed on the map as shown in FIG. 15 (b). In this case, the present apparatus 10 can perform POI search (search) based on the corrected position information as shown in FIG. 15 (b). That is, the positional information of the user terminal 1 recognized by the present apparatus 10 can be corrected by manually fine-tuning it by user input, for example.

16 is a flowchart for explaining an example in which the apparatus 10 automatically determines whether there is an error in the location information of the user terminal acquired by the automatic local area information recognizing apparatus 10 according to an embodiment of the present invention FIG.

16, pos 1 represents an example of a position corresponding to the position information of the user terminal 1 acquired by the present device 10, that is, an example of the position coordinates of the user terminal 1 recognized by the present device 10. pos 2 represents an example of the positional coordinates of the actual user terminal 1.

16, the extracting unit 11 performs an image analysis on an image corresponding to the search region s displayed on the user terminal 1, as shown in FIG. 16A, The set can be calculated. In this case, the image analysis based width ratio set calculated from the image displayed on the user terminal 1 may mean a width ratio set calculated at the position of the actual pos 2. In other words, the image displayed on the user terminal 1 in FIG. 16A may represent an image recognized at the position of pos 2.

In addition, the extracting unit 11 may calculate an object information-based width ratio set for a plurality of candidate feature objects based on the feature objects stored in the object storage unit 15. [ In this case, the calculated object information-based width ratio set may be, for example, a set of width ratios calculated at position pos 2 in FIG. 16 (b).

At this time, the extracting unit 11 may determine whether the set of the image analysis-based width ratio calculated from the image displayed on the user terminal 1 and the set of the object information-based width ratio calculated at the pos 2 position are inconsistent.

The extracting unit 11 changes the position of pos 1 to the first position pos 1 ', the second position pos 1' ', etc. in a state in which the direction information is kept constant, ) While searching for other sets of object information-based width ratios corresponding to the image analysis-based width ratio sets. Here, the object information-based width ratio set calculated at the first position (pos 1 ') may mean another object information-based width ratio set corresponding to the image analysis-based width ratio set detected by the extracting unit 11 . Therefore, as a result of the search, since the different object information-based width ratio sets corresponding to the image analysis-based width ratio sets exist for the first position (pos 1 '), the extracting unit 11 extracts It is possible to correct the position information (pos 1) of the user terminal 1 recognized by the present apparatus 10 to the first position (pos 1 ') because there is an error in the position information.

At this time, for example, when the position of pos 1 is changed (shifted), the extracting unit 11 extracts the image information of the feature objects stored in the object storage unit based on the digital map, The window width of the image recognizable by the sensor can be changed (shifted) by a distance corresponding to one building width with respect to the lateral direction.

As described above, the present apparatus 10 is configured to perform the object information based on the image analysis-based width ratio set (differently, the building size ratio pattern on the image) calculated through the image analysis and the position of the user terminal recognized by the apparatus 10 It is possible to judge inconsistency by comparing the width ratio set (the pattern of the size ratio of the building to the position of the user terminal recognized by the apparatus 10 based on the numerical map in different manners). If the determination result is inconsistent, the apparatus 10 searches the object information based width ratio set corresponding to the image analysis based width ratio set by comparing the object information based width ratio set for the shifted position while shifting the position of the user terminal , And the position of the user terminal can be corrected to the detected position.

The present apparatus 10 can automatically determine whether there is an error in the location information of the user terminal through digital map and image analysis. At this time, various image recognition (analysis) algorithms exist or may be developed at the time of image analysis.

17 and 18 are diagrams for explaining correction of the location information of the user terminal in the automatic local area information recognizing apparatus 10 according to an embodiment of the present invention.

17 and 18, as shown in (a) of FIG. 17, the position of the user terminal 1 recognized through the apparatus 10 is equal to pos 1, and the position of the actual user terminal 1 Let the coordinates be the same as pos 2.

The extraction unit 11 can extract the contours of the buildings displayed on the screen as shown in FIG. 17 (b) by performing an image analysis on the images displayed on the screen of the user terminal 1.

17 (c), the extraction unit 11 extracts an image analysis-based width ratio set R1 (i.e., a building size ratio pattern on the image by different expressions) based on the contours of the buildings extracted through the image analysis, Can be calculated. At this time, for example, the extracting unit 11 may calculate the image analysis-based width ratio set R1 based on the vertical line of the outline.

17 (d), the extracting unit 11 can identify the search valid area that can be recognized within the error range in consideration of the error range of the position sensor of the user terminal 1. [ Then, the extracting unit 11 extracts, based on the plurality of candidate feature objects located in the search valid region, the position of pos 1 within the error range, You can perform a search for an information-based width ratio set.

At this time, the extraction unit 11 changes (shifts) the position of pos 1 in the error range to the right and left direction while keeping the direction information constant as shown in FIG. 18 (a) It is possible to search for the corresponding object information-based width ratio set R2.

If there is an object information information based width ratio set R2 corresponding to the search result image analysis base width ratio set R1 and there is an error in the position information pos 1 obtained from the user terminal 1, The controller 11 compares the position information pos 1 of the user terminal 1 recognized by the apparatus 10 with the image analysis based width ratio set R 1 as shown in FIG. 18 (c) Can be corrected to a position pos 1 'at which acquisition of another object information-based width ratio set R2 is performed.

As described above, the present apparatus 10 can perform POI search (search) based on the position information of the corrected user terminal, thereby providing more accurate POI information.

Hereinafter, the operation flow of the present invention will be briefly described based on the details described above.

FIG. 19 is a flowchart illustrating a method of automatically recognizing a local area through focusing on a user using the automatic area recognition apparatus 10 according to an embodiment of the present invention.

The automatic local area recognition method shown in FIG. 19 can be performed by the local area automatic recognition apparatus 10 (this apparatus) described above. Therefore, even if the contents are omitted in the following description, the contents described for the automatic local area information recognizing apparatus 10 (the present apparatus) can be equally applied to the description of the automatic local area information recognizing method.

Referring to FIG. 19, in step S11, based on the location information and the direction information obtained from the user terminal, among the feature objects stored in the object storage unit of the local information automatic recognition apparatus, a search area set to be visually displayed on the user terminal The corresponding candidate feature objects can be extracted.

If it is determined in step S11 that there are a plurality of feature objects whose entire widths are all exposed in the search area through image analysis for the search area, An analysis-based width ratio set and a plurality of candidate feature objects corresponding to the positions and directions of each of the plurality of feature objects based on the feature objects stored in the object storage unit, And determine whether there is a mismatch between the set of object information-based width ratios calculated.

In step S11, when it is determined that the determination result is not consistent, while the direction information is kept constant, the position information is changed within the error range of the position sensor of the user terminal while the other object information Based width ratio set. Thereafter, in step S11, if it is determined that there is a different object information-based width ratio set corresponding to the image analysis-based width ratio set, it is possible to correct the position information because there is an error in the position information.

In step S11, the candidate feature object objects can be extracted based on the corrected position information and orientation information.

Next, in step S12, among the candidate feature objects extracted in step S11, the visible feature objects visually recognizable in the search area can be selected.

In step S12, the candidate feature objects are sequentially searched in the search area in the order close to the position corresponding to the position information, and the visible feature objects are selected. If the width of the selected visible feature object in the search area The region can be excluded from the search range from the next search.

The object storage unit considered in step S11 may store information on the position and shape of the feature objects as object information. Accordingly, the width area corresponding to the selected visible feature object considered in step S12 may be an area corresponding to the width of the object appearing when the selected visible feature object is projected in the search area in consideration of the object information have.

Here, for example, information on the shape in the object storage unit may be stored in a MBR (Minimum Bounding Rectangle) form. In another example, information about the shape in the object storage unit may be stored in a MAR (Minimum Area Rectangle) form.

The MAR shape is a minimum area of all the cases in which an MBR shape is formed based on a local axis including a line connecting two adjacent vertexes among the vertexes contained in the contour of the feature object included in the digital map May mean a corresponding form.

Next, in step S13, the visible feature object located closest to the center of the search area among the visible feature objects selected in step S12 may be recognized as the area information request object.

In addition, although not shown in the drawing, the method for automatically recognizing local information according to an embodiment of the present invention may further include the step of, upon receiving the point of interest (POI) information corresponding to the local information request object recognized in step S13, And displaying the overlay on the display screen.

In the above description, steps S11 to S13 may be further divided into further steps or combined into fewer steps, according to embodiments of the present application. Also, some of the steps may be omitted as necessary, and the order between the steps may be changed.

The method for automatically recognizing local information according to an embodiment of the present invention may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and configured for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

Further, the above-described method of automatically recognizing the area information may be implemented in the form of a computer program or an application executed by a computer stored in a recording medium.

It will be understood by those of ordinary skill in the art that the foregoing description of the embodiments is for illustrative purposes and that those skilled in the art can easily modify the invention without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included within the scope of the present invention.

1: User terminal
10: Local information automatic recognition device
11:
12:
13:
14:
15: Object storage unit

Claims (19)

A method for automatically recognizing a local area through focusing on a user using a local area automatic recognition apparatus,
(a) a candidate feature item corresponding to a search area set to be visually displayed on the user terminal among the feature objects stored in the object storage unit of the automatic localization device, based on the positional information and orientation information obtained from the user terminal Extracting objects;
(b) selecting visible feature objects visually recognizable within the search area among the candidate feature objects; And
(c) recognizing a visible feature object located closest to a center of the search area among the visible feature objects as a region information request object,
The method comprising the steps of: The method according to claim 1,
The step (b)
The candidate feature objects are sequentially searched in the search area in the order close to the position corresponding to the position information to select the visible feature objects, and the selected visible feature objects include a width area occupied in the search area The method of automatically recognizing local information, which is excluded from the search range from the next search. 3. The method of claim 2,
Wherein the object storage unit stores information on the location and shape of the feature objects as object information,
Wherein the width area corresponding to the selected visible feature type object is an area corresponding to an object width appearing when the selected visible feature type object is projected on the search area in consideration of the object information, Way. The method of claim 3,
Wherein the information on the shape is stored in an MBR (Minimum Bounding Rectangle) format. The method of claim 3,
Wherein the information on the shape is stored in a MAR (Minimum Area Rectangle) format. 6. The method of claim 5,
The MAR form is a minimum area of all the cases in which the MBR form is formed based on the local axis including the line connecting two adjacent vertexes among the vertexes included in the contour of the feature object included in the digital map Is a form corresponding to the local area information. The method of claim 3,
The step (a)
If it is determined through the image analysis of the search area that there are a plurality of feature objects whose entire widths are all exposed in the search area, the image analysis based on the plurality of feature objects A plurality of candidate feature objects corresponding to positions and directions of each of the plurality of feature objects based on the feature objects stored in the object storage unit; And determining whether there is a discrepancy between the set of object information-based width ratios calculated by the automatic matching method. 8. The method of claim 7,
The step (a)
When the position information is determined to be inconsistent, the position information is changed within the error range of the position sensor of the user terminal while the direction information is kept constant, and the other information information based width ratio Search the set,
Wherein the location information is corrected based on an error in the location information when it is determined that the image analysis based width ratio set and another object information based width ratio set corresponding to the image analysis based width ratio set exist. The method according to claim 1,
(d) overlaying the POI (Point of Interest) information corresponding to the local information request object recognized in the step (c) on the local information request object,
The method comprising the steps of: An apparatus for automatically recognizing a local area through focusing on a user's surroundings,
Based on the location information and direction information obtained from the user terminal, candidate feature objects corresponding to the search area set to be visually displayed on the user terminal among the feature objects stored in the object storage unit of the automatic local area information device are extracted And
A sorting unit for sorting the visible feature objects visually recognizable within the search area among the candidate feature objects; And
A recognition unit for recognizing a visible feature object located closest to the center of the search area among the visible feature objects as a region information request object,
A local information automatic recognition device. 11. The method of claim 10,
The sorting unit,
The candidate feature objects are sequentially searched in the search area in the order close to the position corresponding to the position information to select the visible feature objects, and the selected visible feature objects include a width area occupied in the search area And then excluded from the search target range from the next search. 12. The method of claim 11,
Wherein the object storage unit stores information on the location and shape of the feature objects as object information,
Wherein the width area corresponding to the selected visible feature type object is an area corresponding to an object width appearing when the selected visible feature type object is projected on the search area in consideration of the object information, Device. 13. The method of claim 12,
Wherein the information on the shape is stored in the form of a minimum bounding rectangle (MBR). 13. The method of claim 12,
Wherein the information on the shape is stored in a MAR (Minimum Area Rectangle) form. 15. The method of claim 14,
The MAR form is a minimum area of all the cases in which the MBR form is formed based on the local axis including the line connecting two adjacent vertexes among the vertexes included in the contour of the feature object included in the digital map Wherein the local area information is in a form corresponding to the area information. 13. The method of claim 12,
The extracting unit extracts,
If it is determined through the image analysis of the search area that there are a plurality of feature objects whose entire widths are all exposed in the search area, the image analysis based on the plurality of feature objects A plurality of candidate feature objects corresponding to positions and directions of each of the plurality of feature objects based on the feature objects stored in the object storage unit; And determining whether there is a mismatch between the set of object information-based width ratios calculated by the automatic determination unit. 17. The method of claim 16,
The extracting unit extracts,
When the position information is determined to be inconsistent, the position information is changed within the error range of the position sensor of the user terminal while the direction information is kept constant, and the other information information based width ratio Search the set,
Wherein the positional information correction unit corrects the positional information based on the presence of an error in the positional information when it is determined that the image analysis based width rate set and other object information based width rate sets corresponding to the set exist. 11. The method of claim 10,
A display controller for overlaying a point of interest (POI) information corresponding to the area information requested by the recognizing unit on the area information request object,
And an automatic local area information recognizing device. A computer-readable recording medium on which a program for executing the method of any one of claims 1 to 9 is recorded.

Images