JP2007150681A - Location specifying system and location specifying method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a location specifying system and location specifying method which certainly specify a location of a portable terminal in good precision. <P>SOLUTION: The location specifying system is provided with an image processing means 7 which recognizes information regarding to an indicator group from a photographed image of a portable terminal 1; an indicator data base 10 in which information regarding to the indicator group is registered beforehand; a data base search means 8 which searches a indicator group candidate from the indicator data base 10 based on information regarding the indicator group; a map data base 5 in which information regarding a landmark is registered beforehand; and an interaction processing means 9 which, while extracting information regarding to the landmark around the indicator group candidate from the map data base 5 based on the indicator group candidate, gives information regarding the landmark to the portable terminal 1 in the question format, and outputs location specifying information for specifying the location of the portable terminal 1 based on answer information from the portable terminal 1 side for the information regarding to the landmark shown in the question format. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a position specifying system and a position specifying method for specifying the position of a mobile terminal such as a mobile phone.

  Conventionally, as a position specifying system and a position specifying method for specifying the position of a mobile terminal such as a mobile phone, a hybrid positioning using GPS (Global Positioning System) or base station information of a mobile phone is used. Is generally known.

  However, this type of system and method may cause large positioning errors in urban areas and buildings due to factors such as a decrease in multipath and supplementary satellites. In fact, the applicant conducted a walking survey in Shinjuku, Shibuya and Shimokitazawa in July 2004 using a mobile phone equipped with a self-supporting GPS as a measurement terminal. It was confirmed that a large error of about 120 m at the maximum and several tens of m on average occurred in the downtown area and its surroundings. Thus, since the accuracy of GPS is not necessarily guaranteed in urban areas, there is a risk of confusion as to whether the current position indicated by GPS is really the correct location for the user. . In addition, although the accuracy of GPS itself is improved, it is difficult to completely eliminate these effects in the current GPS technology.

Therefore, in recent years, methods for reliably identifying the position of a mobile terminal have been studied. For example, a surrounding landscape is imaged with a mobile phone, and image data of the landscape and image data registered in a database in advance are recorded. Based on the above, a method for specifying the current position of the mobile phone has been proposed (see, for example, Patent Document 1 or 2).
JP 2003-111128 A JP 2004-191339 A

  However, since the conventional methods disclosed in Patent Documents 1 and 2 described above are methods for searching for similar images by comparing the color characteristics and the like of images captured by a mobile phone with images in a database, an enormous number of images Must be stored in the database, and the time required for the search becomes longer, resulting in increased system construction costs and introduction costs.

  The present invention has been made to solve the above-described problems, and can reduce the construction cost and introduction cost of the system and can accurately and accurately specify the position of the portable terminal. And it aims at providing a location specifying method.

  In order to achieve the above object, the present invention is a position specifying system for specifying a position of a mobile terminal, and an image for recognizing information related to a sign set which is a combination of road signs from an image taken by the mobile terminal. Processing means, a sign database in which information relating to a sign set is registered in advance, a database search means for searching for a sign set candidate from the sign database based on information related to the sign set recognized by the image processing means, Based on the map database in which information on marks is registered and the marker set candidates searched by the database search means, information on landmarks around the marker set candidates is extracted from the map database and the landmarks are related Information is presented to the mobile terminal in question format and presented in question format. Based on the response information from the portable terminal to the information about the landmark which is characterized in that it comprises a dialogue processing means for outputting the position specifying information for specifying the position of the mobile terminal.

  The database search means requests the portable terminal to shoot a second image in the vicinity of the shooting point of the first image when the number of extracted marker set candidates exceeds a predetermined number. And searching for a second marker set candidate from the marker database based on information about the second marker set recognized by the image processing means from the second image taken in response to the request, and the dialog processing means Performs grouping by combining the first marker group candidate and the second marker group candidate based on the first marker group candidate and the second marker group candidate of the first photographed image. The groups may be sorted in descending order of score calculated by a predetermined evaluation function.

  Further, the present invention is a position specifying method for specifying the position of a mobile terminal, wherein (A) recognizing information related to a sign set which is a combination of road signs from an image taken by the mobile terminal; (B) based on the information on the marker set recognized in the step (A), searching for a marker set candidate from a marker database in which information on the marker set is registered in advance; and (C) in the step (B) Based on the searched marker group candidates, information on landmarks around the marker group candidates is extracted from a map database in which information on landmarks is registered in advance, and information on the landmarks in question format Based on the response information from the mobile terminal to the information about the landmark presented in the question format. It can, characterized in that it includes a step of outputting the position specifying information for specifying the position of the mobile terminal.

  In the step (B), when the number of the extracted marker set candidates exceeds a predetermined number, the second image is shot in the vicinity of the shooting point of the first image with respect to the portable terminal. Requesting and searching for a second marker set candidate from the marker database based on information about the second marker set recognized from the second image taken in response to the request, and the step (C) includes: Based on the first marker set candidate and the second marker set candidate of the first photographed image, the first marker set candidate and the second marker set candidate are combined to form a group, There may be included a step of sorting the groups in descending order of score calculated by the evaluation function.

  Furthermore, the present invention is a position specifying program for causing a computer to execute the above steps.

  According to the present invention, since the location of the mobile terminal is specified using the existing infrastructure and database called road signs, the introduction cost of the system can be kept low. Also, compared to conventional methods such as landscape recognition, the recognition target is limited to only road signs, so that the processing based on pattern matching is completed, and advanced image processing is not required. Therefore, the cost for image processing can be kept low. Furthermore, since the direction in which the user is facing can be detected, navigation can be performed for the user by displaying the direction of travel superimposed on the captured image. Furthermore, by interacting with the user, the user can determine whether the displayed point is really correct, and the user can recognize the current location by grasping the surrounding environment. Various excellent effects can be obtained.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Here, FIG. 1 is a conceptual diagram showing a schematic configuration of the position specifying system according to the embodiment of the present invention.

  In general, road signs are information signs for the purpose of providing information such as directions and distances to destinations, famous points, and municipal boundaries, warning signs that give notice of dangerous locations such as intersections and railroad crossings, and road use It is defined as including a regulation sign indicating a regulation to be observed by a person and an instruction sign indicating a necessary point such as a pedestrian crossing. In addition, road signs are managed by road administrators and the Public Safety Commission, and the installation status is managed by road sign originals or road management ledgers, most of which are GIS (Global Information System). System) database management is promoted.

  In this way, road signs are different from chain store signs, etc., and the shape and installation standards are defined in detail throughout the country. Since these data are already in some form of database, new data can be collected. There is no need to do it. In addition, since there are few temporal changes such as buildings and signboards, and there are very few combinations of road signs in a specific range, it is suitable as a position marker. For this reason, the present applicant recalls that road signs are used as clues for position information among various feature points that overflow in the city, and defines the combination of road signs as a sign set.

The position specifying system according to the present embodiment is configured by connecting a processing server 2, a base station 3 of a mobile terminal 1, a base station registration database 4, and a map database 5 via a mobile terminal communication network 6. The processing server 2 is provided with an image processing unit 7, a database search unit 8, a dialogue processing unit 9, a sign database 10, and a management database 11.
Next, the position specifying method according to the embodiment of the present invention will be described with reference to FIGS. Here, FIG. 2 is a flowchart showing the procedure of the position specifying method according to the embodiment of the present invention, FIG. 3 is a conceptual diagram showing a method of grouping marker group candidates in the position specifying system, and FIG. 4 is a position specifying system. FIG. 5 is a diagram illustrating a landmark selection screen displayed on the mobile terminal in the position specifying system, and FIG. 6 is a screen displayed on the mobile terminal in the position specifying system. FIG. 7 is a diagram showing a navigation screen displayed on the portable terminal in the position specifying system.

  The position specifying method according to the present embodiment can be broadly divided into an image processing step S1 for acquiring an input image and recognizing information related to a marker set in the image, and a marker database 10 based on the information related to the recognized marker set. Based on the marker set candidate obtained from the search result, the database search step S2 for searching for the marker set candidate from the map database 5 extracts information on the landmarks around the marker set candidate and relates to the landmark. In the interactive specification step S3, the information is presented to the mobile terminal 1 in a question format, and the position of the mobile terminal 1 is specified based on the response information from the mobile terminal to the information about the landmark presented in the question format. It consists of one step.

  As shown in FIG. 2, first, the user takes an image of a landscape including a sign set with a camera attached to the mobile terminal 1 using an application installed in the mobile terminal 1 at a place where the user wants to specify the position. To do. (S101). The captured image is transmitted to the processing server 2 via the mobile terminal communication network 6 (S102).

  The image transmitted to the processing server 2 is input to the image processing unit 7 (S103), and pre-processing is performed (S104), and then the recognition processing of the marker group position is performed (S105). In this recognition processing, the image processing unit 7 determines whether or not there is a marker set in the image (S106). If it is determined that the marker set exists in the image, shape recognition ( Recognizing information on the marker set is performed using the technique of S107) and pattern matching (S108). On the other hand, if the image processing unit 7 determines in step S106 that there is no marker set in the image, the user's mobile terminal 1 is requested to re-capture the image, and the user is requested in response to the request. (S109), the same processing as described above is performed again on the image transmitted to the processing server 2.

  After the information processing related to the marker group is recognized in the image processing unit 7 in this way, the database search unit 8 then searches for and extracts the corresponding marker group candidate from the marker database 10 based on the information (S110). (S111), information regarding the marker group candidate is stored in the management database 11. At this time, the information of the global estimated position which becomes clear from the base station registration database 4 of the portable terminal 1 is acquired and used (S112). Next, the database search unit 8 determines whether or not the number of label group candidates extracted in step 111 is within a predetermined range (for example, less than 15) (S113), and the number of label group candidates is within the predetermined range. If it is determined, information on the first marker set candidate of the first photographed image is acquired (S114), and the process proceeds to step 118 described later.

  On the other hand, when the database search unit 8 determines in step 113 that the number of marker group candidates is not within the predetermined range, the user's portable terminal 1 is requested to re-capture the image, and in response to the request. Each process similar to that described above is performed again on the second image re-photographed by the user (S109) and transmitted to the processing server 2. Thereafter, the image processing unit 7 has a plurality of label candidates extracted from the label database 10 based on the information on the first marker set of the first photographed image and the information on the second marker set of the second photographed image. It is determined whether or not there is (S115).

  In step 115, when the image processing unit 7 determines that there are a plurality of marker candidates, the dialogue processing unit 9 combines the first marker group candidate and the second marker group candidate to group them. (S116). At this time, since it is assumed that the user is in the vicinity of the second image taken, this grouping is performed as shown in FIG. 3 by selecting the second marker set candidate extracted from the second image. (White circle in FIG. 2) and the first label set candidate (black circle in FIG. 2) extracted from the first image existing within the radius r centered on the second label set candidate are the same group. To do so. On the other hand, when the image processing unit 7 determines that there are not a plurality of label candidates (that is, one) in step 115, the process directly proceeds to step 118 described later.

ここで、上式中のα、β、γは各パラメータ間の比率を調整するための係数である。 Then, for each group of the first label set candidate and the second label set candidate grouped in this way, a score is calculated using a predetermined evaluation function, and the groups are sorted in descending order of the score. (S117). As shown in FIG. 4, the parameters used for the evaluation function at this time are the distance d _s2a-s1d between the first label set candidate S _1d and the second label set candidate S _2a , and each label set candidate S _1d , S _2a. There are three types of recognition matching degrees P _s1d , P _s2a , and display angles φ _s1d , φ _s2a of the respective label set candidates S _1d , S _2a . Here, the recognition matching degrees P _s1d and P _s2a are _obtained by parameterizing the number of recognized road signs among the road signs constituting the sign sets in the image. The higher the degree of match between the road sign constituting the road sign and the road sign constituting the sign set recognized by the image processing unit 7, the higher the value of this parameter. The display angles φ _s1d and φ _s2a are angles formed by straight lines connecting the respective marker group candidates S _1d and S _2a and the display direction of the marker groups (arrows in FIG. 4), and are in the range of 0 to 180 °. The angle of And, as the distance and the display angle become larger, the label set candidates are separated from each other, and the higher the recognition matching degree, the more likely each position candidate is. It was defined as:

Here, α, β, and γ in the above equation are coefficients for adjusting the ratio between the parameters.

  Next, information on landmarks around the second marker set candidate is extracted from the map database 5 (S118), and the information is presented to the portable terminal 1 in question format (S119). Specifically, as shown in FIG. 5, about 2 to 4 landmarks are displayed as a list using icons and characters. Then, attributes are set in advance for the landmarks to be presented so that different candidates do not present the same kind of landmarks, for example, a “convenience store” is presented to both of the two label set candidates.

  If the image processing unit 7 determines that there are not a plurality of marker candidates in step 115 and skips steps 116 to 118 and proceeds directly to step 119, as shown in FIG. The question is presented in a format for confirming the position by superimposing the direction, type, and name (eg, “convenience store called ampm”, “big street called Meiji Street”, etc.) of the landmark on the photographed image of the person.

  On the other hand, for example, in the case of FIG. 5, the user selects the “gas station” icon from the mobile terminal 1, or in the case of FIG. 6, selects the “Yes” check box. When the answer is made (S120), the answer is transmitted to the processing server 2, and the processing server 2 determines whether or not there are a plurality of marker group candidates (S121). As a result, when the dialogue processing unit 9 determines that there are not a plurality of marker group candidates (that is, one), the position of the user's mobile terminal 1 is specified (S122), and the position information is stored in the mobile terminal. Mobile terminal 1 acquires the position information (S123). In addition, the dialogue processing unit 9 requests the map database 5 to transmit map data, and in response to this, the map database 5 transmits map data to the mobile terminal 1, and the mobile terminal 1 receives the position data. A map displaying the information is displayed (S124).

  On the other hand, when the dialog processing unit 9 determines that there are a plurality of marker group candidates in the step 121, the process returns to the step 119 again, and information on the landmark is presented to the portable terminal 1 in a question format. This process is repeated until the dialog processing unit 9 determines that there are not a plurality of marker group candidates (that is, one) in step 121.

  As described above, according to the embodiment of the present invention, since the position of the mobile terminal 1 is specified using the existing infrastructure and database called road signs, the introduction cost of the system can be kept low. Also, compared to conventional methods such as landscape recognition, the recognition target is limited to only road signs, so that the processing based on pattern matching is completed, and advanced image processing is not required. Therefore, the cost for image processing can be kept low. Furthermore, since the direction in which the user is facing can be detected, navigation can be performed for the user by displaying the direction of travel superimposed on the captured image as shown in FIG. Furthermore, by interacting with the user, the user can determine whether the displayed point is really correct, and the user can recognize the current location by grasping the surrounding environment. Can also be increased.

  In addition, there is a survey result that about 60% of non-users indicate their intention to use the location information of users of the mobile terminal 1, and the use will increase in the future due to enhancement of applications, contents and functions. there's a possibility that. At that time, it is expected that various requests more than the current state will be generated for the position information, and it is conceivable that various methods are used in combination with the positioning method depending on the location and the situation. Therefore, the technique according to the present invention is useful as a technique for assisting and supplementing GPS in order to meet such a demand.

  Furthermore, as a need for the location information of the user of the mobile terminal 1, the Ministry of Internal Affairs and Communications announced a policy to obligate the mobile phone released after April 2007 to notify the location of the caller with high accuracy. In addition to pedestrian navigation, the use for emergency calls is currently attracting attention. The position identifying technology of the portable terminal 1 according to the present invention automatically turns on the camera by remote control from the center side at the time of an emergency call, and the reporter points the camera to the surrounding scenery so that the position can be determined with higher accuracy. It can be specified and is also effective for use in emergency calls as described above.

It is a conceptual diagram which shows schematic structure of the position specification system which concerns on embodiment of this invention. It is a flowchart which shows the procedure of the position specification method which concerns on embodiment of this invention. It is a conceptual diagram which shows the method of grouping a marker group candidate in the position specifying system which concerns on embodiment of this invention. It is explanatory drawing which shows the parameter of the evaluation function when sorting each group in the position specifying system according to the embodiment of the present invention. It is a figure which shows the selection screen of the landmark displayed on a portable terminal in the position identification system which concerns on embodiment of this invention. It is a figure which shows the confirmation screen of the landmark displayed on a portable terminal in the positioning system which concerns on embodiment of this invention. It is a figure which shows the navigation screen displayed on a portable terminal in the position identification system which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Mobile terminal 5 Map database 7 Image processing part 8 Database search part 9 Dialogue processing part 10 Sign database

Claims (5)

A location system for identifying the location of a mobile device,
Image processing means for recognizing information about a sign set that is a combination of road signs from an image taken by a mobile terminal;
A sign database in which information about sign sets is registered in advance,
Database searching means for searching for a marker set candidate from the marker database based on information on the marker set recognized by the image processing means;
A map database in which information about landmarks is registered in advance,
Based on the marker set candidates searched by the database search means, information on landmarks around the marker set candidates is extracted from the map database, and information on the landmarks is presented to the portable terminal in a question format. Interactive processing means for outputting position specifying information for specifying the position of the mobile terminal based on response information from the mobile terminal to the information about the landmark presented in the question format;
A positioning system characterized by comprising: The database search means, when the number of extracted marker set candidates exceeds a predetermined number, requests the portable terminal to shoot a second image in the vicinity of the shooting point of the first image, Based on the information on the second marker set recognized by the image processing unit from the second image photographed in response to the request, the second marker set candidate is searched from the marker database,
The dialogue processing means combines the first marker set candidate and the second marker set candidate based on the first marker set candidate and the second marker set candidate of the first photographed image. The position specifying system according to claim 1, configured to perform grouping and sort the groups in descending order of scores calculated by a predetermined evaluation function. A location identification method for identifying the location of a mobile device,
(A) recognizing information about a sign set that is a combination of road signs from an image taken by a mobile terminal;
(B) searching for a marker group candidate from a marker database in which information about a marker group is registered in advance based on the information regarding the marker group recognized in the step (A);
(C) Based on the marker set candidates searched in the step (B), information on landmarks around the marker set candidates is extracted from a map database in which information on landmarks is registered in advance, and the landmarks are extracted. Information on the mobile terminal in the form of a question, and output position specifying information for specifying the position of the mobile terminal based on response information from the mobile terminal to the information on the landmark presented in the question form And steps to
A position specifying method characterized by comprising: In the step (B), when the number of the extracted marker set candidates exceeds a predetermined number, the portable terminal is requested to capture the second image in the vicinity of the first image capturing point. , Searching for a second marker set candidate from the marker database based on the information about the second marker set recognized from the second image taken in response to the request,
The (C) step combines the first label set candidate and the second label set candidate based on the first label set candidate and the second label set candidate of the first photographed image. The position specifying method according to claim 3, further comprising a step of sorting the groups in descending order of score calculated by a predetermined evaluation function. A position specifying program for causing a computer to execute each step according to claim 3 or 4.

Images