JP6466188B2 - POSITION ESTIMATION DEVICE, POSITION ESTIMATION SYSTEM, POSITION ESTIMATION METHOD, AND POSITION ESTIMATION PROGRAM - Google Patents

Description

  The present invention relates to a position estimation technique for determining a position such as indoors using a captured image captured by a camera or the like.

  As a position estimation technique, for example, there is a technique for estimating the position of an automobile using an in-vehicle camera described in Patent Document 1 below. The apparatus disclosed in Patent Document 1 recognizes a so-called movable object such as a circuit breaker, a signboard, and the like that is in a certain place and moves according to a certain law while maintaining a certain shape by an in-vehicle camera. The feature points extracted from the captured image of the in-vehicle camera are classified into a movable object feature point group and a fixed object feature point group. Based on the classified feature point group and the image of the in-vehicle camera, the position of the vehicle in which the in-vehicle camera is installed can be estimated.

JP 2012-99010 A

  When indoors, unlike outdoors, there are rarely movable objects that are in a certain place and move with a certain law while maintaining a certain shape. Sometimes people stop in the aisle, or they move suddenly. Also, the posters that were stuck on the wall until yesterday are often gone, changed to another posters, or moved to another location. That is, there are things that do not move regularly, and indoor conditions tend to change.

  An object of the present invention is to make it possible to estimate a position even in an indoor environment where there is no regular movement even if the indoor environment changes from moment to moment.

  According to one aspect of the present invention, it includes image position information and feature point information, and includes a pre-registration information storage unit that stores pre-registered pre-registration information, and image position information and feature point information. A position estimation device associated with a feature point database having an automatic registration information storage unit for storing automatic registration information automatically registered by the device, the position estimation target image being stored in the feature point database Based on matching processing of feature point information with an image, a position estimation processing unit that performs position estimation processing of the position estimation target image, and automatic feature point registration that performs automatic registration processing of the automatic registration information in the feature point database And the automatic feature point registration processing unit is in a state in which the feature point information of the position estimation target image and the pre-registration information match at a certain rate. If that, the automatic registration information storage section position estimation apparatus and storing, in is provided as the automatic registration information.

  Since the feature point information of the position estimation target image and the pre-registration information continues to be matched at a certain rate, the automatic registration information that functions as substitute information for the pre-registration information is retained, It can be used for position estimation.

  In the automatic feature point registration processing unit, the feature point information of the position estimation target image and the pre-registration information is determined in advance as the position estimation rate of the pre-registration image and all other automatic registration images. Are within the range of the automatic registration minimum estimation rate and the automatic registration maximum estimation rate, and the position estimation rate with the pre-registered image is within the range of the pre-registration minimum estimation rate and the pre-registration maximum estimation rate. In some cases, the position estimation target image is stored in the automatic registration information storage unit as automatic registration information.

  Further, the automatic registration information is stored as the automatic registration information when the minimum number of stays within the threshold value registered in advance is within the range between the pre-registration minimum estimation rate and the pre-registration maximum estimation rate. It stores in a part.

  If the pre-registration information continues to be matched at a certain rate, and if it is found that matching with all the registered information continues at a certain rate, the registered feature information The common feature point information is registered in the feature database. As a result, erroneous estimation can be prevented and the latest feature information can always be captured.

  An automatic registration information deleting unit that sequentially deletes the unregistered automatic registration information from the automatic registration information storage unit.

  Feature information that is no longer used for matching at the present time, that is, feature information that has already been discarded in the feature point database, can be deleted by the automatic feature point deletion unit.

  Pre-registration information or auto-registration that matches the highest presumption rate with the common feature point among the pre-registration information and the auto-registration information in the feature point database when the position estimation processing unit is able to match correctly The last matching date and time of information is updated, and the registration information that is not used for matching based on the last matching date and time of all pre-registration information and all automatic registration information in the same registration position in the automatic registration information deletion unit It is characterized by deleting.

  The image processing apparatus further includes a common feature point processing unit that performs processing for sharing image feature point information and registers only the shared image position information and feature point information in the feature point database.

  It is determined that the situation is different from the pre-registration information, and the common feature point information is registered in the feature database. After the moving part is removed by the common feature point processing unit, a process for sharing the feature points of the image is performed.

  Further, the present invention is composed of image position information and feature point information, and includes a pre-registration information storage unit for storing pre-registered pre-registration information, image position information and feature point information, and the apparatus An automatic registration information storage unit for storing automatically registered automatic registration information, and a position estimation system having a feature point database and a position estimation device associated with the feature point database, wherein the position estimation The apparatus includes: a position estimation processing unit that performs position estimation processing of the position estimation target image based on matching processing between the position estimation target image and feature point information of the image stored in the feature point database; and the feature point database. An automatic feature point registration processing unit that performs automatic registration processing of the automatic registration information, and the automatic feature point registration processing unit includes the position estimation target image and the pre-registration. Position estimation, wherein the position estimation target image is stored in the automatic registration information storage unit as automatic registration information when the feature point information with the information continues to be matched at a certain rate System.

  According to another aspect of the present invention, the image includes position information and feature point information of an image, and includes a pre-registration information storage unit that stores pre-registered pre-registration information, and image position information and feature point information. A position estimation method in a position estimation device associated with a feature point database, the automatic registration information storage unit storing automatic registration information automatically registered by the device, the position estimation target image and the feature point Based on the matching process of the feature point information of the image stored in the database, the position estimation processing step for performing the position estimation process of the position estimation target image, and the automatic registration process of the automatic registration information in the feature point database Automatic feature point registration processing step, wherein the feature point information of the position estimation target image and the pre-registration information is constant. If the state in which the matching rate is continuing, the position estimation method characterized by storing in said automatic registration information storage unit the position estimation target image as an automatic registration information is provided.

  Further, the present invention may be a position estimation program for causing a computer to execute the position estimation method described above.

  According to the present invention, it is possible to prevent erroneous position estimation due to a change or loss of a pre-registered image. In addition, the latest feature information can always be taken in.

  Furthermore, feature information that is no longer used for matching, that is, feature information that is no longer needed in the feature point database, is deleted by the automatic registration information deletion unit, so that it is always the latest Matching can be performed while retaining only feature information.

It is a figure which shows the image of a general position estimation technique. It is a figure which shows the image of the position estimation technique by embodiment of this invention, and is a figure corresponding to FIG. 1A. It is a figure which shows the image of the position estimation technique by this form, and is a figure following FIG. 1B. It is a figure which shows the image of the position estimation technique by this Embodiment, and is a figure following FIG. 1C. It is a functional block diagram which shows the example of 1 structure of the position estimation apparatus by this Embodiment. It is a functional block diagram which shows the example of 1 structure of the common feature point process part of the position estimation apparatus by this Embodiment. It is a functional block diagram which shows the example of 1 structure of the position estimation process part of the position estimation apparatus by this Embodiment. It is a functional block diagram which shows one structural example of the automatic feature point registration process part of the position estimation apparatus by this Embodiment. It is a figure which shows the relationship between the system configuration of the whole position estimation apparatus, and the data group exchanged. It is a figure which shows the detailed structural example of the registration threshold value data which is an input of an automatic feature point registration process part. It is a figure which shows the detailed structural example of the estimation detailed data passed from a position estimation process part to an automatic feature point registration process part. It is a figure which shows the detailed structural example of the positional information stored in the position database. It is a figure which shows the specific example of the positional information of FIG. It is a figure which shows the detailed structural example of the pre-registration information and automatic registration information which are stored in the feature point database. It is a figure which shows the detailed structural example of the common feature point threshold value data which is an input of a common feature point process part. It is a figure which shows the detailed structural example of the common feature point data which are the input / output of a common feature point process part, and are the input of a position estimation process part. It is a figure which shows the detailed structural example of the estimation threshold value data used as the input of a position estimation process part. It is a figure which shows the specific example of the present stay frequency used as the input-output of an automatic feature point registration process part. It is a figure which shows the specific example of the movable object node list | wrist which is the output of the movable object list production | generation process part. It is a figure which shows the specific example of a feature point. It is a flowchart which shows the outline | summary of a process of a camera image feature point process part and a common feature point process part. It is a flowchart figure which shows the flow of a process of the movable object list production | generation process part and a position estimation process part. It is a flowchart figure which shows the flow of a setting process of estimation detailed data. It is a flowchart figure which shows the flow of a process of an automatic feature point registration process part and an automatic registration information deletion part. It is a flowchart figure which shows the flow of a common feature point registration process.

Hereinafter, an embodiment for carrying out the present invention will be specifically described with reference to the drawings.
FIG. 1A is a diagram illustrating an image when a general position estimation technique is applied to indoor position estimation. Image information or feature point information (hereinafter referred to as “feature point information”) is registered in advance in the position estimation apparatus system. This is referred to as a pre-registered image (information). This pre-registered image X is information registered by a system administrator or the like based on an image taken indoors in advance at a certain position. For example, the pre-registered image X is confirmed every time a system maintenance inspection is performed every month. Or updated. Information related to the pre-registered image is referred to as pre-registered information.

  Next, using a mobile terminal such as a smartphone having a camera function, the captured image (image captured in the image sensor, Y1, Y2 in FIG. 1A) and the pre-registered image X and the pre-registered image X are image features. Comparing based on points, etc., the position of the smartphone, that is, the position of the owner (actually, the position is accurately estimated by the position and direction as will be described later), for example, map information on the smartphone Etc. and this position information can be used to provide guidance (navigation) of department stores, facilities, etc. to the user who owns the smartphone. By having the direction as information in addition to the position, it is possible to accurately estimate from which direction the image is taken.

  Whether the degree of coincidence (also referred to as matching rate or position estimation rate) of feature points between the currently captured images Y1 and Y2 and the pre-registered image X exceeds a certain threshold (value indicated by AR101 in FIG. 1A). Therefore, the image Y1 that has been estimated (the position is correct) is distinguished from the image Y2 that cannot be estimated (the position is not correct), and the estimated position information is provided for the image Y1 that has been estimated. can do.

  By the way, in the method shown in FIG. 1A, for example, when an object captured in the pre-registered image X has moved even partially, there is a possibility that the position cannot be estimated.

  Therefore, the position estimation technique according to the embodiment of the present invention has been devised as described below.

  1B to 1D are diagrams showing images of the position estimation technique according to the present embodiment, and are diagrams corresponding to FIG. 1A.

  As in FIG. 1A, a pre-registered image A1 is registered in advance in the system of the position estimation device. In addition, a new concept of automatic registration images is introduced.

  An automatically registered image is an image (position information, feature point information) that is automatically registered in the system when the following conditions are satisfied.

(Condition 1)
As shown in FIG. 1B, with respect to the photographed image C1, the position estimation rate of the pre-registered pre-registered image A1 and all other automatically registered images (for example, B1, B2, etc.) is determined in advance. It is within the range of the automatic registration minimum estimation rate and the automatic registration maximum estimation rate (for example, it is within the range of 50% to 70%, that is, it is quite matched).

  The condition 1 means that the captured image C1 satisfies the condition 1 that is automatically registered in the system as an automatically registered image when it exists in the hatched area.

(Condition 2)
As shown in FIG. 1C, the position estimation rate of the pre-registered pre-registered image A1 with respect to the captured image C1 is within the range of the pre-registered minimum estimated rate and the pre-registered maximum estimated rate (for example, position The estimation rate is in the range of 10% to 50%, that is, the degree of matching is intermediate).

Condition 2 indicates an area AR2 in the vicinity of the outer periphery of the elliptical area AR1.
Then, if a state satisfying both of the above conditions 1 and 2 continues for a minimum stay count value, for example, 5 times, the captured video is registered in the system as an automatically registered image.

  By doing so, as shown in FIG. 1D, the range of an image for which position estimation is effective extends to AR21, and position estimation can be performed within a larger range than the range AR101 of FIG. 1A. That is, position estimation can be performed using an automatically registered image in addition to the pre-registered image.

  It should be noted that automatic registration is prevented from being performed indefinitely by preventing further automatic registration of the automatically registered image.

  The automatic registration information can function as alternative information even when the indoor state corresponding to the pre-registration information has changed.

FIG. 2A is a functional block diagram showing a configuration example of the position estimation apparatus 1 according to the present embodiment.
As illustrated in FIG. 2A, the position estimation device 1 includes a common feature point processing unit 24 that performs processing related to common feature points of an image, a position estimation processing unit 12 that performs position estimation processing of a captured image, and the automatic registration described above. And an automatic feature point registration processing unit 11 that performs image registration processing. Further, the position estimation device 1 includes a position database (DB) 9 that stores position information (position and direction), pre-registration information (storage unit) 17, automatic registration information (storage unit) 18, and the like, for example, It is associated with a feature point database (DB) 16 that stores feature points.

  As shown in FIG. 2B, the common feature point processing unit 24 includes a feature point extracting unit 24-1 for extracting feature points of an image, a common feature point calculating unit 24-2 for obtaining a common part of the extracted feature points, and And a common feature point registration unit 24-3 for registering the obtained common feature points in the apparatus.

  As shown in FIG. 2C, the position estimation processing unit 12 is calculated by a movable node extraction unit 12-1 that extracts a movable node, and a matching position calculation unit 12-2 that calculates a matching position of the extracted node. Based on the matching position, the position of the captured image, that is, the current position estimation unit 12-3 that estimates the current position, and the maximum matching rate calculation unit that obtains the maximum matching rate at the position with the highest matching rate with the pre-registration information 12-4, and a final matching date and time update unit 12-5 that updates the final matching date and time with the matching date and time at that time, and an estimated detailed data setting unit 12-6 that sets estimated detailed data to be described later.

  As shown in FIG. 2D, the automatic feature point registration processing unit 11 satisfies all the feature point estimation rate matching determination unit 11-1 that determines whether or not the condition 1 is satisfied, and whether or not the condition 2 is satisfied. As a result of each determination described above, a pre-registered feature point estimation rate matching determination unit 11-2 that determines whether or not, a stay frequency determination unit 11-3 that determines whether or not the current stay frequency has reached a threshold, A common feature point registration unit 11-4 that stores common feature points in the feature point database 16 as automatic registration information, and an invalid auto registration information deletion unit 11-5 that automatically deletes invalid auto registration information due to reasons such as oldness And have.

  FIG. 2E is a diagram illustrating the relationship between the overall system configuration of the position estimation apparatus 1 and the exchanged data group. Although partially overlapping with the above description, the overall system configuration will be described here.

  The position estimation apparatus 1 includes, for example, a camera image reading unit 20 that reads a camera image (captured image) read by a camera 27 built in a smartphone or the like, and a feature point processing unit that performs processing such as extracting feature points from the camera image. 21 as input, and feature point threshold data 25, which is a threshold value indicating whether or not the feature point 22 is extracted from the camera image feature point processing unit 19 and the common feature point. The common feature point processing unit 24 for creating and updating the common feature point data 23, and the immediately preceding position 3 estimated immediately before by the position estimating device 1 as input, from the position database 9 holding the position information 10, for example, from the immediately preceding position A movable target list generation processing unit 4 that generates a movable target node list 8 that is a list of movable nodes, common feature point data 23 acquired from the common feature point processing unit 24, and position estimation Position estimation that searches the feature point database 16 that holds the pre-registration information 17 and the auto-registration information 18 by using the estimated threshold data 6 set as a value and obtains the current estimated position 2 and generates the estimated detailed data 7 Automatic that registers only the common feature point data 23 to be automatically registered in the feature point database 16 while updating the current stay count 56 with the processing unit 12, the estimated detailed data 7 and the registration threshold data 5 as inputs. A feature point registration processing unit 11 and an automatic registration information deletion unit 26 for automatically deleting from the feature point database 16 automatic registration information that may be invalidated that has not been matched recently.

Below, the structural example etc. of each data are demonstrated exemplarily.
As shown in FIG. 3, the registration threshold value data 5 that is an input of the automatic feature point registration processing unit 11 includes an automatic registration estimation range 28 including an automatic registration maximum estimation rate 32 and an automatic registration minimum estimation rate 33, and pre-registration. Pre-registration estimation range 29 with maximum estimation rate 34 and pre-registration minimum estimation rate 35, and auto-registration estimation range 28 and pre-registration estimation range 29 will continue at least how many times Or has a minimum residence count of 30 that defines the residence count.

  The automatic registration estimation range 28 defines a range that should be taken by the estimation rate when position estimation is performed from all feature points for automatic registration. The pre-registration estimation range 29 defines a range to be taken by the estimation rate when limited to pre-registration information. The two ranges for automatic registration and pre-registration are defined, as explained in the above conditions 1 and 2, with a predetermined (constant) estimation rate that is an automatic registration estimation range. This is to ensure that the feature points when pre-registered in the pre-registration estimation range have a predetermined (constant) estimation rate.

  By providing the pre-registration estimation range, it is possible to eliminate the possibility of automatically registering registration information at completely different positions.

  The minimum stay count 30 defines how many times the situation within the automatic registration estimation range 28 and the pre-registration estimation range 29 should be continued. If the minimum stay count continues for 30 minutes and falls within the estimated range for automatic registration and the estimated range for pre-registration, it is determined that the system is in a state for automatic registration.

  Referring to FIG. 1B to FIG. 1D, in the case of the specific example of the registration threshold data illustrated in FIG. 3, when the position is estimated from pre-registered feature points and all automatically registered feature points The latest feature point at the position of the pre-registered feature point when the estimated rate of 50% to 70% of the maximum pre-registered feature point is 10 to 50% This means that it is determined to be information and the common feature points are automatically registered.

  As shown in FIG. 4, the estimated detailed data 7 passed from the position estimation processing unit 12 to the automatic feature point registration processing unit 11 matches the estimated position 36 estimated by the position estimation processing unit 12 with all feature points. Estimation rate 37 of all feature points that is the maximum estimation rate when pre-registered, pre-registration feature point estimation rate 38 that is the maximum estimation rate for all pre-registered feature points at the estimated position 36, and common feature points 39 to be matched And.

  This estimated detailed data 7 is obtained by estimating the estimated position 36 that indicates what estimated position the position estimation processing unit 12 has obtained in the estimation process and all the feature points that indicate the estimated rate of the estimated position. And a pre-registered feature point estimation rate 38. It is determined from the estimated rate 37 of all feature points and the estimated rate 38 of pre-registered feature points whether or not the captured image should be automatically registered by the automatic feature point registration processing unit 11.

  The common feature point 39 is a common feature point to be automatically registered. In the specific example of the estimated detailed data 7 illustrated in FIG. 4, the node 18 (direction 3, position 18-3), the node 19 (direction 2, position 19-2), the node 20 (direction 1, position 20- 1) Among the four pieces of position information of the node 20 (direction 4, position 20-4), it can be estimated that the current node 20 (direction 1, position 20-1) is present. The estimation rate at the time of the estimation process is 65%, indicating that the estimation rate is lower than 65% at other positions. In addition, the estimation rate of only the pre-registered feature point at the position of the node 20 (direction 1, position 20-1) is 20%. The common feature point 39 is a common part of the feature point parts of a plurality of captured images.

  As shown in FIG. 5, the position database (DB) 9 which is an input of the movable object list generation processing unit 4 includes position information 10 including an estimated position 40 and data of the next movable node 41. Is stored.

  As shown in FIG. 6, the position information 10 represents a certain indoor location, and in the case of the specific example of FIG. 6, the indoor passage L1. In this path L1, three nodes are defined, and one direction is defined for each of the nodes 18 and 19, and two directions are defined for the node 20. That is, in this path, Indicates that there are four positions (combination of position and orientation). As described above, the position information is information obtained by replacing the indoor with the direction of the node, and the position estimation apparatus 1 according to the present embodiment performs position estimation from the position information. In the specific example of the position information 10 illustrated in FIG. 5, the position 18-3 that is 3 in the direction of the node 18 indicates that there is a node 20 as the next movable node. When there are a plurality of pieces of position information like the node 20, a plurality of pieces of position information 10 are stored. The reason why the position information is defined by the node and the orientation is that the content of the image depends not only on the shooting position but also on the shooting direction when shooting with the camera.

  As shown in FIG. 7, the feature point database 16 is input information for the position estimation processing unit 12 to perform matching, and information that the automatic feature point registration processing unit 11 and the position estimation processing unit 12 output. Yes, pre-registration information 17 with pre-registration / auto-registration type 43, common feature point 44 and last matching date 45 with registration position 42 and 'pre-registration', registration position 46 and 'auto-registration' Stored is automatic registration information 18 including a pre-registration / automatic registration type 47, a common feature point 48, and a last matching date and time 49 in which 'is set.

  The registration position 42 and the registration position 46 represent the positions represented by the respective registration information, that is, the positions and orientations of the nodes. Pre-registration / auto-registration type 43 and pre-registration / auto-registration type 47 are used to determine whether the corresponding registration information is pre-registration information manually registered in advance or auto-registration information automatically registered by the position estimation device 1. Information.

  The common feature point 44 and the common feature point 48 are registered common feature points. The last matching date and time 45 and the last matching date and time 49 represent when each registered information was last used. The more this date is in the past, the more recently it is not used.

  In the case of the specific example shown in FIG. 7, at the position 18-3 in the direction 18 toward the node 18, two pieces of information of the common feature point 44 registered in advance and the common feature point 48 automatically registered are stored in the feature point database. Is stored. The pre-registered common feature point 44 was used for location estimation around 9:30 on December 1, 2014, and the common feature point 48 for automatic registration was December 15, 2014. It is used for position estimation around 17:00 on the day.

  As shown in FIG. 8, the common feature point threshold value data 25 that is input to the common feature point processing unit 24 includes a minimum number of common feature points 50 and a maximum number of times 51 of common use. The minimum number of common feature points 50 is the definition of the lower limit feature points of effective common feature points. The maximum number of times of commonality is information that defines how many times of commonization are to be performed. In the case of the specific example shown in FIG. 8, it is necessary that the number of feature points to be regarded as common feature points is 50 or more, and even if the common processing is performed, the maximum number of feature points is five. That is, it is set that the common feature point conversion process is performed up to five times while the number of common feature points between images is 50 or more.

  When the number of common feature points is less than 50, it is determined that the position represented by the common feature point having the number of feature points of 50 or more immediately before is the place where the latest photographed image position is different. Do. In addition, even if 50 or more feature points are held, if the sharing is performed five times, the sharing process is temporarily ended.

  As shown in FIG. 9, the common feature point data 23, which is an input / output of the common feature point processing unit 24 and an input of the position estimation processing unit 12, includes an effective flag 52, a common count 53, and a common feature. And 54. The valid flag 52 indicates whether the common feature point 54 is valid. The number of times of commonalization 53 indicates how many times the commonalization process has been performed. The common feature point 54 is a common feature point obtained as a result of the common processing performed by the number of times 53 of common. In the case of the specific example shown in FIG. 9, the current common feature point data is valid, sharing is performed twice, and only the feature points as shown in the common feature points remain. .

  Based on the first common feature point and the second captured image feature point, a common feature point obtained by ANDing both of them as shown above is calculated.

  The estimation threshold data 6 that is input to the position estimation processing unit 12 has an effective minimum estimation rate 55 as shown in FIG. The effective minimum estimation rate 55 is a threshold value for determining whether the position estimation processing unit 12 can estimate the position or not. In the specific example shown in FIG. 10, when the estimation result falls below 10%, it is determined that the position cannot be estimated.

  The current number of stays 56, which is the input / output of the automatic feature point registration processing unit 11, is that there are currently estimated rates remaining in the automatic registration estimation range 28 and the pre-registration estimation range 29 in FIG. It shows how many times it continues. In the case of the specific example shown in FIG. 11, this means that it has continued twice.

  The movable object node list 8 that is the output of the movable object list generation processing unit 4 is a list that represents all nodes that can be directly moved from the node at the immediately preceding position 3. In the case of the specific example shown in FIG. 12, since the node 18 is in the direction 3 immediately before, the next movable node is the two nodes 18 and 20 including the own node. This movable object node list 8a is illustrated in FIG.

  A feature point 22 (FIG. 2E) represents a characteristic portion of an image taken by the camera. In the case of the specific example shown in FIG. 13, a characteristic portion of the image is extracted as a feature point 22 from the image taken by the camera 27 (FIG. 2E) or the like.

The contents of the processing will be described in detail below with reference to FIGS.
FIG. 14 is a flowchart showing an overview of the processing flow of the camera image feature point processing unit 19 and the common feature point processing unit 24 of FIG. 2E.

  First, the camera image reading unit 20 reads the latest camera image from the camera 27 (step S100).

  Next, the feature point processing unit 21 extracts the feature points of the read image. The extracted feature point is the feature point 22 (step S101).

  Next, the common feature point extraction unit 24-1 of the common feature point processing unit 24 determines whether the valid flag 52 of the common feature point data 23 is valid in order to implement the feature point 22 as a common feature point. Is determined (step S102).

  Next, when the determination in step S102 is “No”, that is, when the common feature point 54 of the common feature point data 23 is invalid (for example, for the first time), the feature point extracting unit 24-1 extracts it. The common feature point registration unit 24-3 sets the selected feature point 22 as the common feature point 54 of the common feature point data 23 (step S103-1), and the valid flag 52 is validated (step S103-2). The number of times 53 is set to 1 (step S103-3), and the processing after step S109 is executed.

  Next, when the determination in step S102 is “Yes”, that is, when the common feature point 54 of the common feature point data 23 is valid, the common feature point calculation unit 24-2 A common feature point with the feature point 22 of the read image is calculated (step S104).

  Next, it is determined whether or not the number of common feature points of the common feature points extracted in step S104 is equal to or greater than “50” set in the minimum common feature point number 50 of the common feature point threshold data in FIG. 8 (step S 105).

  If the determination in step S 105 is “No”, that is, if the number of common feature points extracted in step S 104 is less than the threshold, the position indicated by the feature point 22 and the position indicated by the common feature point data 23 , The common feature point data 23 is input to the position estimation processing unit 12, the position estimation processing unit 12 is called to estimate the position (step S106), and the extracted feature point 22 is newly The feature point is set to the common feature point 54 of the common feature point data 23 (step S107-1), the valid flag 52 is enabled (step S107-2), and the number of times of commonization 53 is set to 1 (step S107-3). ).

  If the determination in step 105 is “Yes”, that is, if the number of common feature points extracted in step S 104 is equal to or greater than the threshold value, it is determined that the feature remains in the same position and the common feature points in the common feature point data 23 In 54, the common feature point registration unit 24-3 sets the common feature point extracted in step S104 (step S108-1), and increments the value of the common count 53 by "1" (step S108- 2).

  Next, it is determined whether or not the common count 53 counted up in step S 108-2 is a value equal to or greater than 5 that is the maximum common count 51 (FIG. 8) of the common feature point threshold data 25 (step S). 109). Here, if the determination in step S109 is “Yes”, that is, if the number of times of commonization 53 has reached the maximum value, the current common feature point data 23 is input as it is assumed to remain at the same position. Then, the position estimation processing unit 12 is called to perform position estimation (step S110), and the valid flag of the common feature point data 23 is invalidated (step S111).

  As described above, by repeatedly executing the processing flow of FIG. 14 at a certain time interval, the position estimation processing unit 12 can be repeatedly called while an appropriate common feature point is generated as data of the common feature point data 23. .

  FIG. 15A and FIG. 15B are flowcharts showing an outline of the processing flow of the movable object list generation processing unit 4 and the position estimation processing unit 12 in FIG. 2E.

  First, in order to obtain a list 8 of nodes that can be moved in a short time from the immediately preceding position 3 in a position estimation image that is a target image for which position estimation is to be performed, for example, taken with a smartphone or the like, a movable node extraction unit 12-1 generates, from the position database 9, a list of all movable next nodes 41 in which the node at the immediately preceding position 3 matches the node at the estimated position 40 as the movable object node list 8a. At this time, the node itself at the immediately preceding position 3 is also added to the node list 8a (step S112).

  Next, the matching position calculation unit 12-2 sets the common feature point 54 of the common feature point data 23 and the registered position of each node in the feature point database 16 for all the nodes in the movable target node list 8. By comparing the total pre-registration information 17 and the fully automatic registration information 18, the current position estimation unit 12-3 obtains a position including registration information having the highest estimation rate (matching rate). With this process, it can be determined which node of which direction is estimated to be present (step S 113).

  Next, it is determined whether the estimation rate obtained in step S113 is equal to or greater than the effective minimum estimation rate 55 of the estimation threshold data 6. That is, it is determined whether or not the estimation rate obtained in step S113 is valid (whether it has been estimated) (step S114).

  Next, if the determination in step S114 is “No”, that is, if the estimation cannot be made, the current stay count 56 is set to 0 (step S115), and the current estimated position 2 is set to the immediately preceding position 3. (Step S116).

  If the determination in step S114 is “Yes”, that is, if estimation is possible, the current estimated position 2 is set to the immediately previous position 3 and then the estimated position is set to the current estimated position 2 (step S117). ).

  Next, the maximum matching rate calculation unit 12-4 performs pre-registration corresponding to the common feature point 54 of the common feature point data 23 and the estimated position in the feature point database 16 for the position obtained in step S113. By comparing with the information 17 and obtaining the highest estimation rate, it is checked how much it matches the pre-registration information at the position obtained in step S113. This check is performed so that the automatic registration process is performed only when matching with the pre-registration information at a medium estimation rate (corresponding to condition 2 in FIG. 1C) (step S118).

  Next, the last matching date and time update unit 12-5 updates the last matching date and time of the automatic registration information that has the highest estimation rate in step S113 with the current date and time. By updating this date, unnecessary automatic registration information is identified (step S119).

  Next, the estimated detailed data setting unit 12-6 sets the position obtained in step S113 to the estimated position 36 of the estimated detailed data 7, and the estimation when the estimated rate 37 of all feature points is obtained in step S113. The pre-registered feature point estimation rate 38 is set to the estimation rate obtained in step S118, and the common feature point 39 is set to the common feature point 54 of the common feature point data 23 (step S120). ), The automatic feature point registration processing unit 11 is called (step S121).

  FIG. 15B is a flowchart showing a detailed process flow of step S120. First, in step S120-1, the position obtained in step S113 is set in the estimated position 36 of the estimated detailed data, and in step S120-2, the estimation rate obtained in step S113 is obtained as the estimation rate 37 of all feature points. Set the rate. Next, in step S 120-3, the estimation rate obtained in step S 118 is set as the pre-registered feature point estimation rate 38. Next, in step S120-4, the common feature point 54 of the common feature point data 23 is set to the common feature point 39.

  By repeatedly calling the processing flow of FIG. 15A at regular intervals, position estimation is performed from the common feature point data 23, the estimated detail data 7 is created while updating the current estimated position 2, and then automatic feature point registration processing is performed. Part 11 will be called repeatedly.

  FIG. 16A is a flowchart showing an outline of the processing flow of the automatic feature point registration processing unit 11 and the automatic registration information deletion unit 26 in FIG. 2E.

  First, the total feature point estimation rate matching determination unit 11-1 determines whether or not the estimation rate 37 of all feature points of the estimated detailed data 7 is within the range of the automatic registration estimation range 28 of the registration threshold data 5. Determination is made (step S122).

  Next, if the determination in step S122 is "No", that is, if the common feature point data 23 is not an object of automatic registration, the current stay count 56 is set to 0 and continued for automatic registration The number of times is reset to 0 (step S123).

  Next, when the determination in step S122 is “Yes”, that is, when the common feature point data 23 is within the automatic registration target range, the pre-registered feature point estimation rate matching determination unit 11-2 performs estimation. It is determined whether or not the pre-registration feature point estimation rate 38 of the detailed data 7 is within the pre-registration estimation range 29 of the registration threshold data 5 (step S124).

  Next, if the determination in step S124 is “No”, that is, if the common feature point data 23 is not the target of automatic registration, the current stay count 56 is set to “0” for automatic registration. Is reset to “0” (step S 125).

  Next, if the determination in step S124 is “Yes”, that is, if the common feature point data 23 is within the pre-registration target range, the stay count determination unit 11-3 counts the current stay count 56 by 1 (Step S126), and it is determined whether the current stay count 56 has reached the minimum stay count 30 of the registration threshold data 5 (step S127).

  Next, when the determination in step S 127 is “Yes”, that is, when the common feature point data 23 is to be registered in the feature point database, the common feature point registering unit 11-4 The automatic registration information 18 is created and registered using the common feature point 54 of the data 23 as the automatic registration information. At that time, the registered position 46 sets the estimated position 36 of the estimated detailed data 7, the pre-registration / automatic registration type 47 sets automatic, and the common feature point 48 sets the common feature point 39 of the estimated detailed data 7 The final matching date 49 is set to the current date (step S128).

In step S128, the automatic registration information 18 of the feature point database 16 automatically increases inexhaustively, so the invalid automatic registration information deletion unit 11-5 automatically registers that has not been matched most recently as necessary. Information 18 is deleted. At that time, delete only the latest registration information 17 or automatic registration information 18 at the same position that has been used for position estimation recently (step S129), and set the current stay count 56. It is set to 0 (the number of continuations for automatic registration is reset to 0), and the number of continuations for automatic registration is reset to 0 (step S130).

  FIG. 16B is a flowchart showing details of the registration process in step S128. In step S128-1, 1) The estimated position of the estimated detailed data is set as the registered position. In step S128-2, 2) Pre-registration and automatic registration type are automatically set. In step S128-3, the common feature point is set to the common feature point of the estimated detailed data. In step S128-4, the current date and time are set as the final matching date and time.

  In the present embodiment, after the moving part is removed by the common feature point processing unit, the pre-registration information in the feature point database registered in advance and the automatic registration information are matched. At that time, the pre-registration information and the pre-registration estimation range in the registration information, that is, a certain estimation rate, and the minimum number of stays in the registration threshold data within the automatic registration estimation range in the registration threshold data If it stays only, it is a mechanism for automatic registration.

  Therefore, if it is found that matching with pre-registration information at a certain rate continues and matching with all registration information continues to be at a certain rate, registered It is determined that the situation is different from the feature information, and the common feature point information is registered in the feature database. As a result, erroneous estimation can be prevented and the latest feature information can always be captured.

  In addition, in the case of feature information that is no longer used for matching at the present time, that is, old feature information that is already unnecessary in the feature point database and that has the latest feature information, it can be deleted by the automatic feature point deletion unit. . By having such a function, it is possible to perform matching while always holding only the latest feature information.

In the above-described embodiment, the configuration and the like illustrated in the accompanying drawings are not limited to these, and can be changed as appropriate within the scope of the effects of the present invention. In addition, various modifications can be made without departing from the scope of the object of the present invention.
Note that indoors and the like may include facilities and theme parks.

  Further, the position estimation may be performed by combining the general position estimation technique described in Patent Document 1, FIG. 1A, and the like and the position estimation technique described in the present embodiment. For example, the former may be used when there are many objects that move regularly, and the latter may be used otherwise.

  This technology can be applied to various devices such as a wearable device in addition to a smartphone.

  Each component of the present invention can be arbitrarily selected, and an invention having a selected configuration is also included in the present invention.

  The present invention can be used for position estimation.

DESCRIPTION OF SYMBOLS 1 ... Position estimation apparatus 4 ... Moveable object list production | generation part 9 ... Position database 11 ... Automatic feature point registration process part 12 ... Position estimation process part 16 ... Feature point database 17 ... Pre-registration information storage part 18 ... Automatic registration information storage Unit 19: Camera image feature point processing unit 20 ... Image reading unit 21 ... Feature point processing unit 24 ... Common feature point processing unit 26 ... Automatic registration information deletion unit

Claims (9)

It consists of image position information and feature point information, and consists of a pre-registration information storage unit that stores pre-registered pre-registration information, and image position information and feature point information. An automatic registration information storage unit for storing registration information, and a position estimation device associated with a feature point database,
A position estimation processing unit for performing position estimation processing of the position estimation target image based on matching processing of feature point information between the position estimation target image and an image stored in the feature point database;
An automatic feature point registration processing unit for performing automatic registration processing of the automatic registration information in the feature point database,
The automatic feature point registration processing unit
Feature point information of the pre-registration information and the position estimation target image, the if you are matching at a constant rate, be stored in the automatic registration information storage unit the position estimation target image as an automatic registration information A position estimation device characterized by the above. The automatic feature point registration processing unit,
Previous Article before the registration information, the position estimation rate with all other automatic registration image, there within the range of the automatic registration minimum estimated rate and automatic registration maximum estimated rates are predetermined,
And,
When the position estimation rate with the pre-registered image is within the range of the pre-registration minimum estimation rate and the pre-registration maximum estimation rate,
The position estimation apparatus according to claim 1, wherein the position estimation target image is stored in the automatic registration information storage unit as automatic registration information. Furthermore, the pre-registration information and the position estimation rate of all other automatic registration images are within a threshold value that is pre-registered within the range of the pre-registration minimum estimation rate and the pre-registration maximum estimation rate. The position estimation apparatus according to claim 2, wherein the position estimation target image is stored in the automatic registration information storage unit as automatic registration information when the minimum number of stays has remained.   The position estimation according to any one of claims 1 to 3, further comprising an automatic registration information deletion unit that sequentially deletes the unregistered automatic registration information from the automatic registration information storage unit. apparatus. When the position estimation processing unit can correctly match,
Of the pre-registration information in the feature point database, among the automatic registration information, the highest matching rate with the common feature point is updated, and the last matching date of the pre-registration information or the automatic registration information is updated,
The automatic registration information deletion unit deletes registration information that is not used for matching based on the last matching date and time of all pre-registration information and full automatic registration information for each same registration position. Item 5. The position estimation device according to Item 4. further,
2. A common feature point processing unit that performs processing for sharing feature point information of an image and registers only the position information and feature point information of the shared image in the feature point database. The position estimation apparatus according to any one of 5 to 5. It consists of image position information and feature point information, and consists of a pre-registration information storage unit that stores pre-registered pre-registration information, and image position information and feature point information. An automatic registration information storage unit for storing registration information, and a feature point database, and a position estimation device associated with the feature point database,
The position estimation device includes:
A position estimation processing unit for performing position estimation processing of the position estimation target image based on matching processing of feature point information between the position estimation target image and an image stored in the feature point database;
An automatic feature point registration processing unit for performing automatic registration processing of the automatic registration information in the feature point database,
The automatic feature point registration processing unit
Feature point information of the pre-registration information and the position estimation target image, the if you are matching at a constant rate, be stored in the automatic registration information storage unit the position estimation target image as an automatic registration information A position estimation system characterized by It consists of image position information and feature point information, and consists of a pre-registration information storage unit that stores pre-registered pre-registration information, and image position information and feature point information. A position estimation method in a position estimation device associated with a feature point database having an automatic registration information storage unit for storing registration information,
A position estimation processing step for performing position estimation processing of the position estimation target image based on matching processing of feature point information between the position estimation target image and an image stored in the feature point database;
Automatic feature point registration processing step for performing automatic registration processing of the automatic registration information in the feature point database,
The automatic feature point registration processing step includes:
Feature point information of the pre-registration information and the position estimation target image, the if you are matching at a constant rate, be stored in the automatic registration information storage unit the position estimation target image as an automatic registration information A position estimation method characterized by the above.   A position estimation program for causing a computer to execute the position estimation method according to claim 8.

Images