JP5305031B2 - Feature amount extraction apparatus and method, and position estimation apparatus and method - Google Patents

Description

  The present invention relates to a feature quantity extraction apparatus and method for extracting a local feature quantity from an input image, and a position estimation apparatus and method using the same.

  Identification of self-location is an essential capability for humans and machines. Knowing where you are now is always important for robots and computer vision. Especially for mobile robots, knowing where you are in the world is a fundamental requirement for navigation systems.

  For such self-location identification, how to extract local features accurately is a big point. Conventionally, there are affine-invariant feature quantities (MSER, Harris-Affine, Hessian-Affine, Salient Region, etc.), feature-invariant feature quantities (SIFT: Non-Patent Document 1), SURF (Non-Patent Document 2) .

D. G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l Jour. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004 H. Bay, T. Tuytelaars and L. V. Gool, "SURF: Speeded Up Robust Features," Proc. European Conf. Computer Vision, 2006

  However, these feature quantities have the following problems. First, when the same object is photographed from different positions, if the conventional local features are extracted as they are, a large amount of different local features that cannot be matched are extracted. Next, even if it is desired to recognize them as the same, if there is a change in the appearance to some extent, all local features must be learned for each appearance, and the use efficiency of the storage capacity is poor. Furthermore, a feature that appears only in a certain way of appearance may be a source of misrecognition, and when a conventional local feature is used as it is, the feature amount is very noisy.

  Moreover, the feature-value corresponding to both the conventional affine change and a magnitude | size change is under research, and a useful method is not reported.

  The present invention has been made to solve such problems, and it is an object of the present invention to provide a feature amount extraction device that extracts a feature amount robust to a change in position, and a position estimation device equipped with the feature amount extraction device. And

  The position estimation apparatus according to the present invention includes a feature amount extraction unit that extracts an invariant feature amount from an input image that is a continuously captured image, and a position based on the invariant feature amount extracted by the feature amount extraction unit. A position estimation unit for estimation, wherein the feature amount extraction unit extracts a local feature amount from each of the input images, and the local feature amount extracted by the local feature amount extraction unit. A feature amount matching unit that performs matching between the continuous input images, and a continuous feature amount selection unit that selects, as the continuous feature amount, a local feature amount that is matched between a predetermined number of consecutive images by the feature amount matching unit. And an invariant feature quantity calculating means for obtaining an average of the continuous feature quantities as an invariant feature quantity.

  In the present invention, using images that are taken consecutively, matching feature amounts between two consecutive images, and further extracting only those features that have been matched appear continuously, By obtaining the average local feature amount, it is possible to extract a feature that is robust to changes in the shooting position, and to estimate the position using this, it is possible to accurately identify the position. A continuous image is a set of a plurality of images taken continuously from a video image.

  Here, the position estimation means uses an area database associating the invariant feature amount extracted from a plurality of the teacher images with area identification information indicating the area from which the invariant feature amount is extracted, using the teacher image as an input image. Storage means for storing; an image obtained by photographing the surroundings as an input image; each of a plurality of local feature quantities of the input image extracted by the local feature quantity extraction means; and the invariant feature quantity registered in the area database; And an estimation result output means for voting on the area corresponding to the invariant feature quantity matching each local feature quantity and estimating the area with the maximum number of votes as the current position. it can. By constructing an area database by learning or the like, various positions can be specified.

  Moreover, the said position estimation means can recognize the said area as an unlearned area, when the said number of votes is less than a predetermined threshold value. As a result, when the position is not clear, the area is not specified, so that it is possible to prevent inaccurate estimation, and by making the area as an unlearned area, the second learning can be accurately performed. it can.

  Further, the local feature amount may be a feature amount of SIFT (Scale Invariant Feature Transformation) and / or SURF (Speed Up Robustness Features). In addition to these SIFTs and SURFs, other local feature quantities that are robust to scale, rotation fluctuation, noise, and the like can be used. Thus, by using these existing local feature amounts, the performance of these feature amounts is inherited as it is, and it becomes possible to extract and describe as features that are robust against illumination changes and the like.

  In addition, the data processing apparatus includes optimization means for optimizing the database, and the optimization means can delete redundant individual invariant feature quantities among a plurality of individual invariant feature quantities constituting the invariant feature quantity. Thereby, the capacity of the database can be reduced.

  Further, the optimization unit reuses the teacher image, votes for the individual invariant feature amount used to identify the teacher image, and deletes the redundant individual feature amount according to the voting result. Can do. Thereby, the individual invariant feature quantity with low discrimination ability can be deleted, and the reduction of discrimination ability can be suppressed by reducing the capacity.

  Furthermore, the optimization means can perform the optimization offline or online. Depending on the optimization method, it is possible to select online or offline.

  The optimization unit may delete redundant data for the invariant feature and the area identification information registered in the area database. Thereby, the capacity of the database can be reduced.

  Further, the redundant data may have a low probability of moving again to the area indicated by the area identification information. As a result, data that has not been used for a while can be deleted.

  The feature amount extraction apparatus according to the present invention includes a local feature amount extraction unit that extracts a local feature amount from each of input images that are continuously captured images, and the local feature amount extraction unit that extracts the local feature amount. For feature quantities, a feature quantity matching unit that performs matching between the continuous input images, and a continuous feature quantity that selects a local feature quantity that is matched between a predetermined number of consecutive images by the feature quantity matching unit as a continuous feature quantity. A selection unit; and an invariant feature amount calculation unit that obtains an average of the continuous feature amounts as an invariant feature amount.

  ADVANTAGE OF THE INVENTION According to this invention, the feature-value extraction apparatus and method which can extract the local feature-value robust to the change of imaging | photography position, and the position estimation apparatus and method using the same can be provided.

It is a figure which shows the position estimation apparatus concerning embodiment of this invention. It is a figure explaining the method of extracting the feature-value PIRF concerning this Embodiment. It is a flowchart which shows the invariant feature-value extraction method of embodiment of this invention. It is a figure explaining the method of estimating which place an input image is. It is a figure which shows the image used in Example 1 of this invention. It is a figure which shows the result of Example 1 of this invention. It is a figure which shows the result of Example 2 of this invention. It is a figure which shows the result of Example 3 of this invention. It is a figure which shows the result of Example 4 of this invention.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. In this embodiment, the present invention is applied to a position estimation device equipped with a feature amount extraction device capable of extracting a local feature amount that is robust to changes in the photographing position. The present invention can also be applied to scene recognition / object recognition for real environment images, for example.

  FIG. 1 is a diagram illustrating a position estimation apparatus according to an embodiment of the present invention. The position estimation apparatus 1 according to the present embodiment includes a feature quantity extraction unit 20 that extracts an invariant feature quantity from an input image that is a series of continuously captured images, and the invariant feature quantity that the feature quantity extraction section 20 has extracted. And a position estimation unit 10 that estimates the target position based on the above.

  The feature amount extraction unit 20 matches a local feature amount extraction unit 21 that extracts a local feature amount from each of the input images and the local feature amount extracted by the local feature amount extraction unit 21 between successive input images. A feature quantity matching unit 22, a continuous feature quantity selection unit 23 that selects a feature quantity matched between a predetermined number of consecutive images by the feature quantity matching unit 22 as a continuous feature quantity, and an average of each continuous feature quantity is unchanged. And an invariant feature amount calculation unit 24 to be obtained as a quantity.

  The position estimation unit 10 includes a storage unit 12 that stores an area database 11, an estimation result output unit 13, and a DB optimization unit 14 that optimizes the database. The area database 11 stores a teacher image as an input image, and stores invariant feature amounts extracted from a plurality of the teacher images and area identification information indicating an area from which the invariant feature amounts are extracted. The estimation result output unit 13 uses an image whose position is to be specified as an input image, each of the plurality of local feature amounts of the input image extracted by the local feature amount extraction unit 21, and the invariant feature amount registered in the area database 11. Is voted on the area corresponding to the invariant feature amount matching each local feature amount, and the area having the maximum number of votes is estimated as the position of the specific target.

  In the following description, the invariant feature amount extracted by the invariant feature amount calculation unit 24 is referred to as a feature amount PIRF (Position-Invariant Robust Features). First, the feature amount extraction unit 20 that extracts the feature amount PIRF. Will be described in detail. The feature quantity extraction unit 20 extracts the feature quantity PIRF as a (local) feature quantity that is not easily affected by changes in the shooting position.

  As a result of the present inventor's earnest experiment research to solve the problem of self-position estimation of a mobile robot in a real environment, the difference in appearance (change in feature amount) due to changes in shooting position and shooting time period for nearby objects However, since the change is small for a distant object (the feature amount of the landmark does not change so much), a method for extracting the feature amount PIRF has been found.

  The feature amount extraction unit 20 according to the present embodiment simply matches local features between consecutive images, selects features that are continuously matched between a predetermined number of images, and is selected. The average of all the features that are matched with the extracted feature is extracted and described as the feature amount PIRF.

  This feature quantity extraction unit 20 can extract and describe only local features that are robust to changes in the shooting position. Various descriptors can be used as long as they are local features. By using the existing local feature as described above, the performance of the existing feature is inherited as it is, and the illumination change etc. Can be extracted and described as robust features

  Hereinafter, each block will be described in detail. FIG. 2 is a diagram for explaining a method of extracting the feature amount PIRF according to the present embodiment. The local feature amount extraction unit 21 receives a continuous image captured continuously as an input image. Here, a continuous image required by PIRF is a video set that is a certain image set and is continuously captured every second, such as two frames every second, for example, two frames every second. That is, the images captured from the video are generally continuous, and the continuous images in the PIRF must use video images. The image acquisition rate is set according to the speed of the camera. For example, if the camera is mounted in a car, the camera speed is about 1000 m / min per minute and the continuous image captured from the video is approximately 50 to 100 frames / sec.

  In the present embodiment, an omnidirectional image is used as an input image, and continuous images obtained by photographing one place are used. As will be described later, this place is, for example, an area from an intersection to an intersection. The place is divided into a plurality of sub-places composed of several continuous images. That is, one place is composed of a plurality of sub-places. One or a plurality of invariant feature quantities are extracted for each subplace. This set of invariant features is called a sub-place PIRF dictionary (PIRF-dictionary). A set of all invariant features extracted from one place, that is, a set of sub-place PIRF dictionaries constitutes a PIRF dictionary. One place corresponds to one PIRF dictionary. The sub-place PIRF dictionary and the place identification information from which the PIRF dictionary is extracted are stored in the area database 11 together with the PIRF dictionary.

First, the local feature quantity extraction unit 21 extracts a local feature quantity using an existing local feature quantity extraction method. The local feature quantity extraction unit 21 can use, for example, a feature quantity of SIFT (Scale Invariant Feature Transformation) or SURF (Speed Up Robustness Features). Of course, the present invention is not limited to these SIFTs and SURFs, and other local feature quantities can be used. In particular, it is preferable to use other local features that are robust against scale, rotation fluctuations, noise, and the like. By using these local feature amounts, the performance of the existing feature amounts is inherited as it is, and it becomes possible to extract and describe as features that are robust against changes in illumination. Here, the number of omnidirectional images in the i-th area is n _i, and a set of local feature values extracted from the j-th image is U _j ⁱ . Here, the local feature amount is represented by u →.

Next, the feature amount matching unit 22 performs matching of each feature amount constituting the local feature amount u → for all two consecutive images. That is, matching is performed on all local feature values of the j = q + 1-th image with respect to all local feature values of the j = q-th image. Here, the index to the matched feature is obtained as a matching result vector m _q ⁱ →.

Here, an example of the matching method will be described using SIFT as an example. _{Let the} feature extracted from the image Ia be v. It is determined whether or not the feature v matches the feature v ′ of the next image I _{a + 1} . First, a dot product between the feature v and all the features extracted from the image I _{a + 1} is obtained. Then, this result is sorted in order to obtain the most similar feature v _first and the second most similar feature v _second . if,
(V _first · v) / (v _second · v)> θ
When is established, it is determined that the matched feature v ′ = v _first . Here, the threshold value θ can be set to 0.6, for example. If the above is not satisfied, it characterized matching the characteristic v of the image I _a is judged not to exist in the image I _{a + 1.}

As shown in FIG. 2, a case where six local feature amounts are extracted from each input image will be described. Matching is performed between these six local feature quantities, and an index to the feature quantity that has been matched is attached only when matching is achieved. For example, the _first local feature value of m ₁ ⁱ → indicates that the third local feature value of m ₂ ⁱ → is matched, and the third feature value of m ₂ ⁱ → is m _3. ^This indicates that matching with the sixth feature quantity of ⁱ → is achieved.

Next, the continuous feature value selector 23 selects a continuous feature value. First, it is determined how many m _q ⁱ → to obtain the continuous feature amount. The number of m _q ⁱ → is also referred to as a window size w in this specification. A set of m _q ⁱ → included in the window size w is called a sub-place. Here, as the window size w is larger, it is possible to extract only a continuous feature amount that is more robust and has a high discriminating power. On the other hand, if it is too small, feature quantities that are not robust and have no discriminating power will be extracted.

In the present embodiment, the window size w is 3. Therefore, continuous feature values are obtained using four consecutive input images. That is, as shown in FIG. 2, the first sub-place includes m ₁ ⁱ →, m ₂ ⁱ →, m ₃ ⁱ →, and the input images I ₁ , I ₂ , I ₃ , and I ₄ correspond to each other. To do. When the number of indexes is 0, it indicates that there is no feature amount to be matched next. Therefore, in the case of FIG. 2, the first sub-place includes three continuous feature values.

When the window size w is set, the continuous feature value selection unit 23 shifts the windows w one by one, and extracts features that appear in common in the four omnidirectional images included in the window size as continuous feature values. When the window size w is set, a function used to extract continuous feature values is defined as follows. Here, b is the number of the index vector of interest.
Then, f ( _{mx, y} ⁱ ) is calculated for all matching result vectors m _q ⁱ →, and only the local feature quantity u _{x, y} ⁱ → when f ( _{mx, y} ⁱ )> 0 is obtained. Extract. When the number of input images is n _i and the window size is w, the number of sub-places is n _i −w + 1.

The invariant feature quantity calculation unit 24 obtains an average of local feature quantities that have been matched within sub-places that are the same window group. A PIFR dictionary is constituted by a vector composed of these average values. Registered in all sub place _(n i -w + 1) extracted from pieces _(n i -w + 1) pieces of sub-Place PIFR Dictionary ^{_{(D j i, j ≦ n}} i -w + 1) the PIRF dictionary ^{(D i).} Each average of the matched local feature values constituting the PIFR dictionary is PIRF.

Next, a method for extracting the invariant feature amount PIFR of the feature amount extraction unit will be described. FIG. 3 is a flowchart showing the invariant feature amount extraction method. First, i = 1 and j = 1 are initialized (step S1). Here, _{n i} is the number of images belonging to the Place _{A j,} w is the window size to extract the PIRF.

Next, the image I _i of the area A _j is input (step S2). Then, it is determined whether i is larger than the window size w (step S3). If i is less than or equal to the window size w, i is incremented (step S10), and the process returns to step S2.

On the other hand, if it is larger than the window size w, matching is performed between the image I _i and the image I _i-1 (step S4). Then, a stable local feature amount (SIFT) is extracted from w + 1 consecutive input images (I _i-w ,..., I _i ) (step S5). Here, in the present embodiment, the window size w is set to 3. The window size w indicates how many continuous images the PIRF is extracted from. For example, if the window size w is set to 3, PIRF is obtained only when there are features that appear consecutively in four consecutive images. Therefore, if there are fewer than four image images in a place A, it is not sufficient to find matching features between the three two images. Therefore, when there are at least four images, extraction of PIRF from place A can be started. For example, when the current image is I ₆ , PIRF can be obtained from _four consecutive images I ₃ , I ₄ , I ₅ , and I ₆ .

Next, the average of the extracted stable local features is calculated as PIRF (step S6). Next, PIRFs included in the area A _j are collected and registered in the PIRF dictionary D _j (step S7). Then, whether or not i = _{n j} is determined, if i = _{n j,} and i = 1, j = j + 1, and repeats the processing from step S2 (step S9). If i = n _j is not satisfied, the process proceeds to step S10, i is incremented, and the process returns to step S2.

  Next, the position estimation unit 10 will be described. As described above, the storage unit 12 of the position estimation unit 10 includes the area database 11. In the area database 11, area identification information for identifying an area (place) and an invariant feature amount PIRF (PIRF dictionary) included in the area are associated and registered. This area database 11 is generated by learning to extract and register an invariant feature quantity PIRF from a teacher image associated with area identification information in advance using the feature quantity extraction unit 20.

  Here, in the present embodiment, a case will be described in which a position estimation device does not estimate a specific position as a specific target position, but recognizes a certain area within a certain range as an area. Note that the position estimation device may be configured to estimate the current position.

  By the way, in this Embodiment, the area database 11 is produced | generated by the position estimation apparatus 1 itself by the feature-value extraction part 20. FIG. For this reason, for example, when the position estimation device 1 is mounted on a robot apparatus, the robot apparatus first acquires continuous images of the area while moving the area to be recognized by the imaging unit, and identification for identifying the area. Associate information. Then, the continuous image associated with the identification information is used as a teacher image, and the feature amount extraction unit 20 extracts the invariant feature amount PIRF (PIRF dictionary). By registering this in the area database 11 together with the identification information, the area database 11 is constructed. In this case, for example, when an unknown area is entered, it is possible to learn the unknown area and update the area database 11 as necessary. It is of course possible to prepare the area database 11 in advance and use it.

When the robot apparatus estimates its own position, an image of the surroundings is used as an input image, and the local feature amount extraction unit 21 extracts a plurality of local feature amounts from the input image. The estimation result output unit 13 takes a match between the extracted local feature quantity and the PIRF included in the PIRF dictionary registered in the area database 11 and votes for an area having an invariant feature quantity that matches each local feature quantity. The area where the number of votes is maximized is estimated as the current self-position, and the result is output. FIG. 4 is a diagram for explaining a method for estimating which place the input image is. As shown in FIG. 4, it is assumed that a certain environment includes five places. Each place has a PIRF dictionary (D _{1 to} D ₅ ).

First, a local feature amount is extracted from an input image (test image). Matching between each extracted local feature and each individual PIRF included in each PIRF dictionary is performed. A place corresponding to the PIRF dictionary that matches most PIRFs by these matchings is a place to which the input image belongs. In FIG. 4, the first and second images all have the largest matching with the _first PIRF dictionary D1. Thus, Place corresponding to PIRF dictionary _{D 1} is the place of the input image.

  Here, when the number of votes is less than a predetermined threshold, the local feature quantity extraction unit 21 recognizes the area as an unlearned area. As a result, when the area cannot be estimated accurately, the area is not specified, so that only the area that can be reliably specified can be output. Moreover, not only uncertain estimation but also appropriate additional learning can be performed.

  Next, the DB optimization unit 14 will be described. One PIRF dictionary corresponds to one place. Accordingly, as many PIRF dictionaries as the number of recognized places are generated. Accordingly, as the number of places is recognized, the number of PIRF dictionaries increases and the memory capacity thereof also increases. When the position estimation apparatus 1 is mounted on a robot apparatus, for example, PIRF dictionaries corresponding to the number of places visited are learned. However, the storage capacity of the robot apparatus is finite. Therefore, it is necessary to optimize the capacity of the area database 11.

  That is, it may be necessary to reduce the data capacity of the area database 11. In such a case, two data reduction methods are conceivable. One is a method of reducing the number of individual PIFRs constituting the PIRF dictionary, and the other is a method of reducing the number of PIRF dictionaries themselves.

  First, a method for reducing individual PIFRs constituting the PIRF dictionary will be described. The DB optimizing unit 14 optimizes the number of individual PIRFs included in the PIRF dictionary by deleting redundant ones of a plurality of PIFRs (individual invariant feature quantities) constituting the PIFR dictionary.

  The DB optimization unit 14 reuses the teacher image used when the area database 11 is constructed. Then, voting is performed on the individual invariant feature amounts used to identify each teacher image, and redundant individual feature amounts are deleted according to the voting result. That is, a PIRF with a low identification capability that is not often used for identification is deleted. For example, a PIRF that has never been used for identification in a vote 0 has no effect on performance even if it is deleted. Note that such optimization requires a certain amount of teacher images and is a batch test, so when it is installed in a robot apparatus, it is offline work.

  For example, PIRFs may be extracted as feature quantities for a plurality of places such as Tokyo Tower and Mt. Fuji. Such a PIRF can be deleted as a PIRF having a low place identification capability.

  Next, as another method, optimization for reducing the number of PIRF dictionaries themselves will be described. In this case, the DB optimization unit 14 deletes an inappropriate and redundant PIRF dictionary for each place corresponding to the PIRF dictionary registered in the area database 11. For example, in the case of a robot apparatus, it is possible to delete data on a place that has a low probability of moving again, that is, a PIRF dictionary that has a low probability of being used again. For example, a PIRF dictionary that has not been used for a certain period of time may be deleted. This operation corresponds to a function of forgetting a human function. Since such optimization can be performed each time by the robot apparatus, online optimization is possible.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  For example, in the above-described embodiment, the hardware configuration has been described. However, the present invention is not limited to this, and arbitrary processing may be realized by causing a CPU (Central Processing Unit) to execute a computer program. Is possible. In this case, the computer program can be provided by being recorded on a recording medium, or can be provided by being transmitted via the Internet or another transmission medium.

[Example 1]

  For the experimental data, we used continuous images taken on the campus of "Zuzukakedai" and "Ookayama". The image size was 640 × 428. About Suzukakedai campus, 580 sheets were used for learning and 489 sheets were used for the test. On the Ookayama campus, 450 were used for learning and 493 were used for testing.

  The left figure of Fig.5 (a) shows the Suzukakedai campus, and it divided | segmented manually into 23 places. The right figure of FIG.5 (b) shows the Ookayama campus and divided | segmented manually into 13 places. For the Suzukakedai canvas, we acquired a teacher image during the daytime on a sunny day. As test images, images obtained under various weather conditions on weekdays were used. For the Ookayama Campus, both teacher images and test images were randomly acquired for three months for various weather conditions on weekdays.

  The upper left diagram in FIG. 5B shows the place A21, and the lower left diagram shows the place A1. Teacher images were collected on holidays (upper left diagram), while test images were collected on weekdays (upper right diagram). Several teacher images were taken during the day (bottom left) and test images were acquired in the evening (bottom right).

  FIG. 5C shows a sample image of “Ljubljana lab.” Of the COLD data set. Using four printer areas, a corridor, a shared room, and a bathroom, the images were taken in a cloudy, sunny, and night environment. There are about 6000 continuous images, and the size is 640 × 480. FIG. 5D shows a place corresponding to FIG. 5C taken at night.

  As a comparison method, GIST (A. Torralba, KP Murphy, WT Freeman and MA Rubin, "Context-Based Vision System for Place and Object Recognition," Proc. IEEE Int'l Conf. Computer Vision, pp. 1023-1029, 2003) and sPACT (J. Wu and JM Rehg, "Where am I: Place instance and category recognition using spatial PACT," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2008.) A total of four methods were used in the case where 1-NN (1-nearest neighbor) and SVM (Support Vector Machine) were used as discriminators.

  FIG. 6A is a diagram showing the experimental results and the accuracy of each method. The result of PIRF is the rightmost, and it was confirmed that the performance was more than twice as good as other methods. The left shows the experimental results at the Suzukakedai Campus and the right shows the Ookayama Campus. No. 6 (b) shows a mixing matrix of recognition results on the Suzukakedai campus. Rows indicate correct answers and columns indicate guess results. FIG. 6C is a graph showing a confidence value for each test image. The average line used was a cubic polynomial. It can be confirmed that the confidence value at the time of the correct recognition result is high on average, the confidence value at the time of misrecognition is low on average, and the reliability of the recognition result has a correct correlation. FIG. 6D shows the recognition time of each test image. PIRF-n indicates the result when PIRF is reduced to n%. The bottom solid line shows the result when the voting process is executed in parallel with PIRF-50. It is possible to reduce the recognition time by reducing the number of PIRFs, and further, it is possible to achieve a recognition time comparable to that of the existing method by adding parallel processing. In any reduction amount, recognition accuracy superior to that of the existing method is maintained.

  According to this example, it was confirmed that the accuracy of the outdoor location recognition experiment was higher than that of other identification methods.

[Example 2]
COLD dataset (MM Ullah, A. Pronobis, B. Caputo, J. Luo, P. Jensfelt and HI Christensen, "Towards Robust Place Recognition for Robot Localization," Proc. IEEE Int'l Conf. Robotics and Automation, pp .530-537, 2008) (FIGS. 6C and 6D) were used for experiments. There are continuous images taken under three environments, sunny, cloudy, and night, at four indoor locations, and there are two sets of continuous images for each. Therefore, there were a total of 6 consecutive images, which were evaluated as 6-fold cross-validation. FIG. 7 is a diagram showing the results of this example. FIG. 7A shows a case where learning is performed in a fine state, FIG. 7B shows a case where learning is performed in a cloudy state, and FIG. 7C shows a case where learning is performed at night.

  In FIG. 7A to FIG. 7C, the average result is shown from the left, clear, cloudy, night, and average. Among the results, the results for PIRF, Harris-Lasplace + SVM, sPACT + NN, and Gist + NN are shown from the left.

  It has been confirmed that the accuracy equivalent to that of other methods can be obtained by this example.

Example 3
A place recognition experiment was performed using a mixture of the data sets used in Example 1 and Example 2. The result is shown in FIG. Rows represent correct classes and columns represent guess results. The average recognition accuracy of each data set was COLD data set: 95.12%, Suzukakedai campus data: 86.56%, Ookayama campus data: 76.06%.

  According to the present example, even if indoor and outdoor data are mixed, or even if there is a large difference in the amount of learning data for each place, it can be confirmed that the method can be used.

Example 4
Using the data set of Experiment 1, PIRF was converted to ISC (C. Valgren, T. Duckett and A. Lilienthal, "Incremental Spectral Clustering and Its Application to Topological Mapping," Proc. IEEE Int'l Conf. Robotics and Automation, 2007. , C. Valgren and A. Lilienthal, "Incremental Spectral Clustering and Seasons: Appearance-Based Localization in Outdoor Environments," Proc. IEEE Int'l Conf. Robotics and Automation, 2008) Used for FIG. 9 is a diagram showing the results. FIG. 9A shows the calculation time of each image. The upper graph (long-time side) is ISC, and the lower graph is the result of PIRF. FIG. 9B shows the integration time, and similarly, ISC takes longer than PIRF. FIG. 9C shows the recognition rate and the time for testing the first 100 sheets. In the figure, the top graph is IS, and the lower three graphs show the results of PIRF-75, PIRF-50, and PIRF-25. PIRF-75, PIRF-50, and PIRF-25 are obtained by reducing PIRF to 75%, 50%, and 25%, respectively.

  From FIGS. 9 (a) and 9 (b), it was confirmed that it can be executed clearly at a higher speed than the conventional ISC. The accuracy at that time was 40.29% for the ISC of the conventional method, but 93.48% when using PIRF, and it was confirmed that the accuracy was also high.

  According to the present embodiment, it was confirmed that PIRF can be used for additional appearance-based topological mapping such as ISC and can be executed at higher speed than ISC.

DESCRIPTION OF SYMBOLS 1 Position estimation apparatus 10 Position estimation part 11 Area database 12 Storage part 13 Estimation result output part 14 Optimization part 20 Feature-value extraction part 21 Local feature-value extraction part 22 Feature-value matching part 23 Continuous feature-value selection part 24 Invariant feature-value calculation Part

Claims (15)

Feature amount extraction means for extracting an invariant feature amount from an input image consisting of continuous images taken continuously;
Position estimation means for estimating a position based on the invariant feature quantity extracted by the feature quantity extraction means;
The feature amount extraction means includes:
Local feature extraction means for extracting a local feature from each of the input images;
About the local feature amount extracted by the local feature amount extraction unit, a feature amount matching unit that performs matching between the continuous input images;
Continuous feature quantity selecting means for selecting a local feature quantity matched between a predetermined number of consecutive images by the feature quantity matching means as a continuous feature quantity;
A position estimation device comprising: an invariant feature amount calculating means for obtaining an average of the continuous feature amounts as an invariant feature amount. The position estimating means stores an area database in which a teacher image is used as an input image and the invariant feature amount extracted from a plurality of the teacher images is associated with area identification information indicating an area from which the invariant feature amount is extracted. Means,
Taking an image of the surroundings as an input image, matching each of a plurality of local feature amounts of the input image extracted by the local feature amount extraction means with the invariant feature amount registered in the area database, The position estimation apparatus according to claim 1, further comprising estimation result output means for voting on an area corresponding to an invariant feature amount that matches a local feature amount and estimating an area having the maximum number of votes as a current position.   The position estimation device according to claim 1, wherein the position estimation unit recognizes the area as an unlearned area when the number of votes is less than a predetermined threshold.   The position estimation device according to any one of claims 1 to 3, wherein the local feature amount is a feature amount of SIFT (Scale Invariant Feature Transformation) and / or SURF (Speed Up Robustness Features). Having optimization means for optimizing the database;
5. The position estimation device according to claim 1, wherein the optimization unit deletes redundant individual invariant feature amounts among a plurality of individual invariant feature amounts constituting the invariant feature amount. 6.   The optimization unit re-uses the teacher image, votes for the individual invariant feature used to identify the teacher image, and deletes the redundant individual feature according to the voting result. 5. The position estimation device according to 5.   The robot apparatus according to claim 5, wherein the optimization unit performs the optimization offline.   The robot apparatus according to claim 5, wherein the optimization unit performs the optimization online. Having optimization means for optimizing the database;
The position estimation device according to claim 1, wherein the optimization unit deletes redundant data for the invariant feature amount and the area identification information registered in the area database.   The position estimation device according to claim 9, wherein the redundant data has a low probability of moving again to the area indicated by the area identification information. A feature amount extraction step of extracting an invariant feature amount from an input image composed of continuous images taken continuously;
A position estimation step of estimating a position based on the invariant feature amount extracted by the feature amount extraction unit;
The feature amount extraction step includes:
A local feature amount extracting step for extracting a local feature amount from each of the input images;
A feature amount matching step for matching between the continuous input images for the local feature amount extracted by the local feature amount extraction step;
A continuous feature amount selection step of selecting a local feature amount matched between a predetermined number of consecutive images by the feature amount matching step as a continuous feature amount; and
A position estimation method comprising: an invariant feature amount calculating step of obtaining an average of the continuous feature amounts as an invariant feature amount. A local feature amount extracting means for extracting a local feature amount from each of the input images composed of continuous images taken continuously;
About the local feature amount extracted by the local feature amount extraction unit, a feature amount matching unit that performs matching between the continuous input images;
Continuous feature quantity selecting means for selecting a local feature quantity matched between a predetermined number of consecutive images by the feature quantity matching means as a continuous feature quantity;
An invariant feature amount calculating means for obtaining an average of the continuous feature amounts as an invariant feature amount.   The feature quantity extraction device according to claim 12, wherein the local feature quantity is a feature quantity of SIFT (Scale Invariant Feature Transformation) and / or SURF (Speed Up Robustness Features). A local feature extraction step of extracting a local feature from each of the input images consisting of continuous images taken continuously;
A feature amount matching step for matching between the continuous input images for the local feature amount extracted by the local feature amount extraction step;
A continuous feature quantity selection step of selecting a local feature quantity matched between a predetermined number of consecutive images by the feature quantity matching means as a continuous feature quantity;
An invariant feature amount calculating step of obtaining an average of the continuous feature amounts as an invariant feature amount. A program for causing a computer to execute a position estimation operation for estimating a position based on a feature amount,
A local feature extraction step of extracting a local feature from each of the input images consisting of continuous images taken continuously;
A feature amount matching step for matching between the continuous input images for the local feature amount extracted by the local feature amount extraction step;
A continuous feature quantity selection step of selecting a local feature quantity matched between a predetermined number of consecutive images by the feature quantity matching means as a continuous feature quantity;
And an invariant feature amount calculating step for obtaining an average of the continuous feature amounts as an invariant feature amount.