WO2014073204A1 - 特徴量抽出装置及び場所推定装置 - Google Patents
特徴量抽出装置及び場所推定装置 Download PDFInfo
- Publication number
- WO2014073204A1 WO2014073204A1 PCT/JP2013/006550 JP2013006550W WO2014073204A1 WO 2014073204 A1 WO2014073204 A1 WO 2014073204A1 JP 2013006550 W JP2013006550 W JP 2013006550W WO 2014073204 A1 WO2014073204 A1 WO 2014073204A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- feature amount
- feature
- invariant
- matching
- extracted
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
- G06V20/36—Indoor scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
- G06V20/38—Outdoor scenes
- G06V20/39—Urban scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/11—Technique with transformation invariance effect
Definitions
- the present invention relates to a feature amount extraction device, method and program for extracting local feature amounts from an input image, and a location estimation device, method and program using the feature amount extraction device.
- PIRF has the following problems.
- local feature quantities hereinafter also simply referred to as feature quantities or feature points
- invariant feature quantities the local feature quantities appearing in images of several consecutive frames are extracted as invariant feature quantities.
- feature quantities or feature points local features whose positions do not change in space are extracted.
- it is a quantity or a local feature quantity whose position is changed, if it appears continuously in the time direction, it is regarded as an invariant feature quantity and becomes an extraction target. That is, the local feature quantity that does not change the position and the dynamic local feature quantity that changes the position are not separated and are treated as equivalent to the invariant feature quantity.
- a local feature amount of a dynamic object that is originally not suitable for location estimation such as a moving person, is also used.
- the accuracy and calculation speed were adversely affected.
- the present invention has been made to solve such a problem, and is a feature amount extraction device, method and program capable of extracting a position invariant local feature amount, and a location estimation device, method and program equipped with the feature amount extraction device.
- the purpose is to provide.
- the location estimation apparatus refers to a feature amount extraction unit that extracts a position invariant feature amount from an input image, a database in which each registered location and a position invariant feature amount are stored in correspondence with each other, A matching means for obtaining matching with a registered place; a similarity calculating means for calculating a similarity including a registered place in the vicinity of the selected registered place when the matching is equal to or greater than a predetermined threshold; and A location recognition unit that recognizes that the input image is a registered location when the input image is equal to or greater than a predetermined threshold, and the feature amount extraction unit is configured to input each of the input images including continuous images taken continuously, A local feature amount extracting unit that extracts a local feature amount, and a feature amount matching unit that performs matching between successive input images with respect to the local feature amount extracted by the local feature amount extracting unit.
- corresponding feature quantity selection means for selecting a feature quantity matched between successive images by the feature quantity matching means, and position invariant feature quantity extraction for obtaining a position invariant feature quantity based on the corresponding feature quantity
- position-invariant feature amount extraction unit extracts, as the position-invariant feature amount, a corresponding feature amount whose position change is equal to or less than a predetermined threshold among the corresponding feature amounts.
- the location estimation method refers to a feature amount extraction step for extracting an invariant feature amount from an input image, and a database in which each registered location and the invariant feature amount are stored in correspondence with each other.
- a matching step for obtaining matching a similarity calculating step for calculating a similarity including a registered location in the vicinity of the selected registered location when the matching is equal to or greater than a predetermined threshold, and the similarity is a predetermined value
- a location determination step for determining that the input image is a registered location when the input image is equal to or greater than a threshold value
- the feature amount extraction step includes a local feature from each of the input images composed of continuously captured images.
- a local feature extraction step for extracting a quantity; a feature matching process for matching between successive input images for the local feature extracted in the local feature extraction process; and A corresponding feature amount selecting step for selecting a feature amount matched between successive images in the collection amount matching step as a corresponding feature amount; a position invariant feature amount extracting step for obtaining a position invariant feature amount based on the corresponding feature amount;
- the position-invariant feature quantity extraction step extracts a corresponding feature quantity whose position change is not more than a predetermined threshold value as the position-invariant feature quantity from the corresponding feature quantities.
- the feature amount extraction apparatus includes a local feature amount extraction unit that extracts a local feature amount from each of input images that are continuously captured images, and a local feature extracted by the local feature amount extraction unit.
- a feature amount matching unit that performs matching between successive input images, a corresponding feature amount selection unit that selects a feature amount that has been matched between consecutive images by the feature amount matching unit, and Position-invariant feature quantity extraction means for obtaining a position-invariant feature quantity based on the corresponding feature quantity, wherein the position-invariant feature quantity extraction means has a position change of a predetermined threshold value or less in the corresponding feature quantity
- Corresponding feature amounts are extracted as the position-invariant feature amounts.
- the feature amount extraction method includes a local feature amount extraction step of extracting a local feature amount from each of input images composed of continuous images taken continuously, and the local feature amount extracted in the local feature amount extraction step.
- a feature amount matching step for matching feature values between successive input images, and a corresponding feature amount selection step for selecting a feature amount matched between consecutive images in the feature amount matching step as a corresponding feature amount
- a position-invariant feature amount extraction step for obtaining a position-invariant feature amount based on the corresponding feature amount
- the position-invariant feature amount extraction step includes a change in position of the corresponding feature amount equal to or less than a predetermined threshold
- the corresponding feature quantity is extracted as the position invariant feature quantity.
- the program according to the present invention is a program for causing a computer to execute the place estimation method or the feature amount extraction method.
- a feature quantity extraction device, method and program capable of extracting a position-invariant local feature quantity as a robust feature quantity, and a location estimation apparatus, method and program using the feature quantity extraction apparatus.
- FIG. 6 illustrates an ICGM for a one-way approach.
- FIG. 6 shows an ICGM of a bidirectional approach. It is a figure which shows the comparison of a one way approach and a two way approach. It is a figure which shows the comparison of a one way approach and a two way approach. It is a figure which shows the extraction experiment result of the feature-value by ICGM.
- a technique for extracting a feature quantity whose position does not change over a long period of time in the environment that is, a position invariant feature quantity, and using this for location estimation.
- the environment includes static local feature quantities, that is, feature quantities whose positions do not change over a long period of time. For example, there are many pedestrians at a railway station, but the feature amounts of these pedestrians are not considered static feature amounts because their positions generally change in a short time.
- the position of the feature quantity related to elements such as walls and signs does not change over a long period. It is preferable to use such a position invariant feature amount for the location estimation.
- the present invention is applied to a place estimation apparatus for estimating a place, which is mounted on a mobile robot apparatus or the like.
- FIG. 1 is a block diagram showing a location estimation apparatus according to an embodiment of the present invention.
- the location estimation apparatus 10 includes a feature quantity extraction unit 11 that extracts a position invariant feature quantity from an input image that is a series of continuously captured images, a common dictionary 12, a matching unit 13, a similarity calculation unit 14, and a location recognition unit 15.
- the feature quantity extraction unit 11 includes a local feature quantity extraction unit 21, a feature quantity matching unit 22, a corresponding feature quantity selection unit 23, and a position-invariant feature quantity extraction unit 24.
- the local feature amount extraction unit 21 extracts a local feature amount from each input image.
- the feature amount matching unit 22 performs matching between successive input images with respect to the local feature amount extracted by the local feature amount extraction unit 21.
- the corresponding feature quantity selection unit 23 extracts the feature quantity matched between successive images by the feature quantity matching unit as the corresponding feature quantity.
- the feature extraction method include SIFT (Scale Invariant Feature Transformation) and SURF (Speed Up Robustness Features).
- the position invariant feature amount extraction unit 24 is a processing unit that performs the characteristic processing of the present invention. Among the corresponding feature amounts extracted by the corresponding feature amount selection unit 23, the feature amount (position invariant) whose position has not changed. Only the feature amount is extracted.
- this position-invariant feature extraction method is referred to as ICGM (Incremental Center of Gravity Matching).
- the matching unit 13 refers to a database in which a place and a position invariant feature amount of the place are associated with each other, matches the input image with the registered place, and calculates a matching score.
- the similarity calculation unit 14 calculates the similarity including the registration location in the vicinity of the selected registration location.
- the place recognition unit 15 determines that the input image is a registered place when the similarity is equal to or greater than a predetermined threshold.
- FIG. 2 is a flowchart showing a location estimation method according to the present embodiment.
- the continuous image required by ICGM refers to an image continuously photographed at a predetermined frame rate (for example, two frames per second).
- a predetermined frame rate for example, two frames per second.
- the local feature quantity extraction unit 21 extracts a local feature quantity using an existing local feature quantity extraction method (step S1).
- the local feature quantity extraction unit 21 can use a feature quantity extraction technique such as SIFT (Scale Invariant Feature Transformation) or SURF (Speed Up Robustness Features), for example.
- SIFT Scale Invariant Feature Transformation
- SURF Speed Up Robustness Features
- the present invention is not limited to SIFT and SURF, and other local feature quantities can be used.
- the performance of the existing feature amounts is inherited as it is, and it becomes possible to extract and describe as features that are robust against changes in illumination.
- SIFT extracts 2000 to 3000 or more feature quantities as local feature quantities.
- the SURF extracts 200 to 300 local feature amounts, the calculation amount is small.
- Feature matching unit 22 uses the image I t acquired at the current time t, and an image I t-1 obtained at time t-1 of the immediately preceding matching of local features between these successive images I do.
- the matching can be performed by various known methods used in, for example, SIFT (Scale Invant Feature Transformation) or SURF (Speed Up Robustness Features). For example, a matching score is calculated using the feature amount extracted from each image, and if the matching score is equal to or greater than a predetermined threshold, both local feature amounts are regarded as matched.
- the position-invariant feature amount extraction unit 24 a set p of the corresponding feature quantity, using p ', extracts the position invariant features in the image I t of the current time t (step S2).
- the position invariant feature amount extraction processing algorithm is shown in the flowchart of FIG. 3 and the list of FIG. The algorithm will be described below with reference to the flowchart of FIG.
- Step 1 Two sets of corresponding local feature values are selected from two consecutive images. That is, from the set p of the corresponding feature value of the image I t, selects two local feature quantity p 0, p 1. Further, the local feature amounts p ′ 0 and p ′ 1 are selected from the set p ′ of the corresponding feature amounts of the image It -1 .
- p 0 and p ′ 0 , and p 1 and p ′ 1 are pairs determined to be matched with each other by the feature amount matching unit 22.
- Step 3 Compare the vectors CGV0 and CGV1, and if they are not similar, return to Step 1.
- the local feature amounts p 0 and p 1 are recognized as position-invariant feature amounts. That is, if the difference between the two vectors is equal to or smaller than a predetermined threshold value and
- both vectors are similar means that the geometric positional relationship between the local features p 0 , p 1 and p ′ 0 , p ′ 1 has hardly changed between the two images. That is, the positions of the feature points p 0 and p 1 can be regarded as unchanged.
- Step 5 Subsequently, the remaining corresponding feature amounts of the image I t and the image I t ⁇ 1 are sequentially tested for whether or not the position is unchanged.
- a pair of corresponding local feature values p 2 and p ′ 2 is selected from the images I t and I t ⁇ 1 . This selection can be performed in the order of indexes, for example.
- Step 6 These vectors are compared, and if they are similar, the selected local feature is recognized as a position-invariant feature. That is, if the difference between the two vectors is equal to or smaller than the threshold Thr and
- the fact that both vectors are similar means that the geometric positional relationship between the center of gravity CG0 and the local feature amount p 2 and the center of gravity CG1 and the local feature amount p ′ 2 is hardly changed between the two images. ; that is, the position of the feature point p 2 can regarded as immutable.
- Step 7 the position invariant feature amount p 2 extracted from the image I t, is deleted from the p, is stored in the variable P R. Similarly, the feature quantity p ′ 2 of the image It -1 is deleted from p ′ and stored in the variable P ′ R. Further, in each of the images, the centroid of the previous centroid CG0 and p 2, and the previous center of gravity of the center of gravity CG1 and p '2, respectively are calculated and these and new centroids CG0, CG1.
- Step 8 On the other hand, if
- Step 9 When the test is completed for all local features included in p and p ′, that is, when p and p ′ are air raids, the process is terminated. At this time, the local feature quantity included in the P R is the position invariant features. This completes the position invariant feature amount extraction process.
- the matching unit 13 calculates the matching score s m with reference to the common dictionary 12 (step S3).
- the common dictionary 12 holds models m, m + 1, m + 2,... Which are sets of feature values of each place for each of consecutive places (places) L m , L m + 1 , L m + 2 .
- Matching score s m of the image I t, the model m locations L m is calculated by Equation (2).
- s m n m ⁇ num_appear (2)
- s m indicates the matching score of the model m is a characteristic quantity set Place L m
- the similarity calculation unit 14 calculates a second state score (first estimated value) b m in consideration of adjacent places (step S4).
- a second state score (first estimated value) b m in consideration of adjacent places.
- the matching score of these adjacent places is predicted to be approximately the same as or slightly lower than the matching score s m of L m . That is, for example, even if s m is a high score, if s m ⁇ 1 or s m + 1 is 0, the value of the matching score s m is strange, that is, the location cannot be estimated correctly.
- the second state score b m weighted by the Gaussian function p t (m, i) is obtained by the following equation (3).
- w indicates the number of adjacent places to be considered. For example, if the frame rate is constant, the value of w can be set to 1, for example, if the speed is fast, and the value of w can be set to 2, if the speed is slow.
- the image I t is consistent with a model m, i.e., it is also possible to determine that a known location (known place), in this embodiment
- the certification rate is further improved by normalizing the second state score b m .
- the normalized score (second estimated value) b_norm m can be obtained from equation (4) (step S5).
- n is a value corresponding to the moving speed of the place estimation device, and can be the maximum number of position-invariant feature values obtained by ICGM.
- the similarity calculation unit 14 calculates the normalized score B_norm m, location discriminating section 15, the value is larger than a predetermined threshold value, matching the image I t and the model m, i.e., a known location (place) (Steps S6 and S7).
- the image I t is, if they match the model m, by adding the shift-invariant feature amounts that were not included in the original model m to the model m, it is possible to update the characteristic quantity of the model m.
- the feature amount of each place is stored as an index as in Patent Document 3, it is only necessary to increase the index, and an increase in memory capacity can be suppressed. Further, the feature amount of the model m does not increase the memory capacity if, for example, the first-in first-out method is adopted.
- the location certification unit 15 recognized as indicating a new location (place) the image I t (step S8), and for example, the image I t was taken and location registers the position invariant feature quantity extracted from the image I t to a common dictionary 12.
- the feature quantity extraction unit 11 extracts feature quantities that exist continuously at substantially the same position in the time direction as robust feature quantities. Thereby, since the feature quantity which moves with time can be separated, the feature quantity effective for the location recognition can be extracted effectively.
- the centroid of the robust feature quantity is sequentially updated, and the robustness of other feature quantities is determined using the centroid.
- the center of gravity includes information on the positional relationship between the feature amounts, the robustness including the position information can be tested by using the center of gravity.
- the center of gravity is easy to calculate and can be processed at high speed.
- the center of gravity used in the feature point robustness test is the center of gravity of all feature points determined to be robust up to that point. In other words, it is not necessary to refer to all the position information of a huge amount of other feature values, and if the relationship with only one center of gravity is evaluated, the stability of the position of feature points can be evaluated. can do.
- the one-way approach is an approach for extracting a position-invariant feature amount by comparing the current image with a past image.
- Position invariant feature quantity extracted in this way is well robust from the feature value extracted only from the image I t (SIFT, SURF due).
- significant loss of position invariant features may occur. Specific cases where loss can occur will be described later.
- FIG. 6 shows the ICGM concept of the bidirectional approach.
- the current image I t extracts the past image I t-1 and the position invariant feature amount A by comparing, then the current image I t, and a future image I t + 1
- the position invariant feature quantity B is extracted by comparison.
- a logical OR C A ⁇ B both, and the position invariant feature of the image I t.
- the bidirectional approach can extract the position invariant feature more effectively. Specifically, the speed, movement, etc. of the camera can affect the position-invariant feature values that can be extracted.
- the inventor has considered two situations that can occur when using a known single lens camera. There are a case where the camera rotates at a constant speed and a case where the camera approaches or moves away from an object at infinity at a constant speed. And in both of these two typical situations, the two-way approach was found to be superior to the one-way approach.
- T Duration the time (from t ⁇ 1 to t) required to extract the feature quantity by the unidirectional approach. It is assumed that the position invariant feature amount is uniformly distributed in the visual field.
- T Duration is used with the same meaning as described above.
- the viewing angles in the vertical and horizontal directions be ⁇ and ⁇ . It is assumed that the position-invariant feature amount is uniformly distributed in the field of view.
- ⁇ ⁇ T Duration ⁇ ⁇ / d (11)
- FIG. 8 shows a comparison between the one-way approach and the two-way approach.
- the bi-directional approach can extract position-invariant features from a dynamic environment more effectively than the unidirectional approach.
- the main differences between these two approaches are: In the one-way approach, only feature quantities that have existed in the environment before the present time are extracted as position-invariant feature quantities.
- feature quantities existing in the environment from the present to the future are also extracted as position invariant feature quantities.
- the bidirectional approach is effective in either of two typical situations related to camera movement. Since general camera motion can be broken down into a combination of these simple situations, it can be generally said that the bi-directional approach can extract more robust features than the one-way approach.
- ICGM of one-way approach and two-way approach is advantageous for PIRF.
- PIRF is also a technique used to extract robust feature values from successive images.
- the feature quantity extracted in the one-way approach ICGM approaches the feature quantity extracted by PIRF.
- the window size 2 that is, when two images are used, a sufficiently robust feature amount cannot be extracted. If a more robust feature value is to be extracted by PIRF, it is necessary to increase the window size.
- Feature amount extraction experiment by ICGM This experiment is an experiment for confirming the accuracy of feature amounts extracted by ICGM. Feature quantities are extracted from a plurality of images using ICGM and SURF, and whether or not matching is achieved between the plurality of images of these feature quantities is compared.
- the data sets used in this experiment are both taken from the indoor environment (taken indoors).
- the environment also includes several moving objects.
- the right spray bottle surrounded by an ellipse is moving in the preceding and following images.
- the shooting range of the camera is also moved in the horizontal direction.
- FIG. 9B shows a situation in which feature points are extracted from two images by SURF and corresponding feature points are matched with each other.
- corresponding feature points are connected by bright lines. If the matching is correct, all the bright lines should be horizontal, but in this image you can see that many bright lines are tilted. That is, in this example, the matching includes many errors.
- matching is also performed on the moved object.
- FIG. 9C shows a situation in which position-invariant feature amounts are extracted from two images by ICGM and corresponding feature points are matched. In this image, most of the bright lines are horizontal, and it can be seen that matching is performed correctly. In addition, the moved object is not a matching target and is ignored.
- This experiment is not strictly SLAM, but is suitable for testing the accuracy of the location recognition of lCGM.
- the data set used for this experiment was an image taken at a rate of 0.5 frames per second using a handheld camera (resolution was resized to 480 * 320). At the time of filming, Shibuya Station was crowded with many people. The route taken for learning data acquisition was about 80 meters, and the learning time was 5 minutes (FIG. 10).
- FIG. 11 shows a comparison in the case of using the two-way approach or the one-way approach when using ICGM. It can be seen that the bi-directional approach can obtain more position invariant features than the unidirectional approach.
- Fig. 13 shows the experimental results.
- the solid line indicates the route where the location was learned.
- the dots indicate the coordinates where the location has been successfully recognized. It can be seen that the place learned in the first lap of the route is accurately recognized in the second lap.
- Fig. 14 shows the accuracy of this experiment.
- the accuracy of location recognition using ICGM is superior to PIRF-nav 2.0 (the method described in Patent Document 3 and Non-Patent Document 1) and known FAB-MAP.
- Proposed method real-time
- Proposed method uses a two-way approach in the location estimation phase.
- Proposed method non-real-time
- Proposed method is more accurate than Proposed method (real-time) because the number of extracted feature quantities is larger.
- FAB-MAP is the fastest because it is a batch processing technique.
- the image I t + 1 is also required. In other words, information about future events (images) is required. That is, it is necessary to extract a feature value of the image I t after obtaining the image I t + 1, can not be the feature amount extraction in real time at time t, take some time lag. Therefore, in a real-time system such as a robot, the bi-directional approach cannot be used in the place recognition phase where real-time performance is required. In this case, it is necessary to use a one-way approach. However, even in a real-time system, it is possible to use a bidirectional approach in a dictionary creation phase where real-time properties are not required. Also, in applications such as pedestrian navigation, strict real-time properties are not required so it is possible to improve system performance by using a bidirectional approach in both the dictionary creation and location recognition phases. is there.
- the location can be identified from the image, and the dictionary can be updated online. Therefore, for example, when combined with a mobile video shooting function, the following applications are possible.
- the server can analyze the image and return where it is located, and what kind of facilities and shops are around.
- the search moving image sent from the user can be used as data for updating the dictionary and map at the same time. For this reason, the dictionary and the map can always be updated. In conventional car navigation systems and the like, it is basically impossible to update map data, or it takes considerable time and money to update.
- each base station Since there are base stations that share and manage service areas in the mobile phone network, each base station should have a map of the area in charge and update it. In other words, a huge dictionary is not necessary, and memory and calculation speed can be greatly saved. In the future, wearable vision (camera) like glasses is likely to appear, and such glasses can always identify the location and present useful information.
- camera wearable vision
- Image distortion correction Camera lens characteristics can cause image distortion. In order to extract the corresponding feature amount and the position invariant feature amount with higher accuracy from the image, it is preferable that the image does not have such distortion.
- OpenCV As a technique for performing image distortion correction, for example, OpenCV is known. According to OpenCV, showing internal parameter (f x, f y, c x, c y), the coefficient showing the radial distortion (radial strain) (k 1, k 2) , tangential distortion (the distortion in the circumferential direction)
- the coefficients (p 1 , p 2 ) are acquired by performing camera calibration, and distortion correction can be performed using the acquired internal parameters and distortion coefficients. Note that the internal parameters and distortion coefficients described above are values unique to the camera.
- the local feature amount extraction unit 21 performs the above-described distortion correction processing before extracting the local feature amount from the image. Thereby, the corresponding feature quantity selection unit 23 and the position invariant feature quantity extraction unit 24 can extract the corresponding feature quantity and the position invariant feature quantity with better accuracy.
- the corresponding feature amount selection unit 23 has described the process of extracting a set of corresponding feature amounts.
- the set of corresponding feature amounts may include a set of feature amounts that are erroneously determined as corresponding feature amounts although they do not truly correspond.
- the inventor has developed an order constraint as an additional method for eliminating a set of feature amounts that are determined to be erroneously matched in this way.
- W (a, b, c, d, e)
- W ′ (a ′, b ′, c ′, d ′, e ′) are extracted from two images.
- Corresponding feature sets are shown.
- a and a ′, b and b ′, c and c ′, and e and e ′ are sets of feature quantities that are correctly matched.
- the corresponding feature amount selection unit 23 obtains a relative distance vector D i , D i ′ of the point i.
- D a (b, c, d, e).
- b, c, d, and e are sorted in the order from the point closest to the point a to the point farthest from the point a.
- D a ′ (d ′, b ′, c ′, e ′).
- the corresponding feature amount selection unit 23 obtains an index offset by using D i and D i ′ of the points i and i ′.
- FIG. 16 shows an offset calculation method. S-th element of D i is W b, W is matched with its W b 'b is D i' when that is the k th element of the offset s
- Equation (12) shows the definition of diff (D i , D i ′ ).
- diff (D i , D i ′ ) is not an affine invariant quantity and is not sensitive to the noise ratio. Therefore, diff normal obtained by normalizing diff (D i , D i ′ ) is considered.
- the diff normal can be calculated as shown in Expression (13) using the average ⁇ diff and the standard deviation ⁇ diff .
- the corresponding feature quantity selection unit 23 calculates diff normal for a set of feature quantities i and i ′.
- diffnormal > TOC the combination of the feature quantities i and i ′ is determined to be excluded from the set of corresponding feature quantities, that is, matched erroneously.
- T OC is an arbitrary threshold value.
- the position invariant feature amount extraction unit 24 has described the process of extracting the position invariant feature amount. However, the inventor has a method for calculating the position invariant feature amount with higher accuracy as a technique. An area constraint was developed.
- FIG. 17 shows an example of affine transformation.
- rotation and reduction are performed between two images.
- W (a, b, c, d) and W ′ (a ′, b ′, c ′, d ′) are sets of corresponding feature amounts related to two images.
- o and o ′ indicate centroids of one or more points included in W and W ′.
- S aob / S abcd S a′o′b ′ / S a′b′c′d ′
- S aob / S abc S a′o′b ′ / S a′b′c ′ Yes
- S aob / S aoc S a′o′b ′ / S a′o′c ′
- a quantity that is invariant to affine transformation like this area ratio is called an affine invariant, and such a property is called affine invariant.
- the position-invariant feature quantity extraction unit 24 calculates the total area S ⁇ formed by the feature points included in W by Expression (14).
- the position-invariant feature amount extraction unit 24 calculates the center of gravity o of the feature points included in W by Expression (15).
- the position-invariant feature quantity extraction unit 24 calculates the deviation of the area ratio of the figure formed using a certain feature point i by using the equation (16). Note that o is the center of gravity, and j is an arbitrary feature point other than i.
- the processing according to this algorithm 2 is as follows.
- Step1 First, initial values of the feature value sets W tmp and W ′ tmp are set to W and W ′. For each of W tmp and W ′ tmp , the centroids o and o ′ and the total areas S ⁇ and S ′ ⁇ are calculated by the equations (14) and (15). The size of W tmp, i.e. the number of feature amounts included in W tmp, save as SizePrevious.
- Step 2 Paying attention to the set of corresponding feature quantities i and i ′ included in W tmp and W ′ tmp , (Hereinafter referred to as AveDev) is calculated according to the equation (17). If where AveDev> T AC, corresponding feature quantity i, the set of i 'is qualified as not being affine transformation. Therefore, i and i ′ are removed from W tmp and W ′ tmp , respectively. The AveDev determination process is performed for all the corresponding feature amounts included in W tmp and W ′ tmp .
- the size of W tmp is compared with the SizePrevious stored in Step 1. If both sizes are the same, it is assumed that all the corresponding feature amounts to be removed have been removed, and the process ends. On the other hand, if the two sizes are different, the removal of the corresponding feature amount is still in progress, so the process returns to Step 1 and continues.
- Algorithm 3 shown in FIG. 19 is a process for correcting the calculation result of algorithm 2.
- the position-invariant feature amount extraction unit 24 re-inspects the feature amount excluded by the algorithm 2 by using the centroids o and o ′ at the end of the processing of the algorithm 2 by the algorithm 3. Thereby, at the initial stage of calculation of the algorithm 2, that is, when the reliability of the centroids o and o ′ is still low, the feature amounts are erroneously excluded by the algorithm 2 and should be originally position-invariant feature amounts. The feature amount can be saved without omission.
- the processing according to algorithm 3 is as follows.
- Step1 First, for each of W tmp and W ′ tmp , the centroids o and o ′ and the total areas S ⁇ and S ′ ⁇ are calculated according to equations (14) and (15).
- the size of W tmp i.e. the number of feature amounts included in W tmp, save as SizePrevious.
- Step 2 Paying attention to the set of corresponding feature quantities i and i ′ included in W and W ′, if i and i ′ are not included in W tmp and W ′ tmp , AveDev is calculated by Expression (17).
- a AVEDEV ⁇ T AC corresponding feature quantity i, i 'set of reliable centroid o, o' makes to the determination that the affine transformation. Therefore, i and i ′ are transferred to W tmp and W ′ tmp , respectively.
- Such determination processing is performed for all the corresponding feature amounts included in W and W ′.
- the size of W tmp is compared with the SizePrevious stored in Step 1.
- the process ends.
- the contents of W tmp and W ′ tmp are output as W AC and W ′ AC .
- the process returns to Step 1 and continues.
- the similarity s cg (z t , z c ) between the current location z c and the comparison target location z t can be calculated by the equation (18).
- N pair is the size of the set of corresponding feature amounts acquired from the images of the two locations by the corresponding feature amount selection unit 23.
- the set of corresponding feature values here is a set before the correction based on the above-described distance constraint is performed. That is, S Affine indicates the degree of coincidence of feature amounts before and after a series of processing due to distance constraints and area constraints. Note that 0 ⁇ S Affine ⁇ 1.
- D (′) can be calculated by Equation (21).
- S Dispersion is an index for more precisely evaluating the similarity between two images including affine-invariant feature values.
- S Dispersion has the effect of decreasing the similarity as the average of the distances between all feature points included in the set of corresponding feature amounts and the center of gravity o is significantly different between the two images.
- the two images are photographed. It can be determined that the places are not the same. Note that 0 ⁇ S Displacement ⁇ 1.
- N zt and N zc indicate the total number of local feature amounts acquired at the location z t and the location z c .
- the similarity is calculated using a position-invariant feature set with less noise extracted by geometric constraints such as distance constraint and area constraint, so that the similarity calculation with higher accuracy can be performed. Is possible.
- affine-ICGM real-time
- affine-ICGM real-time
- Racall 97.5%
- Precision 98.5%
- Total Processing Time 194.3 sec. Met. That is, affine-ICGM (real-time) is superior in both accuracy and processing time to ICGM (real-time) and ICGM (non-real-time) of the above-described embodiment.
- the present invention has been described on the assumption that the feature points are two-dimensional.
- the feature point may be an arbitrary dimension of three or more dimensions as long as it can be acquired from the environment.
- Kinect registered trademark
- depth information can be acquired in addition to two-dimensional image information, so that a three-dimensional feature point can be extracted.
- position-invariant feature quantities can be extracted by the algorithms shown in FIGS. X and Y regardless of the number of dimensions of feature points. That is, this algorithm can be applied if the topology can be defined for the feature points. For example, it can be applied to similarity determination of gene sequences.
- the current position (location) can be estimated from the camera image without using the known current position (location) detection method such as GPS. For example, it can be applied to navigation in a robot that moves indoors or in places where GPS reception is difficult, or a smartphone with a camera.
- Non-transitory computer readable media include various types of tangible storage media.
- non-transitory computer-readable media examples include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random access memory)).
- the program may be supplied to the computer by various types of temporary computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves.
- the temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
Description
図1は、本発明の実施の形態にかかる場所推定装置を示すブロック図である。場所推定装置10は、連続して撮影された連続画像からなる入力画像から位置不変特徴量を抽出する特徴量抽出部11、共通辞書12、マッチング部13、類似度算出部14及び場所認定部15を有する。また、特徴量抽出部11は、局所特徴量抽出部21、特徴量マッチング部22、対応特徴量選択部23、及び位置不変特徴量抽出部24を有する。
先ず、特徴量抽出部11が、入力画像Itから位置不変特徴量を抽出する処理について説明する。
CG0=(p0+p1)/2,CG1=(p’0+p’1)/2 ・・・(1)
位置不変特徴量が抽出されたならば、つづいて、マッチング部13が、共通辞書12を参照してマッチングスコアsmを求める(ステップS3)。共通辞書12は、環境内における連続する場所(プレイス)Lm、Lm+1、Lm+2・・それぞれについて、各プレイスの特徴量のセットであるモデルm、m+1、m+2・・を保持している。画像Itと、場所Lmのモデルmとのマッチングスコアsmは、式(2)により求められる。
sm=nm×num_appear ・・・(2)
(位置不変特徴量の抽出-一方向アプローチと両方向アプローチ)
実施の形態1では、位置不変特徴量を抽出する手法(ICGM)として、時刻tにおける画像Itと、時刻t-1における画像It-1とを使用する方法について説明した。これを一方向アプローチと称する。実施の形態2では、より効果的に位置不変特徴量を抽出できる手法について説明する。以下、これを両方向アプローチと称する。
λα=TDuration/TDisappear ・・・(7)
s=4・tan(η)tan(θ)・d2 ・・・(8)
s’=4・tan(η)tan(θ)・(d-TDuration・ν)2 ・・・(9)
λβ=TDuration・ν/d ・・・(11)
次に、実施の形態3として、実施の形態1及び実施の形態2として示した特徴量抽出手法、及び場所認識手法を、実際の環境に適用した実験例を開示する。併せて、本発明の効果について説明する。
この実験は、ICGMにより抽出される特徴量の精度を確認するための実験である。複数の画像から、ICGM及びSURFを使用して特徴量を抽出し、これらの特徴量の複数の画像間でマッチングが取れているかを比較する。
以下の実験は、ICGMを、SLAM(Simultaneous Localization and Mapping)に適用したものである。特許文献3及び非特許文献1の同様の実験においては、PIRFにより抽出された特徴量が使用されたが、本実験ではICGMにより抽出された特徴量を使用する。
この実験に使用するデータセットは、手持ちカメラを使用し、毎秒0.5フレームのレートで撮影された画像である(解像度は480*320にリサイズした)。この実験では、撮影時の環境は混雑してはいなかったものの、いくつかの動的な物体(車、人間)が存在していた(図12)。学習データの取得のため撮影したルートは約170メートルであり、学習時間は9.5分であった。
上述したように、本発明においては、画像から場所同定ができ、辞書のオンライン更新が可能である。そこで、例えば、携帯の動画撮影機能と組み合わせると、以下のような応用が可能である。
実施の形態5では、画像の歪みや、回転(rotation)、剪断変形(shearing)、平行移動(translation)、拡大縮小(scaling)等に対しても頑健な位置不変特徴量を抽出し、より精度の高い場所推定を実現する手法を提案する。
カメラのレンズ特性が、画像の歪みを引き起こすことがある。画像からより精度の高い対応特徴量、及び位置不変特徴量を抽出するためには、画像にこのような歪みのないことが好ましい。
上述の実施の形態では、対応特徴量選択部23が対応特徴量のセットを抽出する処理について説明した。ところで、対応特徴量のセットには、真に対応するものでないのに、誤って対応特徴量と判定された特徴量の組も含まれ得る。発明者は、このように誤ってマッチするものと判断された特徴量の組を排除するための追加的手法として、順序制約を開発した。
上述の実施の形態では、位置不変特徴量抽出部24が、位置不変特徴量を抽出する処理について説明したが、発明者は、より高精度な位置不変特徴量の計算を行う為の手法として、面積制約を開発した。
まず、対応特徴量のセットW,W’を入力する。ここで、W,W’は、いずれも上述の順序制約により抽出されたものであることが好ましい。
まず、特徴量のセットWtmp、W’tmpの初期値を、W,W’とする。このWtmp、W’tmpのそれぞれについて、重心o,o’、総面積SΣ、S’Σを式(14)、(15)により計算する。また、Wtmpのサイズ、すなわちWtmpに含まれる特徴量の数を、SizePreviousとして保存する。
Wtmp、W’tmpに含まれる対応特徴量i,i’の組に着目し、
かかる処理ののち、Wtmpのサイズと、Step1で保存したSizePreviousとを比較する。ここで、両者のサイズが同じであれば、除去すべき対応特徴量はすべて除去されたものとして、処理を終了する。一方、両者のサイズが異なる場合は、対応特徴量の除去はまだ進行中であるから、Step1に戻り処理を続行する。
まず、対応特徴量のセットW,W’およびWtmp、W’tmpを入力する。ここで、W,W’は、アルゴリズム2に入力されたものと同じ対応特徴量のセットである。またWtmp、W’tmpは、アルゴリズム2の出力である。
まず、Wtmp、W’tmpのそれぞれについて、重心o,o’、総面積SΣ、S’Σを式(14)、(15)により計算する。また、Wtmpのサイズ、すなわちWtmpに含まれる特徴量の数を、SizePreviousとして保存する。
W、W’に含まれる対応特徴量i,i’の組に着目し、i,i’がWtmp、W’tmpに含まれていない場合は、AveDevを式(17)により計算する。ここでAveDev<TACであれば、対応特徴量i,i’の組は、信頼できる重心o,o’により、アフィン変換されたものと判定されたことになる。よって、i,i’をWtmp、W’tmpにそれぞれ繰り入れる。かかる判定処理を、W、W’に含まれる対応特徴量すべてについて実施する。
かかる処理ののち、Wtmpのサイズと、Step1で保存したSizePreviousとを比較する。ここで、両者のサイズが同じであれば、救済すべき位置不変特徴量はすべて救済されたものとして、処理を終了する。このとき、Wtmp、W’tmpの内容をWAC、W’ACとして出力する。一方、両者のサイズが異なる場合は、位置不変特徴量の救済はまだ進行中であるから、Step1に戻り処理を続行する。
上述の一連の処理により、精度の高い位置不変特徴量のセットWACが得られる。これを用いることによって、より精度の高い場所推定が可能となる。
最後に、実施の形態5として示した特徴量抽出手法、及び場所認識手法を、実際の環境に適用した実験例を開示する。
なお、本発明は上述した実施の形態のみに限定されるものではなく、本発明の要旨を逸脱しない範囲において種々の変更が可能であることは勿論である。
11 特徴量抽出部
12 共通辞書
13 マッチング部
14 類似度算出部
15 場所認定部
21 局所特徴量抽出部
22 特徴量マッチング部
23 対応特徴量選択部
24 位置不変特徴量抽出部
Claims (14)
- 入力画像から位置不変特徴量を抽出する特徴量抽出手段と、
各登録場所と位置不変特徴量が対応づけられて保存されているデータベースを参照し、入力画像と登録場所とのマッチングを求めるマッチング手段と、
マッチングが所定の閾値以上である場合に、選ばれた登録場所の近傍の登録場所を含めて類似度を算出する類似度算出手段と、
前記類似度が所定の閾値以上である場合に、当該入力画像が登録場所であると認定する場所認定手段とを有し、
前記特徴量抽出手段は、
連続して撮影された連続画像からなる入力画像それぞれから、局所特徴量を抽出する局所特徴量抽出手段と、
前記局所特徴量抽出手段により抽出された局所特徴量について、連続する入力画像間でマッチングをとる特徴量マッチング手段と、
前記特徴量マッチング手段により連続する画像間でマッチングが取れた特徴量を対応特徴量として選択する対応特徴量選択手段と、
前記対応特徴量に基づき位置不変特徴量を求める位置不変特徴量抽出手段とを有し、
前記位置不変特徴量抽出手段は、前記対応特徴量のうち、位置の変化が所定のしきい値以下である対応特徴量を、前記位置不変特徴量として抽出する
場所推定装置。 - 前記位置不変特徴量抽出手段は、連続画像からなる入力画像それぞれにおいて、既に抽出された前記位置不変特徴量の重心と、前記対応特徴量の1つとにより形成されるベクトルを定義し、
連続画像からなる入力画像それぞれにおいて形成された前記ベクトルの差分が、所定のしきい値以下であった場合、前記対応特徴量の1つを、前記位置不変特徴量として新たに抽出し、
前記重心を、前記重心と前記新たな位置不変特徴量との重心で更新する処理を、
すべての前記対応特徴量について繰り返し実行する
請求項1記載の場所推定装置。 - 前記位置不変特徴量抽出手段は、連続画像からなる入力画像それぞれに存在し、互いに対応する前記対応特徴量の組を2組ランダムに選択し、
連続画像からなる入力画像それぞれにおいて、2つの対応特徴量により形成されるベクトルを定義し、
前記ベクトルの差分が、所定のしきい値以下であった場合、これらの前記対応特徴量を、前記位置不変特徴量として最初に抽出する
請求項1記載の場所推定装置。 - 前記局所特徴量は、SIFT(Scale Invariant Feature Transformation)又はSURF(Speed Up Robustness Features)の少なくともいずれか一方の特徴量である
請求項1記載の場所推定装置。 - 前記位置不変特徴量抽出手段は、時刻tに撮影された画像と、前記tより前である時刻t-1に撮影された画像と、から抽出される第1の位置不変特徴量と、前記時刻tにかかる画像と、前記tより後である時刻t+1に撮影された画像と、から抽出される第2の位置不変特徴量と、の論理和を、前記位置不変特徴量として出力する
請求項1記載の場所推定装置。 - 前記対応特徴量選択手段は、所定の前記対応特徴量と、他の前記対応特徴量と、の相対距離に基づいて、前記特徴量マッチング手段による前記マッチングの誤りを検査し、誤って前記マッチングされた前記対応特徴量を除去する
請求項1記載の場所推定装置。 - 前記位置不変特徴量抽出手段は、前記対応特徴量に関係するアフィン不変量を検出することにより、前記位置不変特徴量を抽出する
請求項1記載の場所推定装置。 - 連続して撮影された連続画像からなる入力画像それぞれから、局所特徴量を抽出する局所特徴量抽出手段と、
前記局所特徴量抽出手段により抽出された局所特徴量について、連続する入力画像間でマッチングをとる特徴量マッチング手段と、
前記特徴量マッチング手段により連続する画像間でマッチングが取れた特徴量を対応特徴量として選択する対応特徴量選択手段と、
前記対応特徴量に基づき位置不変特徴量を求める位置不変特徴量抽出手段とを有し、
前記位置不変特徴量抽出手段は、前記対応特徴量のうち、位置の変化が所定のしきい値以下である対応特徴量を、前記位置不変特徴量として抽出する
特徴量抽出装置。 - 前記位置不変特徴量抽出手段は、連続画像からなる入力画像それぞれにおいて、既に抽出された前記位置不変特徴量の重心と、前記対応特徴量の1つとにより形成されるベクトルを定義し、
連続画像からなる入力画像それぞれにおいて形成された前記ベクトルの差分が、所定のしきい値以下であった場合、前記対応特徴量の1つを、前記位置不変特徴量として新たに抽出し、
前記重心を、前記重心と前記新たな位置不変特徴量との重心で更新する処理を、
すべての前記対応特徴量について繰り返し実行する
請求項8記載の特徴量抽出装置。 - 前記位置不変特徴量抽出手段は、連続画像からなる入力画像それぞれに存在し、互いに対応する前記対応特徴量の組を2組ランダムに選択し、
連続画像からなる入力画像それぞれにおいて、2つの対応特徴量により形成されるベクトルを定義し、
前記ベクトルの差分が、所定のしきい値以下であった場合、これらの前記対応特徴量を、前記位置不変特徴量として最初に抽出する
請求項8記載の特徴量抽出装置。 - 前記局所特徴量は、SIFT(Scale Invariant Feature Transformation)又はSURF(Speed Up Robustness Features)の少なくともいずれか一方の特徴量である、
請求項8記載の特徴量抽出装置。 - 前記位置不変特徴量抽出手段は、時刻tに撮影された画像と、前記tより前である時刻t-1に撮影された画像と、から抽出される第1の位置不変特徴量と、前記時刻tにかかる画像と、前記tより後である時刻t+1に撮影された画像と、から抽出される第2の位置不変特徴量と、の論理和を、前記位置不変特徴量として出力する
請求項8記載の特徴量抽出装置。 - 前記対応特徴量選択手段は、所定の前記対応特徴量と、他の前記対応特徴量と、の相対距離に基づいて、前記特徴量マッチング手段による前記マッチングの誤りを検査し、誤って前記マッチングされた前記対応特徴量を除去する
請求項8記載の特徴量抽出装置。 - 前記位置不変特徴量抽出手段は、前記対応特徴量に関係するアフィン不変量を検出することにより、前記位置不変特徴量を抽出する
請求項8記載の特徴量抽出装置。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/440,768 US9396396B2 (en) | 2012-11-06 | 2013-11-06 | Feature value extraction apparatus and place estimation apparatus |
EP13854124.8A EP2922022B1 (en) | 2012-11-06 | 2013-11-06 | Feature value extraction device and location inference device |
JP2014545574A JP6265499B2 (ja) | 2012-11-06 | 2013-11-06 | 特徴量抽出装置及び場所推定装置 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2012-244540 | 2012-11-06 | ||
JP2012244540 | 2012-11-06 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2014073204A1 true WO2014073204A1 (ja) | 2014-05-15 |
Family
ID=50684329
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2013/006550 WO2014073204A1 (ja) | 2012-11-06 | 2013-11-06 | 特徴量抽出装置及び場所推定装置 |
Country Status (4)
Country | Link |
---|---|
US (1) | US9396396B2 (ja) |
EP (1) | EP2922022B1 (ja) |
JP (1) | JP6265499B2 (ja) |
WO (1) | WO2014073204A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2017145711A1 (ja) * | 2016-02-24 | 2017-08-31 | Soinnホールディングス合同会社 | 特徴量抽出方法及び特徴量抽出装置 |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9898677B1 (en) | 2015-10-13 | 2018-02-20 | MotionDSP, Inc. | Object-level grouping and identification for tracking objects in a video |
KR102465332B1 (ko) * | 2015-12-29 | 2022-11-11 | 에스케이플래닛 주식회사 | 사용자 장치, 그의 제어 방법 및 컴퓨터 프로그램이 기록된 기록매체 |
CN107515006A (zh) * | 2016-06-15 | 2017-12-26 | 华为终端(东莞)有限公司 | 一种地图更新方法和车载终端 |
JP7046506B2 (ja) * | 2017-06-12 | 2022-04-04 | キヤノン株式会社 | 情報処理装置、情報処理方法及びプログラム |
CN108256463B (zh) * | 2018-01-10 | 2022-01-04 | 南开大学 | 基于esn神经网络的移动机器人场景识别方法 |
CN110471407B (zh) * | 2019-07-02 | 2022-09-06 | 无锡真源科技有限公司 | 一种模组自动调节的自适应定位系统及方法 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2010033447A (ja) * | 2008-07-30 | 2010-02-12 | Toshiba Corp | 画像処理装置および画像処理方法 |
JP2010115307A (ja) | 2008-11-12 | 2010-05-27 | Sozosha:Kk | モイスチャ処理装置 |
JP2010238008A (ja) * | 2009-03-31 | 2010-10-21 | Fujitsu Ltd | 映像特徴抽出装置、及びプログラム |
JP2011053823A (ja) | 2009-08-31 | 2011-03-17 | Tokyo Institute Of Technology | 特徴量抽出装置及び方法、並びに位置推定装置及び方法 |
JP2011215716A (ja) | 2010-03-31 | 2011-10-27 | Toyota Motor Corp | 位置推定装置、位置推定方法及びプログラム |
WO2011145239A1 (ja) * | 2010-05-19 | 2011-11-24 | 国立大学法人東京工業大学 | 位置推定装置、位置推定方法及びプログラム |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4672175B2 (ja) * | 2000-05-26 | 2011-04-20 | 本田技研工業株式会社 | 位置検出装置、位置検出方法、及び位置検出プログラム |
KR101591471B1 (ko) * | 2008-11-03 | 2016-02-04 | 삼성전자주식회사 | 물체의 특징 정보를 추출하기 위한 장치와 방법, 및 이를 이용한 특징 지도 생성 장치와 방법 |
US10108620B2 (en) * | 2010-04-29 | 2018-10-23 | Google Llc | Associating still images and videos |
-
2013
- 2013-11-06 JP JP2014545574A patent/JP6265499B2/ja active Active
- 2013-11-06 WO PCT/JP2013/006550 patent/WO2014073204A1/ja active Application Filing
- 2013-11-06 US US14/440,768 patent/US9396396B2/en not_active Expired - Fee Related
- 2013-11-06 EP EP13854124.8A patent/EP2922022B1/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2010033447A (ja) * | 2008-07-30 | 2010-02-12 | Toshiba Corp | 画像処理装置および画像処理方法 |
JP2010115307A (ja) | 2008-11-12 | 2010-05-27 | Sozosha:Kk | モイスチャ処理装置 |
JP2010238008A (ja) * | 2009-03-31 | 2010-10-21 | Fujitsu Ltd | 映像特徴抽出装置、及びプログラム |
JP2011053823A (ja) | 2009-08-31 | 2011-03-17 | Tokyo Institute Of Technology | 特徴量抽出装置及び方法、並びに位置推定装置及び方法 |
JP2011215716A (ja) | 2010-03-31 | 2011-10-27 | Toyota Motor Corp | 位置推定装置、位置推定方法及びプログラム |
WO2011145239A1 (ja) * | 2010-05-19 | 2011-11-24 | 国立大学法人東京工業大学 | 位置推定装置、位置推定方法及びプログラム |
Non-Patent Citations (2)
Title |
---|
N. TONGPRASIT; A. KAWEWONG; O. HASEGAWA: "Pirf-nav 2: speededup online and incremental appearance-based slam in highly dynamic environment", IEEE WORKSHOP ON APPLICATIONS OF COMPUTER VISION (WACV, 2011 |
See also references of EP2922022A4 |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2017145711A1 (ja) * | 2016-02-24 | 2017-08-31 | Soinnホールディングス合同会社 | 特徴量抽出方法及び特徴量抽出装置 |
JPWO2017145711A1 (ja) * | 2016-02-24 | 2018-04-05 | Soinnホールディングス合同会社 | 特徴量抽出方法及び特徴量抽出装置 |
US10366305B2 (en) | 2016-02-24 | 2019-07-30 | Soinn Inc. | Feature value extraction method and feature value extraction apparatus |
Also Published As
Publication number | Publication date |
---|---|
JPWO2014073204A1 (ja) | 2016-09-08 |
EP2922022B1 (en) | 2020-01-01 |
JP6265499B2 (ja) | 2018-01-24 |
US20150294157A1 (en) | 2015-10-15 |
EP2922022A4 (en) | 2016-10-12 |
EP2922022A1 (en) | 2015-09-23 |
US9396396B2 (en) | 2016-07-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6265499B2 (ja) | 特徴量抽出装置及び場所推定装置 | |
US11144786B2 (en) | Information processing apparatus, method for controlling information processing apparatus, and storage medium | |
JP2019087229A (ja) | 情報処理装置、情報処理装置の制御方法及びプログラム | |
JP5771413B2 (ja) | 姿勢推定装置、姿勢推定システム、および姿勢推定方法 | |
TW202143100A (zh) | 圖像處理方法、電子設備及電腦可讀儲存介質 | |
CN104281840B (zh) | 一种基于智能终端定位识别建筑物的方法及装置 | |
CN112991447A (zh) | 一种动态环境下视觉定位与静态地图构建方法及系统 | |
CN111094895B (zh) | 用于在预构建的视觉地图中进行鲁棒自重新定位的系统和方法 | |
US10366305B2 (en) | Feature value extraction method and feature value extraction apparatus | |
CN108960211A (zh) | 一种多目标人体姿态检测方法以及系统 | |
JP2018523881A (ja) | データを位置合わせする方法及びシステム | |
US9098744B2 (en) | Position estimation device, position estimation method, and program | |
JP5557189B2 (ja) | 位置推定装置、位置推定方法及びプログラム | |
Tamjidi et al. | 6-DOF pose estimation of a portable navigation aid for the visually impaired | |
CN114608521B (zh) | 单目测距方法及装置、电子设备和存储介质 | |
Terashima et al. | A visual-SLAM for first person vision and mobile robots | |
Poglitsch et al. | [POSTER] A Particle Filter Approach to Outdoor Localization Using Image-Based Rendering | |
JP2014102805A (ja) | 情報処理装置、情報処理方法及びプログラム | |
JP2018124177A (ja) | 床面判断方法 | |
CN116343143A (zh) | 目标检测方法、存储介质、路侧设备及自动驾驶系统 | |
Brauer et al. | Generative 2d and 3d human pose estimation with vote distributions | |
CN115775325B (zh) | 一种位姿确定方法及装置、电子设备和存储介质 | |
CN112927291B (zh) | 三维物体的位姿确定方法、装置及电子设备和存储介质 | |
Johns et al. | Urban position estimation from one dimensional visual cues | |
CN110660134B (zh) | 三维地图构建方法、三维地图构建装置及终端设备 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 13854124 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2014545574 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14440768 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2013854124 Country of ref document: EP |