JP5862344B2 - Image processing apparatus, prior information update method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, a prior information update method, and a program.

  Mobile robots are created in advance by detecting distinctive image areas as landmarks from the surrounding environment using a camera mounted on the main body in order to perform autonomous movement freely in various complex environments. Autonomous driving is performed by collating with the image database and map.

  For example, a technique is known in which a movement path of an autonomous mobile robot is configured by a link that connects several landmarks that can be directly moved by the autonomous mobile robot among landmarks that are fixed positions on the movement path.

  When recognizing landmarks with a camera, the recognition results vary depending on the shooting conditions and the state of the device, so the accuracy of the landmark map update information can be determined by judging the accuracy of the recognition results and the difficulty of recognition. An example of improving is also known.

  However, when considering long-term operation of robots, the environment measured to create a map using landmarks in advance may change.For example, after a certain period of time, a new landmark appears or the original landmark appears. The mark may disappear. In addition, there may be a case where the landmarks are not observed again for a long period of time when the robot autonomously moves due to the fact that the landmarks are registered incorrectly at the time of map creation using the landmarks. In this way, since a map created in advance may cause inconsistencies with the latest environment, the map created in advance is used for a certain period of time in order to maintain the accuracy of robot self-position estimation stably. It is necessary to update every time.

  For this reason, there is known a technique that includes a local map and a global map in a map used for self-location identification, stores landmark candidates and landmark matching results in the local map, and uses them for the next landmark matching. At this time, a landmark candidate having no corresponding landmark is newly registered as a landmark, while those already registered that are not observed over a long period of time are deleted.

Japanese Patent Laid-Open No. 10-143243 JP 2008-51612 A JP 2008-165275 A

  However, when updating the map, it is important to determine the criteria for determining the landmark to be deleted in order to maintain the accuracy of autonomous movement of the robot. Therefore, a method for determining that a landmark already registered in the map has disappeared is necessary. However, in the above example, it is determined that the landmark has disappeared using the sensor, and it is preferable to use an image acquired by the camera in order to make a more accurate determination.

  In view of the above problem, when autonomous movement is performed using information on the surrounding environment acquired in advance such as a map, the robot can update the information such as a map with higher accuracy using an image acquired by the camera. An image processing apparatus, a prior information update method, and a program capable of maintaining the self-position estimation and the accuracy of autonomous movement are provided.

  An image processing apparatus according to one aspect includes a prior information acquisition unit, an autonomous information acquisition unit, a disappearance belief degree calculation unit, and an update unit. The prior information acquisition unit acquires the prior information including the position information of the landmark and the image feature acquired from the prior image obtained by photographing the surrounding environment of the route for autonomous movement. The autonomous information acquisition unit acquires autonomous information including position information and image features of a feature region related to the landmark acquired from an autonomous image obtained by capturing an image of the surrounding environment during autonomous movement of the autonomous mobile device. The extinction belief calculation unit calculates an extinction belief indicating the certainty of whether or not the landmark has disappeared from the surrounding environment based on the prior image and the autonomous image based on the prior information and the autonomous information. The updating unit determines whether or not to delete the landmark from the prior information based on the disappearance belief degree, and updates the prior information.

  The prior information update method which is another aspect is a method in which the image processing apparatus acquires the prior information and the autonomous information, calculates the disappearance belief degree, and updates the prior information. In this method, the image processing apparatus obtains prior information including landmark position information and image features obtained from a prior image obtained by photographing the surrounding environment of a route on which autonomous movement is performed. The image processing apparatus acquires autonomous information including position information and image features related to the landmark acquired from an autonomous image obtained by capturing the surrounding environment during the autonomous movement of the autonomous mobile apparatus. The image processing device has an extinction belief degree indicating the certainty of whether or not the landmark based on the prior image and the autonomous image has disappeared from the surrounding environment based on the prior information and the autonomous information. calculate. Further, the image processing apparatus determines whether or not to delete the landmark from the prior information based on the disappearance belief degree, and updates the prior information.

  Note that even a program for causing a computer to perform the method according to the present invention described above has the same operational effects as the method according to the present invention described above by causing the computer to execute the program. The problem that was solved is solved.

  According to the above-described aspect, when performing autonomous movement using a map, it is possible to maintain the accuracy of autonomous movement by updating the map with higher accuracy using an image acquired by a camera. An image processing apparatus, a prior information update method, and a program can be provided.

It is a figure which shows the hardware constitutions of the robot by one Embodiment. It is a block diagram which shows the function of the image processing apparatus of the robot by one embodiment. It is a figure which shows an example of the landmark database by one Embodiment. It is a figure which shows an example of the pattern ID list by one Embodiment. It is a figure which shows an example of the BOF table by one embodiment. It is a figure which shows an example of the landmark map by one embodiment. It is a figure which shows an example of the annihilation belief degree by one embodiment. It is a figure which shows the extraction area | region list by one Embodiment. It is a figure which shows the corresponding | compatible list by one Embodiment. It is a figure which shows the autonomous BOF table by one embodiment. It is a figure which shows the vector similarity table by one Embodiment. It is a figure which shows the related SLAM image table by one Embodiment. It is a figure which shows the autonomous BOF similarity list by one Embodiment. It is a figure which shows the BOF similarity comparison table by one Embodiment. It is a figure which shows the score list classified by pattern by one Embodiment. It is a figure which shows the annihilation belief degree by one Embodiment. It is a flowchart explaining the landmark database creation processing by one embodiment. It is a flowchart explaining the landmark map creation process by one Embodiment. It is a flowchart explaining the outline | summary of the operation | movement at the time of operation | movement of the robot by one Embodiment. It is a flowchart explaining the disappearance belief degree update process by one Embodiment. It is a flowchart explaining the disappearance belief degree update process by one Embodiment. It is a hardware block diagram of a standard computer.

  Hereinafter, a robot 1 according to an embodiment will be described with reference to the drawings. First, the configuration of the robot 1 will be described with reference to FIGS. FIG. 1 is a diagram illustrating a hardware configuration of the robot 1, and FIG. 2 is a block diagram illustrating functions of the image processing device 9 of the robot 1.

  As shown in FIG. 1, the robot 1 includes a stereo camera 3, a wheel unit 5, a sensor information acquisition device 7, an image processing device 9, and a storage device 17. The stereo camera 3 includes a camera 3-1 and a camera 3-2 that are arranged at a predetermined distance, and simultaneously captures an image of an object from two different directions, thereby providing information on the distance of the object together with the respective image information. It is a camera that can also record. The wheel unit 5 includes wheels that move the robot 1 and wheels / encoders 5-1 and 5-2 that include encoders that measure the rotation of the wheels. The wheel unit 5 moves in accordance with the control from the movement controller 15 of the image processing device 9 described later to move the robot 1 and outputs information by an encoder according to the rotation of the wheel. The sensor information acquisition device 7 acquires an output from the encoder of the wheel unit 5.

  The image processing device 9 includes an image acquisition device 11, an information processing device 13, and a movement controller 15. The image acquisition device 11 controls the operation of the stereo camera 3 and acquires a captured image. The information processing device 13 is an arithmetic processing device that acquires information on the encoder of the wheel unit 5 from the sensor information acquisition device 7 and performs processing described later based on the image acquired from the image acquisition device 11. Further, the information processing device 13 controls the wheel unit 5 by the movement controller 15 based on the information from the wheel unit 5 acquired via the sensor information acquisition device 7 and the processing result based on the image acquired from the image acquisition device 11. Output information. The information processing device 13 further stores information in the storage device 17 or reads information from the storage device 17.

  The movement controller 15 controls the operation of the wheels of the wheel unit 5 based on an instruction from the information processing device 13. The storage device 17 is a storage device capable of writing and reading as needed to store a control program for operating the robot 1 and various information necessary for control. The storage device 17 may include a readable storage device or a removable storage device. Note that the information processing apparatus 13 may control the operation of the robot 1 by reading and executing a control program stored in the storage device 17.

  As shown in FIG. 2, the image processing apparatus 9 has functions of a task manager 30, a simultaneous localization and mapping (SLAM) task control unit 32, an autonomous movement task control unit 34, and a maintenance task control unit 36. The task manager 30 controls operations of the SLAM task control unit 32, the autonomous mobile task control unit 34, and the maintenance task control unit 36. The SLAM task control unit 32 controls the operation of the SLAM unit 40, the autonomous movement task control unit 34 controls the operation of the autonomous movement unit 60, and the maintenance task control unit 36 controls the operation of the landmark deletion unit 100. To do.

  Hereinafter, moving the robot 1 along a route set in advance manually or the like before the operation of the robot 1 is referred to as advance movement. At this time, for example, images acquired every predetermined time by the stereo camera 3 are referred to as SLAM images, and various data described later such as landmark maps generated from the SLAM images are collectively referred to as SLAM products 90. In addition, when the robot 1 is operated, moving along a preset route while referring to the SLAM product 90 created by the SLAM unit 40 and the encoder information from the wheel unit 5 is called autonomous movement. In addition, when the robot 1 autonomously moves, an image captured by controlling the stereo camera 3 to cause the stereo camera 3 to capture images every predetermined time is referred to as an autonomous image.

  The SLAM unit 40 creates a SLAM product 90 that is referred to when the robot 1 performs autonomous movement. In addition, environmental measurement data of each sensor such as the stereo camera 3 and the wheel unit 5 is recorded at each time during advance movement, and is stored in a storage device in the form of a log. Further, the SLAM unit 40 creates a landmark map 97 to be described later using this log data (for example, an image photographed by the stereo camera 3) after the completion of the advance movement.

The SLAM unit 40 includes a SLAM image acquisition unit 42, a feature extraction unit 44, an image feature bag (BOF) Features calculation unit 46, a pattern registration unit 48, a self-position estimation unit 52, an encoder data acquisition unit 54, and a map generation unit. It has 56 functions.

  The SLAM image acquisition unit 42 acquires SLAM images from the stereo camera 3 at each time during the preliminary movement before the autonomous movement of the robot 1 and stores the SLAM image in the storage device 17. The SLAM image acquisition unit 42 is a part of the function of the image acquisition device 11. The feature extraction unit 44 extracts, for example, feature points or feature regions having strong features of the image (hereinafter collectively referred to as feature regions) from each SLAM image acquired by the SLAM image acquisition unit 42. The feature region is, for example, a corner, a line segment, a point, or a spot in the image.

  The BOF calculation unit 46 calculates a feature vector for the feature region extracted by the feature extraction unit 44. Further, the BOF calculation unit 46 refers to the feature vector pattern corresponding to each landmark registered in the storage device 17 via the pattern registration unit 48, and the similarity between the calculated feature vector and the registered feature vector. (Hereinafter referred to as vector similarity) is calculated. A method for calculating the vector similarity will be described later. The landmark is a feature area having identification information (pattern ID) and three-dimensional coordinate information.

  If the vector similarity is within a predetermined range, the BOF calculation unit 46 counts the extracted feature region as the same pattern as the feature vector corresponding to the landmark. If the vector similarity is outside the predetermined range, the calculated feature vector is Register as a new pattern. The BOF calculator 46 calculates the BOF by counting the appearance frequency of each feature vector pattern according to the classified result. The calculated appearance frequency for each pattern is called a BOF feature value. By comparing the BOF feature values, the similarity between the two images can be found. For example, when a scene similar to two images is captured, the BOF similarity value is high. On the other hand, when the BOF similarity is low, different scenes may be reflected in the two images.

  In this manner, the BOF calculation unit 46 creates or updates a landmark map 97, a landmark database 92, a pattern ID list 94, and a BOF table 98, which will be described later. The pattern registration unit 48 refers to the information stored in the storage device 17 and registers the landmark map 97, landmark database 92, pattern ID list 94, and BOF table 98 in the storage device 17.

  The landmark position measurement unit 50 measures the three-dimensional position of the detected landmark using the SLAM image acquired by the stereo camera 3 during the pre-movement based on the principle of stereo vision measurement, for example. The landmark position measurement unit 50 calculates a three-dimensional position by performing coordinate conversion on the measured global coordinate value of the landmark based on the global coordinate value of the stereo camera 3 of the robot 1.

  The encoder data acquisition unit 54 acquires encoder information from the wheel unit 5. The self-position estimation unit 52 estimates the self-position of the robot 1 based on the landmark position information from the landmark position measurement unit 50 and the encoder information from the encoder data acquisition unit 54. Note that the self-position estimation may be performed by using various conventional methods such as the method described in Patent Document 1 as long as the information using the encoder and the position information of the observed landmark are used. Can do. The map generation unit 56 generates a landmark map 97 from the information obtained as described above. Other SLAM products will be described later. The landmark map 97 is information that collects many landmarks and associates the pattern ID with the three-dimensional coordinate values of the stereo camera.

  The autonomous movement unit 60 includes an autonomous image acquisition unit 62, a feature vector extraction unit 64, an autonomous BOF generation unit 66, an association unit 68, a correspondence list creation unit 70, a related SLAM image extraction unit 72, a related landmark extraction unit 74, and a BOF. It has functions of a similarity calculation unit 76, a disappearance belief calculation unit 78, a disappearance belief update unit 80, and an autonomous movement control unit 82. The SLAM product 90 has a landmark database 92, a pattern ID list 94, an extinction belief table 96, a landmark map 97, and a BOF table 98 and is stored in the storage device 17.

The autonomous image acquisition unit 62 controls the stereo camera 3 when the robot 1 autonomously moves along a preset route with reference to the landmark map 97 generated by the SLAM unit 40, and the like. The stereo camera 3 is photographed every time, and the photographed image is acquired. The autonomous image acquisition unit 62 is a part of the function of the image acquisition device 11.
The feature vector extraction unit 64 extracts a feature region in the autonomous image acquired by the autonomous image acquisition unit 62, extracts a feature vector of the extracted feature region, and generates an extraction region list 110 described later.

  The association unit 68 performs association by calculating the vector similarity between the feature vector extracted by the feature vector extraction unit 64 during autonomous movement and the feature vector corresponding to each landmark recorded in advance. Do. Accordingly, the associating unit 68 generates a vector similarity table 116 described later. At this time, the feature vector corresponding to each landmark refers to the landmark database 92 of the SLAM product 90. The correspondence list creation unit 70 creates a correspondence list 112 to be described later by the association performed by the association unit 68.

  The related SLAM image extraction unit 72 extracts all SLAM images from which landmarks corresponding to the feature regions extracted by the feature vector extraction unit 64 have been extracted with reference to the landmark database 92, and a related SLAM image table described later. 118 is created. The related landmark extracting unit 74 extracts all the landmarks extracted by the related SLAM image extracting unit 72 while referring to a pattern ID list 94 described later for the related SLAM image extracted in relation to the autonomous image. And registered in the related SLAM image table 118.

  The autonomous BOF generating unit 66 calculates the autonomous BOF of the autonomous image by counting the feature area whose predetermined similarity with the feature area corresponding to the pattern ID is confirmed by the vector similarity as the appearance frequency of the landmark. Then, an autonomous BOF table 114 for each autonomous image described later is generated.

  The BOF similarity calculation unit 76 calculates a similarity (hereinafter referred to as “BOF similarity”) from the calculated autonomous BOF and the BOF table 98 generated by the SLAM unit 40, and stores an autonomous BOF similarity list 120 described later. Generate. A method for calculating the BOF similarity will be described later. In addition, the BOF similarity calculation unit 76 generates a BOF similarity comparison table 122 and a pattern-specific score list 124 described later from the landmark database 92, the related SLAM image table 118, and the autonomous BOF similarity list 120.

  The disappearance belief calculation unit 78 calculates the disappearance belief based on the calculated BOF similarity and the related landmarks. The disappearance belief is a value representing the certainty of whether or not the landmark has disappeared from the surrounding environment. The disappearance belief update unit 80 updates the disappearance belief stored in the disappearance belief table 96 to the disappearance belief calculated by the disappearance belief calculation unit 78.

  The autonomous movement control unit 82 acquires and sets a start point and an end point input from the outside, measures the environment with a sensor by the stereo camera 3, compares the measurement information with the landmark map 97, and determines its position and orientation. Along with estimation (self-position estimation), a control signal for movement is output. Further, the autonomous movement control unit 82 controls the movement so as to return to the target path when the robot 1 leaves the target path.

  The landmark deletion unit 100 performs maintenance of the SLAM product 90 by deleting the landmark in the landmark database 92 and the landmark map 97 based on the disappearance belief updated in the disappearance belief table 96.

  Subsequently, the SLAM product 90 will be described with reference to FIGS. 3 shows an example of the landmark database 92, FIG. 4 shows an example of the pattern ID list 94, FIG. 5 shows an example of the BOF table 98, and FIG. FIG. 7 is a diagram illustrating an example, and FIG. 7 is a diagram illustrating an example of the disappearance belief degree table 96.

  As shown in FIG. 3, the landmark database 92 is a database in which feature vectors of all feature regions extracted from all SLAM images are clustered. The landmark database 92 includes a pattern ID (F1 to F7 in the example of FIG. 3) corresponding to each landmark, a feature vector corresponding to each pattern ID, and a SLAM image in which a feature region corresponding to each pattern ID is extracted. Numbers are shown. For example, for pattern ID = F1, feature vector = (0.1, 0.2, 0.4,..., 1.0). Further, the landmark database 92 indicates that the feature region corresponding to the pattern ID = F1 is extracted with the SLAM images of the related SLAM image numbers = # 1, # 4, and # 5. The landmark database 92 is generated by the feature extraction unit 44, for example.

  As shown in FIG. 4, the pattern ID list 94 is a database obtained by clustering feature regions, and indicates pattern IDs detected in each SLAM image. For example, the pattern ID list 94 indicates that in the SLAM image with SLAM image number = # 1 (hereinafter referred to as SLAM image # 1), feature regions corresponding to the pattern IDs = F1 and F2 are extracted. The pattern ID list 94 is generated by the pattern registration unit 48, for example.

  As shown in FIG. 5, the BOF table 98 indicates the extraction frequency of the feature vector corresponding to the pattern ID in each SLAM image. For example, the BOF table 98 indicates that the feature area determined as pattern ID = F1 by the vector similarity is detected 100 times in the SLAM image # 1. Similarly, the BOF table 98 indicates that the feature region determined as pattern ID = F2 was detected 102 times, and the feature regions determined as pattern ID = F3, F4, F5 were not detected. The BOF table 98 is generated by the BOF calculation unit 46.

Here, a method for calculating the vector similarity will be described. A feature vector corresponding to the pattern ID = Fp (p is an integer of 1 or more) is a vector Xp = (x1, x2, x3,..., Xn) (where n is an integer of 1 or more), and the SLAM image When the feature vector corresponding to the feature region extracted from Yq = (y1, y2, y3,..., Yn) (q is an integer), the vector similarity Vpq is expressed by the following equation 1, for example. Is done.
Vpq = 1 / {| x1-y1 | + | x2-y2 | + ... + | xn-yn |}
... (Formula 1)
The BOF calculation unit 46 determines that the two feature vectors have the same pattern when the vector similarity Vpq is equal to or greater than a predetermined value.

  As illustrated in FIG. 6, the landmark map 97 includes a landmark ID, a pattern ID, and a three-dimensional position (for example, a predetermined point as the origin) corresponding to all the landmarks detected in advance movement. X, y, z coordinates). The landmark map 97 is generated by the map generation unit 56.

  As shown in FIG. 7, the disappearance belief table 96 indicates landmark IDs, pattern IDs, and disappearance beliefs corresponding to all landmarks detected in advance movement. The disappearance belief level table 96 is generated by, for example, the map generation unit 56, but when moving in advance, all disappearance beliefs are in an initial state and "0" is input.

  Next, information generated or updated by the autonomous mobile unit 60 will be described with reference to FIGS. 8 is a diagram showing an extraction area list, FIG. 9 is a diagram showing a correspondence list, FIG. 10 is a diagram showing an autonomous BOF table, FIG. 11 is a diagram showing a vector similarity table, and FIG. 12 is a related SLAM. It is a figure which shows an image table. 13 shows an autonomous BOF similarity list, FIG. 14 shows a BOF similarity comparison table, FIG. 15 shows a pattern-specific score list, and FIG. 16 shows disappearance belief degrees. It is.

  As illustrated in FIG. 8, the extraction region list 110 indicates an extraction region ID corresponding to a feature region extracted from the autonomous image (hereinafter referred to as an extraction region) and a corresponding feature vector. The extraction area list 110 indicates that, for example, the feature vector of the extraction area ID = G1 is feature vector = (0.2, 0.2, 0.3,..., 0.99). In the present embodiment, the extraction area list 110 is generated by the feature vector extraction unit 64.

  As shown in FIG. 9, the correspondence list 112 shows the extraction region ID and the pattern ID corresponding to the feature vector determined to be similar to the feature vector corresponding to the extraction region ID. Whether or not the feature vectors are similar is determined by the above-described vector similarity. The correspondence list 112 indicates, for example, that the extraction area ID = G1 corresponds to the pattern ID = F4. The correspondence list 112 is created by the correspondence list creation unit 70 in this embodiment.

  As shown in FIG. 10, the autonomous BOF table 114 indicates the extraction frequency of the feature region determined to be similar to the feature vector corresponding to the pattern ID in the autonomous image. The autonomous BOF table 114 indicates that the feature areas corresponding to the pattern IDs = F1, F3, F4, and F7 are extracted once, for example. The autonomous BOF table 114 is generated by the autonomous BOF generation unit 66 in the present embodiment.

  As shown in FIG. 11, the vector similarity table 116 indicates the similarity between the feature vector of the extraction region and the feature vector corresponding to the pattern ID. In the vector similarity table 116, for example, the vector similarity Vpq between the feature vector corresponding to the extraction region ID = G1 and the feature vector corresponding to the pattern ID = F1 to F7 is 0.5, 0.1 in order. ,..., 0.1. In this example, for example, since the feature vector corresponding to the extraction area ID = G1 has the highest vector similarity Vpq to the feature vector corresponding to the pattern ID = F4, G1 and F1 are associated with each other. The vector similarity table 116 is generated by the association unit 68 in the present embodiment.

  As shown in FIG. 12, the related SLAM image table 118 shows related pattern IDs related to one autonomous image. In the related SLAM image table 118, the corresponding pattern ID is a pattern ID corresponding to the extracted area ID in the corresponding list 112, and the pattern ID corresponding to the feature vector corresponding to the extracted area ID and the highest vector similarity Vpq. Is shown. The SLAM image number is an SLAM image number corresponding to each corresponding pattern ID, and is extracted with reference to the landmark database 92.

  The related SLAM image number is the number of all SLAM images extracted as the SLAM image number. The related pattern ID is a pattern ID corresponding to all feature areas detected by the image of the related SLAM image number, and is extracted with reference to the landmark database 92. The related SLAM image table 118 is generated by the related SLAM image extraction unit 72 in the present embodiment. In FIG. 12, for the convenience of explanation, the extraction area ID and the corresponding pattern ID are shown in the related SLAM image table 118, but these are the same contents as the correspondence list 112 and may be omitted.

  As shown in FIG. 13, the autonomous BOF similarity list 120 shows BOF similarity between one autonomous image and a plurality of SLAM images. For example, the BOF similarity between this autonomous image and SLAM image # 1 is 0.83. The autonomous BOF similarity list 120 is generated by the autonomous BOF generation unit 66 in the present embodiment.

Here, a method for calculating the BOF similarity will be described. In the SLAM image #k (k is an integer), the BOF frequency of the feature region corresponding to the pattern ID = F1 to Fm (m is an integer of 1 or more) = f1 to fm. Further, the BOF frequency of the feature region corresponding to the pattern ID = F1 to Fm in the autonomous image 1 (l is an integer of 1 or more) = g1 to gm. At this time, the BOF similarity Bkl is expressed by, for example, the following formula 2.
Bkl = 1 / {| f1-g1 | + | f2-g2 | + ... + | fm-gm |}
... (Formula 2)

  As illustrated in FIG. 14, the BOF similarity comparison table 122 is generated for one autonomous image, and indicates the BOF similarity Bkl corresponding to the pattern ID related to the autonomous image. In the BOF similarity comparison table 122, the related pattern ID in the autonomous image is extracted from the related SLAM image table 118 as the pattern ID. As the SLAM image, the corresponding SLAM image number is extracted from the landmark database 92 for each of the extracted related pattern IDs.

  As the BOF similarity, the BOF similarity Bkl is extracted from the autonomous BOF similarity list 120 for each SLAM image extracted from the landmark database 92. As the maximum similarity, the maximum BOP similarity is extracted for each pattern ID. As for the similarity after masking, among the extracted maximum similarities, the landmarks corresponding to the corresponding pattern IDs are zero for those having a characteristic value in the autonomous BOF table 114 in the autonomous image. Is replaced by The value after the replacement is the degree of similarity after masking. The BOF similarity comparison table 122 is generated by the BOF similarity calculation unit 76 in the present embodiment.

  As shown in FIG. 15, the pattern-specific score list 124 shows scores based on the BOF similarity used for the calculation of the disappearance belief described later. In the pattern-specific score list 124, the post-mask similarity in the BOF similarity comparison table 122 is associated with each pattern ID as a score.

  For example, since it is determined that the pattern ID = F1 is extracted from the autonomous image, the score = 0. For example, for pattern ID = F2, the largest BOF similarity among the related SLAM images # 1, # 3, # 4 is BOF similarity = 0.83 with the SLAM image # 1, Maximum similarity = 0.83. At this time, since it is not determined that the pattern ID = F2 is extracted in the autonomous image, masking is not performed, and score = maximum similarity = 0.83. In the present embodiment, the pattern-specific score list 124 is generated by the BOF similarity calculation unit 76.

  As shown in FIG. 16, the disappearance belief degree table 96 indicates the disappearance belief degree for each landmark. The disappearance belief is an updated value obtained by accumulating the scores calculated in the pattern-specific score list 124 every time the robot 1 performs autonomous movement. In FIG. 16, for example, in the case of landmark ID = L1, the disappearance belief before the current autonomous movement is 0.9, and the current score = 0 is added to this. For example, in the case of landmark ID = L2, the disappearance belief before the current autonomous movement is 0.2, and the current score = 0.83 is added to this. In the present embodiment, the score list for each pattern 124 is updated by the disappearance belief update unit 80. If the disappearance belief exceeds the reference value, the landmark deletion unit 100 deletes the corresponding landmark from the landmark map 97.

  Hereinafter, the operation of the robot 1 according to the present embodiment will be described with reference to a flowchart. FIG. 17 is a flowchart for explaining the landmark database creation process, and FIG. 18 is a flowchart for explaining the landmark map creation process. FIG. 19 is a flowchart for explaining the outline of the operation during operation of the robot 1, and FIGS. 20 and 21 are flowcharts for explaining the disappearance belief update process.

  First, the creation process of the landmark database 92 will be described with reference to FIG. As shown in FIG. 17, in creating the landmark database 92, first, the SLAM image acquisition unit 42 acquires a SLAM image, and the encoder data acquisition unit 54 acquires encoder data from the wheel unit 5 (S201). . At this time, the SLAM unit 40 stores time-series image data (also simply referred to as an image) from the stereo camera 3, three-dimensional information from the stereo camera 3, and odometry based on encoder data in the storage device 17.

  The SLAM image acquisition unit 42 extracts only the image data of the SLAM image, stores it in the storage device 17, and at least a list of SLAM images in which identification information of the acquired SLAM image is associated with the image data of the SLAM image ( (Not shown) is created (S202). The feature extraction unit 44 extracts one SLAM image from the list of SLAM images (S203), and extracts a feature region having a strong feature of the image (S204). At this time, the feature extraction unit 44 outputs, for example, the position of the feature region above the screen and the feature vector.

  The feature extraction unit 44 creates a feature region list (not shown) in which at least the extracted region identification information and the feature vector are associated with each other for the series of feature regions obtained in S204 (S205).

  The BOF calculator 46 creates a BOF table 98 using a feature area list (not shown) (S206), and registers the corresponding BOF frequency (S207). At this time, the BOF calculation unit 46 calculates the vector similarity Vpq between the extracted feature vector and the feature vector already registered in the BOF table 98, and is already registered when the vector similarity Vpq is equal to or greater than a predetermined value. Increase the frequency of extracting feature vectors. The BOF table 98 is information for distinguishing images.

  The feature extraction unit 44 extracts one feature region from the feature region list (S208), and determines whether or not the extracted feature region is in the landmark database 92 (S209). When the feature extraction unit 44 determines that the extracted feature area already exists in the landmark database 92 (S209: YES), the corresponding pattern ID is acquired from the landmark database 92 (S210). At this time, the feature extraction unit 44 calculates the vector similarity Vpq between the feature vector of the extracted feature region and the feature vector stored in the landmark database 92, and when the vector similarity Vpq is equal to or greater than a predetermined value, It is determined that the extracted feature area already exists in the landmark database 92. The feature extraction unit 44 may update the value of the feature vector of the corresponding pattern ID registered in the landmark database 92 in consideration of the acquired feature vector. Note that the landmark database 92 is in a state where all items of information are not registered in the initial state.

  If the feature extraction unit 44 determines that the extracted feature area is not yet registered in the landmark database 92 (S209: NO), a new pattern ID that is not used for the feature area is assigned (S211). Then, the feature extraction unit 44 registers the extracted feature vector in the landmark database 92 (S212). At this time, the feature extraction unit 44 calculates a vector similarity Vpq between the feature vector of the extracted feature region and the feature vector stored in the landmark database 92. The feature extraction unit 44 determines that the extracted feature region is not yet registered in the landmark database 92 when the vector similarity Vpq is less than a predetermined value. Further, the pattern registration unit 48 registers the extracted pattern ID in the pattern ID list 94 in association with the SLAM image number (S213).

  The feature extraction unit 44 determines whether or not the processing from S208 has been completed for all the feature regions (S214), and if not (S214: NO), returns the processing to S208. If the processing has been completed (S214: YES), the SLAM image acquisition unit 42 determines whether or not the processing has been completed for all SLAM images (S215). If the process has not ended (S215: NO), the SLAM image acquisition unit 42 returns the process to S203. When the process is completed (S215: YES), the SLAM task control unit 32 ends the process of creating the landmark database 92.

  Next, the landmark map creation process will be described with reference to FIG. As shown in FIG. 18, in creating the landmark map 97, the SLAM image acquisition unit 42 acquires a SLAM image, and the encoder data acquisition unit 54 acquires encoder data from the wheel unit 5 (S231). When data is already registered in the storage device 17, the SLAM image acquisition unit 42 acquires data from the storage device 17.

  The landmark position measurement unit 50 extracts one SLAM image from a list of SLAM images (not shown) (S232) and extracts one pattern ID from the pattern ID list 94 (S233).

  The landmark position measurement unit 50 determines whether or not the extracted pattern ID is in the landmark map 97 (S234). If the pattern ID is on the landmark map (S234: YES), the landmark position measuring unit 50 determines whether there are two or more corresponding landmarks (S235). (S235: NO), the process proceeds to S239. When the extracted pattern ID is not in the landmark map (S234: NO), the landmark position measuring unit 50 advances the process to S240.

  When there are two or more corresponding landmarks (S235: YES), the landmark position measuring unit 50 acquires all corresponding landmark positions from the landmark map 97 (S236), and the spatial distance is determined. One shortest corresponding landmark is extracted (S237). That is, since the same image pattern may exist in different places in the space, when the landmark map 97 is searched with the same pattern ID, it may be determined that a plurality of landmarks correspond. Therefore, when there are a plurality of search results, the landmark position measurement unit 50 narrows down the corresponding landmarks to one landmark having the shortest spatial distance.

  The landmark position measurement unit 50 determines whether or not the shortest spatial distance extracted in S237 is greater than or equal to a threshold value (S238). If it is equal to or greater than the threshold (S238: YES), the landmark position measurement unit 50 advances the process to S240. If it is not equal to or greater than the threshold (S238: NO), the landmark position measurement unit 50 determines that the already registered landmark has been observed again. Then, the landmark position measurement unit 50 updates the three-dimensional position of the corresponding landmark to a three-dimensional position corresponding to the pattern ID, or an intermediate value between the registered position and the position corresponding to the pattern ID (S239). ).

  If it is determined that the map generation unit 56 is not registered in the landmark map 97, the map generation unit 56 determines that the extracted pattern ID is a new landmark, and assigns a new landmark ID that is not used (S240). The self-position estimation unit 52 refers to odometry in the data acquired in S231 (S241), and estimates the position and orientation of the robot 1 (S242). The landmark position measurement unit 50 calculates the three-dimensional position of the feature area corresponding to the extracted pattern ID from the information of the odometry and the stereo camera 3 (S243). The map generation unit 56 registers the assigned landmark ID and the calculated three-dimensional position of the landmark (S244).

  In S245, the map generation unit 56 updates the landmark map 97 in the storage device 17 with the registered information (S244). The landmark map 97 is in a state where information is not registered in all items in the initial state, and the landmark ID, the pattern ID, and the three-dimensional position of the landmark are changed by repeating the processing up to S244. It will be registered.

  The landmark position measurement unit 50 determines whether or not processing has been performed for all pattern IDs (S246). If there is a pattern ID that has not been processed (S246: NO), the processing is repeated from S233. When the processing has been completed for all pattern IDs (S246: YES), the landmark position measurement unit 50 determines whether or not processing has been performed for all data (S247). When the processing has not been completed for all data (S247: NO), the landmark position measurement unit 50 repeats the processing from S231, and when the processing has been completed for all data (S247: YES), The creation process of the landmark map 97 ends.

  Next, an overview of operations during operation of the robot 1 will be described with reference to FIG. As shown in FIG. 19, for example, when starting operation of the robot 1 such as guidance work at a shopping center, the autonomous movement control unit 82 first operates the robot 1 by turning on the power of the robot 1. A standby state is made possible (S261). The autonomous movement control unit 82 waits until the autonomous movement task is started by the autonomous movement task control unit 34 (S262: NO). When a new movement task is instructed by the autonomous movement task control unit 34 (S262: YES), the autonomous movement control unit 82 acquires and sets the start point (usually the current position) and the end point input from the outside ( S263).

  The autonomous movement control unit 82 controls the movement controller 15 to start the autonomous movement of the robot 1 (S264). At this time, the autonomous movement control unit 82 estimates the current position of the robot 1 based on the sensor information from the stereo camera 3 and the wheel unit 5 and moves the robot 1 toward the destination without losing its own position and posture.

  The autonomous mobile unit 60 updates the disappearance belief degree (S265). In other words, the autonomous moving unit 60 should acquire an image from the stereo camera 3, measure the current landmark, and display it on the stereo camera 3 with the current position and orientation of the robot 1 based on the landmark map 97. Predict landmarks. In addition, the autonomous mobile unit 60 updates the disappearance belief of the existing landmark in the landmark map 97 based on the difference between actual observation and prediction. Details will be described later.

  The extinction belief is updated until the autonomous movement ends (S266: NO). When the autonomous movement ends (S266: YES), the autonomous movement unit 60 ends the execution of the autonomous movement control program, and the maintenance task control unit 36 scans the disappearance belief degree table 96 stored in the storage device 17. Then, it is determined whether there is a landmark to be deleted (S267). At this time, the maintenance task control unit 36 determines whether or not the disappearance belief of each landmark exceeds a predetermined threshold value, and if so, determines that the corresponding landmark is deleted.

  If there is no landmark to be deleted (S267: NO), the maintenance task control unit 36 returns the process to S261. If there is a landmark to be deleted (S267: YES), the maintenance task control unit 36 performs maintenance by deleting it from the SLAM product 90 (S268).

  Furthermore, the disappearance belief level update process will be described with reference to FIGS. As shown in FIGS. 20 and 21, in the disappearance belief update process, when the robot 1 performs a predetermined autonomous movement by the movement controller 15, the stereo camera 3 captures an autonomous scene, thereby acquiring an autonomous image. The unit 62 acquires an autonomous image (S281). The feature vector extraction unit 64 extracts a feature region from the acquired autonomous image (S282). At this time, the feature vector extraction unit 64 extracts all feature regions from one autonomous image. The feature vector extraction unit 64 associates the identification information and the feature vector for all the extracted regions, and creates the extraction region list 110 (S283). For example, in the extracted area list 110 of FIG. 8, the obtained extracted area identification information is G1, G2, G3, and G4.

  The feature vector extraction unit 64 extracts one extraction region from the list of extraction regions (S284). The associating unit 68 calculates a vector similarity based on the extracted feature vectors for all the extracted regions and the feature vector of the pattern ID registered in the landmark database 92, and creates a vector similarity table 116 (S285). ).

  The correspondence list creating unit 70 refers to the vector similarity table 116, associates an extraction region having a large vector similarity with a pattern ID, and adds the extracted region to the correspondence list 112 (S286). In the correspondence list 112, the identification information G1 and the pattern ID F4, the identification information G2 and the pattern ID F1, the identification information G3 and the pattern ID F7, and the identification information G4 and the pattern ID F3 correspond to each other. Here, the data of the extraction area that cannot be associated is discarded. The autonomous BOF generation unit 66 counts the number of extraction areas associated with each pattern ID based on the correspondence list 112, and creates the autonomous BOF table 114 (S287).

  The feature vector extraction unit 64 determines whether or not the processing has been completed for all the extraction areas (S288), and if not (S288: NO), repeats the processing from S284. When the process is completed (S288: YES), the feature vector extraction unit 64 advances the process to S289.

  The autonomous movement control unit 82 estimates the current position and posture of the robot 1 using the obtained correspondence list 112, and outputs a control signal corresponding to a preset route to the movement controller 15 (S289).

  The correspondence list creation unit 70 extracts one pair of extraction area and pattern ID from the correspondence list 112 (S290). The related SLAM image extraction unit 72 refers to the landmark database 92 to detect which SLAM image the feature region corresponding to the pattern ID extracted as a pair is detected at the stage of SLAM product generation. Get all the SLAM image numbers. Further, the related SLAM image extraction unit 72 associates the acquired SLAM image number with the extracted pair and sets it as the SLAM image number of the related SLAM image table 118 (S291).

  Here, there is a possibility that the feature region corresponding to the same pattern ID stored in the landmark database 92 is detected from a plurality of SLAM images. Since this information is also stored in the landmark database 92, the related SLAM image extraction unit 72 obtains the plurality of SLAM image numbers and pairs them with the autonomous image extraction region.

  For example, it is assumed that the extraction area = G1 and the pattern ID = F4 are extracted as a pair. At this time, the related SLAM image table 118 indicates that the feature region corresponding to the pattern ID F4 was detected from the SLAM images # 2 and # 4 when the SLAM product 90 was generated. Similarly, feature areas corresponding to pattern ID F1 are detected from SLAM images # 1, # 4, and # 5, and feature areas corresponding to pattern ID F7 are SLAM images # 7, # 8, # 9, It is shown that it was detected from # 10. In addition, it is shown that the feature region corresponding to the pattern ID F3 is detected from the SLAM images # 3, # 4, and # 5.

  Note that although the SLAM image numbers acquired in S291 may overlap, the related SLAM image extraction unit 72 extracts the acquired SLAM image numbers without duplication, and related SLAM images in the related SLAM image table 118. Number. Therefore, in the example of the related SLAM image table 118, the related SLAM images are SLAM images # 1, # 2, # 3, # 4, # 5, # 7, # 8, # 9, and # 10.

  The related landmark extraction unit 74 refers to the landmark database 92 and extracts all the pattern IDs related to the related SLAM image, thereby setting the related pattern ID of the related SLAM image table 118 (S292). For example, with respect to the pair-related SLAM image numbers = # 2 and # 4 extracted first, pattern IDs = F1 to F4 are extracted by the landmark database 92.

  The related SLAM image extraction unit 72 determines whether or not the processing of all pairs of extraction regions and pattern IDs in the correspondence list 112 has been completed (S293), and if not completed (S293: NO), from S290 Repeat the process.

  That is, according to the landmark database 92, when the SLAM product 90 is generated, the feature regions corresponding to the pattern ID = F1 and F2 are extracted from the SLAM image # 1, and the pattern ID = A feature region corresponding to F4 is extracted. Also, feature areas corresponding to pattern ID = F2, F3 are extracted from SLAM image # 3, and feature areas corresponding to pattern ID = F1, F2, F3, F4 are extracted from SLAM image # 4. Yes. Similarly, when the pattern IDs of the feature areas extracted from the respective SLAM images are collected without overlapping, as shown in the related SLAM image table 118, the related pattern ID = F1, F2, F3, F4, F6, and F7. . When the processing of all pairs is completed (S293: YES), the related SLAM image extraction unit 72 advances the processing to S301 in FIG.

  As shown in FIG. 21, the related SLAM image extraction unit 72 extracts one related SLAM image from the related SLAM image table 118 (S301). The BOF similarity calculation unit 76 extracts the BOF of the extracted SLAM image from the BOF table 98 (S302), calculates the BOF similarity with the BOF of the autonomous BOF table 114 (S303), and autonomous BOF similarity It is set as list 120.

  On the other hand, the related landmark extraction unit 74 extracts a related pattern ID from the related SLAM image table 118 (S304), and determines whether or not a memory corresponding to the extracted related pattern ID is secured (S305). If the memory is secured (S305: YES), the related landmark extraction unit 74 advances the process to S308. When the memory corresponding to the extracted related pattern ID is not secured (S305: NO), the related landmark extracting unit 74 secures the memory (S306), and resets (initializes) the secured memory to zero. (S307), the process proceeds to S308.

  The disappearance belief calculation unit 78 refers to the autonomous BOF similarity list 120 and the BOF similarity comparison table 122. Then, the disappearance belief calculation unit 78 calculates the autonomous BOF similarity between the autonomous image and one of the SLAM images associated with the extracted pattern ID, and the existing value of the extracted pattern ID in the memory. The larger one is stored in the memory as the maximum similarity (S308). The extinction belief calculation unit 78 performs the same process for other SLAM images associated with the same pattern ID, and stores the maximum BOF similarity in the memory as the maximum similarity for one pattern ID. That is, the disappearance belief calculation unit 78 calculates the disappearance belief based on the larger BOF similarity between the related SLAM image and the autonomous image when one pattern ID is associated with a plurality of related SLAM images. Will do.

  The disappearance belief calculation unit 78 determines whether or not the maximum similarity calculation processing has been completed for all the related SLAM images (S309), and if not (S309: NO), returns to S301. Repeat the process.

  When completed (S309: YES), the disappearance belief calculation unit 78 refers to the autonomous BOF table 114, replaces the maximum similarity corresponding to the pattern ID observed in the autonomous image with zero, The degree of similarity after masking for each ID is calculated. For example, referring to the correspondence list 112, the pattern IDs = F1, F3, F4, and F7 correspond to the extraction regions G2, G4, G1, and G3, respectively, in the autonomous image. Therefore, the disappearance belief calculation unit 78 determines that the landmarks corresponding to the pattern IDs = F1, F3, F4, and F7 have been observed this time, and it is not necessary to increase the disappearance belief, so the post-mask similarity is zero. And the BOF similarity comparison table 122 is generated.

  The disappearance belief calculation unit 78 generates the pattern-specific score list 124 based on the post-mask similarity in the BOF similarity comparison table 122. Further, the disappearance belief calculation unit 78 sets the score in the pattern score list 124 as the disappearance belief for each pattern ID. At this time, if there is an annihilation belief degree that has been calculated and stored in the past for one pattern ID, the annihilation belief calculation unit 78 stores the stored annihilation belief degree and the calculated annihilation belief. The degree is added to obtain a new disappearance belief degree (S310). At this time, the disappearance belief stored in the storage device 17 is updated to a new disappearance belief.

  The autonomous movement control unit 82 determines whether or not the position of the robot 1 is the end point of autonomous movement (S311). If it is determined that the position is not the end point (S311: NO), the autonomous movement control unit 82 proceeds to S281 while continuing the movement. Return and repeat the process. When it is determined as the end point (S311: YES), the autonomous movement control unit 82 ends the movement process, ends the disappearance belief degree update process, and returns to the process of FIG.

  As described above, in this embodiment, the disappearance belief is introduced in addition to the landmark database 92, the pattern ID list 94, the BOF table 98, the landmark map 97, etc., which are products of the SLAM technology. The disappearance belief is a numerical standard for each landmark to determine whether each landmark has disappeared from the environment after the landmark map 97 is generated.

  The extinction beliefs are all set to zero immediately after the SLAM product 90 is generated, such as the landmark map 97. The annihilation belief in the landmark is updated depending on whether or not the landmark can be observed by the stereo camera 3 mounted on the robot 1 in the subsequent autonomous movement of the robot 1 during operation. At this time, although the BOF similarity between the autonomous image and the SLAM image is high and the similarity as the image is high, it is determined that there is a high possibility that the landmark that cannot be observed has disappeared. Then, a value obtained by integrating the similarities related to the unobservable landmarks is set as the disappearance belief degree.

  In the robot 1, the disappearance belief table 96 stored in the storage device 17 is scanned after the autonomous movement is completed, and a landmark having a disappearance belief higher than a predetermined threshold is deleted from the landmark database 92 and the landmark map 97. To do.

  In the above embodiment, the landmark ID is an example of landmark identification information, the SLAM image is an example of a prior image, and the SLAM image number is an example of prior image identification information. The BOF feature value is an example of a prior image feature detection frequency, and the SLAM unit 40 is an example of a prior information acquisition unit. The feature vector extraction unit 64 is an example of an image feature extraction unit, the autonomous BOF generation unit 66 is an example of a feature frequency calculation unit, the BOF similarity calculation unit 76 is an example of a similarity calculation unit, The mark deletion unit 100 is an example of a disappearance determination unit or an update unit.

  The landmark database 92, the pattern ID list 94, the disappearance belief table 96, the landmark map 97, and the BOF table 98 acquired at the time of advance movement are examples of advance information. The extraction area list 110, the correspondence list 112, the autonomous BOF table 114, the vector similarity table 116, and the related SLAM image table 118 are examples of autonomous information. The autonomous BOF similarity list 120, the BOF similarity comparison table 122, and the pattern-specific score list 124 are also examples of autonomous information.

  As described above, according to the present embodiment, the disappearance is based on the disappearance belief for each landmark based on the similarity between the autonomous image captured by the stereo camera 3 and the SLAM image acquired in advance. The landmark determined to be deleted from the SLAM product 90. Thereby, even if the environment changes, the landmark database 92, the landmark map 97, etc. can be updated according to the change. Since the update of the SLAM product 90 is performed according to the extinction belief calculated based on the image of the stereo camera 3, the accuracy can be improved as compared with the update based on the sensor. Thus, when the robot 1 autonomously moves using the landmark map 97, the landmark map 97 with higher accuracy can be updated using the image acquired by the stereo camera 3. Therefore, it is possible to stably maintain the accuracy of self-position estimation and autonomous movement of the robot 1.

  The present invention is not limited to the embodiments described above, and various configurations or embodiments can be adopted without departing from the gist of the present invention. For example, the data configuration illustrated in the above embodiment is an example, and the present invention is not limited to the above as long as the same processing can be performed. Further, the method for calculating the vector similarity and the BOF similarity is not limited to the above. For example, in the above embodiment, the calculation is based on the sum of the absolute values of the differences for each element. However, if the numerical value can represent the similarity, it is based on the square of the difference for each element. It is possible to modify the calculation. Moreover, the numerical value in each table | surface is an example, and is not limited to this.

  In the above embodiment, the example in which the advance movement and the autonomous movement are performed by the same robot 1 has been described. However, there is a modification in which the SLAM product 90 generated by another robot is stored in the robot 1 in advance and used. Is possible.

  Here, an example of a computer that is commonly applied to cause a computer to perform the operation of the prior information update method according to the above embodiment will be described. FIG. 22 is a block diagram illustrating an example of a hardware configuration of a standard computer. As shown in FIG. 22, a computer 400 includes a central processing unit (CPU) 402, a memory 404, an input device 406, an output device 408, an external storage device 412, a medium drive device 414, a network connection device, and the like via a bus 410. It is connected.

  The CPU 402 is an arithmetic processing unit that controls the operation of the entire computer 400. The memory 404 is a storage unit for storing a program for controlling the operation of the computer 400 in advance, or using it as a work area as needed when executing the program. The memory 404 is, for example, a random access memory (RAM), a read only memory (ROM), or the like. The input device 406 is a device that, when operated by a computer user, acquires various information inputs from the user associated with the operation content, and sends the acquired input information to the CPU 402. Keyboard device, mouse device, etc. The output device 408 is a device that outputs a processing result by the computer 400, and includes a display device and the like. For example, the display device displays text and images according to display data sent by the CPU 402.

  The external storage device 412 is, for example, a storage device such as a hard disk, and stores various control programs executed by the CPU 402, acquired data, and the like. The medium driving device 414 is a device for writing to and reading from the portable recording medium 416. The CPU 402 can perform various control processes by reading and executing a predetermined control program recorded on the portable recording medium 416 via the recording medium driving device 414. The portable recording medium 416 is, for example, a Compact Disc (CD) -ROM, a Digital Versatile Disc (DVD), a Universal Serial Bus (USB) memory, or the like. The network connection device 418 is an interface device that manages transmission / reception of various data performed between the outside by wired or wireless. A bus 410 is a communication path for connecting the above devices and the like to exchange data.

  A program that causes a computer to execute the prior information update method according to the above embodiment is stored in, for example, the external storage device 412. The CPU 402 reads a program from the external storage device 412 and causes the computer 400 to perform an image processing operation. At this time, first, a control program for causing the CPU 402 to perform image processing is created and stored in the external storage device 412. Then, a predetermined instruction is given from the input device 406 to the CPU 402 so that the control program is read from the external storage device 412 and executed. Further, this program may be stored in the portable recording medium 416.

Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)
A pre-information acquisition unit that acquires pre-information including landmark position information and image characteristics acquired from a pre-image obtained by photographing the surrounding environment of a route where autonomous movement is performed;
An autonomous information acquisition unit for acquiring autonomous information including position information and image features of a feature region related to the landmark acquired from an autonomous image obtained by capturing an image of the surrounding environment during autonomous movement of the autonomous mobile device;
Based on the prior information and the autonomous information, an extinction belief degree that calculates an extinction belief degree indicating the certainty of whether or not the landmark based on the prior image and the autonomous image has disappeared from the surrounding environment A calculation unit;
Based on the annihilation belief degree, it is determined whether or not the landmark is extinguished from the prior information, and an update unit that updates the prior information;
An image processing apparatus comprising:
(Appendix 2)
The prior information acquisition unit has observed landmark identification information of the landmark, a feature vector corresponding to the landmark, three-dimensional position information of the landmark, and an image feature corresponding to the feature vector. Obtaining pre-image identification information of the pre-image and a pre-image feature detection frequency for each image feature included in the pre-image,
The autonomous information acquisition unit
An autonomous image acquisition unit that acquires an autonomous image captured during autonomous movement of the autonomous mobile device;
An image feature extraction unit that extracts a feature region in the acquired autonomous image and extracts a feature vector of the extracted feature region;
Classifying the feature vectors extracted in the autonomous image to calculate an autonomous image feature detection frequency for each image feature related to the landmark in the autonomous image; and
Including
The disappearance belief calculation unit
A similarity calculation unit that calculates a similarity based on a difference between the prior image feature frequency and the autonomous image feature frequency for each image feature;
Calculating the extinction belief for each image feature based on the similarity for each image feature;
The image processing apparatus according to appendix 1, wherein:
(Appendix 3)
An association unit that associates the feature vector extracted in the autonomous image with the landmark;
Further comprising
The annihilation belief calculation unit calculates the annihilation belief based on the sum of the similarities for each of the image features corresponding to the landmarks not associated by the association unit,
The image processing apparatus according to appendix 2, wherein the disappearance determination unit determines that a landmark corresponding to the image feature is to be disappeared when the disappearance belief exceeds a predetermined value.
(Appendix 4)
The image processing according to appendix 2 or appendix 3, wherein the similarity is calculated based on a sum of absolute values of differences between the prior image feature frequency and the autonomous image feature frequency for each image feature apparatus.
(Appendix 5)
The similarity is calculated based on a maximum value of the similarity between each of the plurality of prior images from which the image features extracted from the autonomous image are extracted and the autonomous image. The image processing apparatus according to 2 or appendix 3.
(Appendix 6)
The image processing device
Obtain prior information including the landmark location information and image features acquired from the previous image of the surrounding environment of the route where the autonomous movement takes place,
The autonomous mobile device acquires autonomous information including position information and image features related to the landmark acquired from an autonomous image obtained by photographing the surrounding environment during autonomous movement,
Based on the prior information and the autonomous information, calculating an extinction belief representing the certainty of whether the landmark based on the prior image and the autonomous image has disappeared from the surrounding environment;
Based on the extinction belief degree, determine whether or not to extinguish the landmark from the prior information, and update the prior information.
The prior information update method characterized by the above.
(Appendix 7)
In the prior information acquisition process, the landmark identification information of the landmark, the feature vector corresponding to the landmark, the three-dimensional position information of the landmark, and the image feature corresponding to the feature vector are observed. The prior image identification information of the prior image and the prior image feature detection frequency for each image feature included in the prior image,
The process of acquiring the autonomous information includes
Acquire an autonomous image taken during autonomous movement of the autonomous mobile device,
Extracting a feature region in the acquired autonomous image, extracting a feature vector of the extracted feature region,
By classifying the feature vectors extracted in the autonomous image, the autonomous image feature detection frequency for each image feature related to the landmark in the autonomous image is calculated.
Including processing,
In the process of calculating the disappearance belief, a similarity is calculated based on a difference between the prior image feature frequency for each image feature and the autonomous image feature frequency, and based on the similarity for each image feature Calculating the disappearance belief for each image feature;
The prior information update method according to appendix 6, characterized in that:
(Appendix 8)
Further, the feature vector extracted in the autonomous image is associated with the landmark,
In the process of calculating the annihilation belief, the annihilation belief is calculated based on the sum of the similarities for each of the image features corresponding to the landmarks that are not associated by the association unit,
The prior information update method according to appendix 2, wherein the disappearance determination unit determines that the landmark corresponding to the image feature is to be disappeared when the disappearance belief exceeds a predetermined value.
(Appendix 9)
The prior information according to appendix 7 or appendix 8, wherein the similarity is calculated based on a sum of absolute values of differences between the prior image feature frequency for each image feature and the autonomous image feature frequency. Update method.
(Appendix 10)
The similarity is calculated based on a maximum value of the similarity between each of the plurality of prior images from which the image features extracted from the autonomous image are extracted and the autonomous image. The prior information update method according to 7 or appendix 8.
(Appendix 11)
Computer
Obtain prior information including the landmark location information and image features acquired from the previous image of the surrounding environment of the route where the autonomous movement takes place,
The autonomous mobile device acquires autonomous information including position information and image features related to the landmark acquired from an autonomous image obtained by photographing the surrounding environment during autonomous movement,
Based on the prior information and the autonomous information, calculating an extinction belief representing the certainty of whether the landmark based on the prior image and the autonomous image has disappeared from the surrounding environment;
Based on the extinction belief degree, determine whether or not to extinguish the landmark from the prior information, and update the prior information.
A program that causes a computer to execute processing.
(Appendix 12)
In the prior information acquisition process, the landmark identification information of the landmark, the feature vector corresponding to the landmark, the three-dimensional position information of the landmark, and the image feature corresponding to the feature vector are observed. The prior image identification information of the prior image and the prior image feature detection frequency for each image feature included in the prior image,
The process of acquiring the autonomous information includes
Acquire an autonomous image taken during autonomous movement of the autonomous mobile device,
Extracting a feature region in the acquired autonomous image, extracting a feature vector of the extracted feature region,
By classifying the feature vectors extracted in the autonomous image, the autonomous image feature detection frequency for each image feature related to the landmark in the autonomous image is calculated.
Including processing,
In the process of calculating the disappearance belief, a similarity is calculated based on a difference between the prior image feature frequency for each image feature and the autonomous image feature frequency, and based on the similarity for each image feature Calculating the disappearance belief for each image feature;
The program according to appendix 11, which is characterized by the above.
(Appendix 13)
Further, the feature vector extracted in the autonomous image is associated with the landmark,
In the process of calculating the annihilation belief, the annihilation belief is calculated based on the sum of the similarities for each of the image features corresponding to the landmarks that are not associated by the association unit,
13. The program according to appendix 12, wherein the disappearance determination unit determines that the landmark corresponding to the image feature is to be disappeared when the disappearance belief exceeds a predetermined value.
(Appendix 14)
The program according to appendix 12 or appendix 13, wherein the similarity is calculated based on a sum of absolute values of differences between the prior image feature frequency and the autonomous image feature frequency for each image feature.
(Appendix 15)
The similarity is calculated based on a maximum value of the similarity between each of the plurality of prior images from which the image features extracted from the autonomous image are extracted and the autonomous image. The program according to 12 or appendix 13.

DESCRIPTION OF SYMBOLS 1 Robot 3 Stereo camera 5 Wheel part 7 Sensor information acquisition apparatus 9 Image processing apparatus 11 Image acquisition apparatus 13 Information processing apparatus 15 Movement controller 17 Memory | storage device 30 Task manager 32 SLAM task control part 34 Autonomous movement task control part 36 Maintenance task control part 40 SLAM unit 42 SLAM image acquisition unit 44 Feature extraction unit 46 BOF calculation unit 48 Pattern registration unit 50 Landmark position measurement unit 52 Self-position estimation unit 54 Encoder data acquisition unit 56 Map generation unit 60 Autonomous movement unit 62 Autonomous image acquisition unit 64 Feature vector extraction unit 66 Autonomous BOF generation unit 68 Association unit 70 Corresponding list creation unit 72 Related SLAM image extraction unit 74 Related landmark extraction unit 76 BOF similarity calculation unit 78 Vanishing belief degree calculation unit 80 Vanishing belief degree update unit 8 Autonomous movement control unit 90 SLAM product 92 Landmark database 94 Pattern ID list 96 Vanishing belief degree table 97 Landmark map 98 BOF table 100 Landmark deletion unit 110 Extraction area list 112 Corresponding list 114 Autonomous BOF table 116 Vector similarity table 118 Related SLAM image table 120 Autonomous BOF similarity list 122 BOF similarity comparison table 124 Pattern-specific score list

Claims (7)

A pre-information acquisition unit that acquires pre-information including landmark position information and image characteristics acquired from a pre-image obtained by photographing the surrounding environment of a route where autonomous movement is performed;
An autonomous information acquisition unit for acquiring autonomous information including position information and image features of a feature region related to the landmark acquired from an autonomous image obtained by capturing an image of the surrounding environment during autonomous movement of the autonomous mobile device;
Based on the prior information and the autonomous information, an extinction belief degree that calculates an extinction belief degree indicating the certainty of whether or not the landmark based on the prior image and the autonomous image has disappeared from the surrounding environment A calculation unit;
Based on the annihilation belief degree, it is determined whether or not the landmark is extinguished from the prior information, and an update unit that updates the prior information;
An image processing apparatus comprising: The prior information acquisition unit has observed landmark identification information of the landmark, a feature vector corresponding to the landmark, three-dimensional position information of the landmark, and an image feature corresponding to the feature vector. Obtaining pre-image identification information of the pre-image and a pre-image feature detection frequency for each image feature included in the pre-image,
The autonomous information acquisition unit
An autonomous image acquisition unit that acquires an autonomous image captured during autonomous movement of the autonomous mobile device;
An image feature extraction unit that extracts a feature region in the acquired autonomous image and extracts a feature vector of the extracted feature region;
Classifying the feature vectors extracted in the autonomous image to calculate an autonomous image feature detection frequency for each image feature related to the landmark in the autonomous image; and
Including
The disappearance belief calculation unit
A similarity calculation unit that calculates a similarity based on a difference between the prior image feature frequency and the autonomous image feature frequency for each image feature;
Calculating the extinction belief for each image feature based on the similarity for each image feature;
The image processing apparatus according to claim 1. An association unit that associates the feature vector extracted in the autonomous image with the landmark;
Further comprising
The annihilation belief calculation unit calculates the annihilation belief based on the sum of the similarities for each of the image features corresponding to the landmarks not associated by the association unit,
The image processing apparatus according to claim 2, wherein the updating unit determines that the landmark corresponding to the image feature is to be deleted when the disappearance belief degree exceeds a predetermined value.   The said similarity is calculated based on the sum total of the absolute value of the difference between the said prior image feature frequency for every said image feature, and the said autonomous image feature frequency, The Claim 2 or Claim 3 characterized by the above-mentioned. Image processing device.   The similarity is calculated based on a maximum value of the similarity between each of the plurality of prior images from which the image features extracted from the autonomous image are extracted and the autonomous image. The image processing device according to claim 2. The image processing device
Obtain prior information including the landmark location information and image features acquired from the previous image of the surrounding environment of the route where the autonomous movement takes place,
Autonomous mobile device acquires autonomous information including positional information and image features of the feature region related to the landmark acquired from an autonomous image obtained by capturing the surrounding environment during autonomous movement,
Based on the prior information and the autonomous information, calculate a disappearance belief degree indicating the certainty of whether the landmark based on the prior image and the autonomous image has disappeared from the surrounding environment,
Based on the extinction belief degree, determine whether or not to extinguish the landmark from the prior information, and update the prior information.
The prior information update method characterized by the above. Computer
Obtain prior information including the landmark location information and image features acquired from the previous image of the surrounding environment of the route where the autonomous movement takes place,
Autonomous mobile device acquires autonomous information including positional information and image features of the feature region related to the landmark acquired from an autonomous image obtained by capturing the surrounding environment during autonomous movement,
Based on the prior information and the autonomous information, calculate a disappearance belief degree indicating the certainty of whether the landmark based on the prior image and the autonomous image has disappeared from the surrounding environment,
Based on the extinction belief degree, determine whether or not to extinguish the landmark from the prior information, and update the prior information.
A program that causes a computer to execute processing.

Images