JP2024062480A - Information processing device, mobile system, information processing method, and computer program - Google Patents

Abstract

[Problem] To provide an information processing device capable of highly accurately correcting route setting information of a moving body in accordance with updates to map information. [Solution] The information processing device comprises: a map information holding means for holding map information for calculating position and orientation information of a measuring device mounted on a moving body; a map information updating means for updating the map information; a route setting information holding means for holding route setting information including at least one of the position or orientation of the moving body on the route of the moving body in association with the map information; and a route setting information updating means for updating the route setting information so as to reduce a difference between a specific position or position and orientation in real space defined by the route setting information before the map information updating means updates the map information and the specific position or position and orientation defined by the route setting information after the map information updating means updates the map information. [Selected Figure] Figure 4

Description

The present invention relates to an information processing device, a mobile system, an information processing method, a computer program, etc. that uses map information.

There are known autonomous mobile systems that move based on measurement information, such as distance information and image information, measured by measuring devices such as cameras and LiDAR (Light Detection and Ranging) mounted on mobile objects such as mobile robots and automated guided vehicles (AGVs).

In Patent Document 1, a moving object is controlled to pass through waypoints consisting of two-dimensional coordinates and azimuth angles as route setting information for the moving object, based on the position and orientation of the measuring device calculated from the measurement information.

To calculate position and orientation based on measurement information from a camera or LiDAR, a two-dimensional occupancy grid map may be used, in which the presence or absence of walls or objects in a space is assigned to each cell as map elements. In addition, map information of the environment may be used, in which image features detected from image information and the three-dimensional position information of the image features calculated from these are used as map elements.

Since the positions of the cells contained in these map elements and the three-dimensional position information of the image features contain errors, in Patent Document 2, the specific positions measured by the measuring device are treated as nodes, and the relative positional relationships between these nodes are treated as edges, and the map information is corrected so that they are consistent.

JP 2019-36227 A JP 2019-124808 A

However, when correcting map information as implemented in Patent Document 2, the coordinates of map elements change from before the map information is corrected, which creates an issue of reduced accuracy in moving objects through intermediate points after the map information is corrected.

In view of the above, the present invention aims to provide an information processing device that can correct the route setting information of a moving object with high accuracy when map information is updated.

An information processing device according to one aspect of the present invention includes:
A map information storage means for storing map information for calculating position and orientation information of a measuring device mounted on a moving object;
A map information update means for updating the map information;
a route setting information holding means for holding route setting information including at least one of a position and an attitude of the moving body on a route of the moving body in association with the map information;
a route setting information updating means for updating the route setting information so that a difference between a specific position or a position and orientation in real space defined by the route setting information before the map information updating means updates the map information and the specific position or the position and orientation defined by the route setting information after the map information updating means updates the map information decreases;
a map information storage means for storing map information for calculating position and orientation information of a measuring device mounted on a moving body;
a map information update means for updating the map information;
a route setting information holding means for holding route setting information including at least one of a position and an attitude of the moving body on a route of the moving body in association with the map information;
a route setting information updating means for updating the route setting information so that a difference between a specific position or a position and orientation in real space defined by the route setting information before the map information updating means updates the map information and the specific position or the position and orientation defined by the route setting information after the map information updating means updates the map information decreases;
The present invention is characterized by having the following.

This invention makes it possible to realize an information processing device that can correct the route setting information of a moving object with high accuracy as map information is updated.

FIG. 2 is a diagram for explaining the concept of processing in the first embodiment. 1 is a functional block diagram showing a configuration of a moving object 100 including an information processing device 1 according to a first embodiment. FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing apparatus 1 according to a first embodiment. 4 is a flowchart illustrating an example of the operation of the information processing apparatus 1 according to the first embodiment. FIG. 13 is a diagram for explaining a second modified example of the first embodiment. FIG. 13 is a diagram for explaining a third modified example of the first embodiment. 10 is a flowchart showing an example of the operation of the device of the second embodiment.

Below, an embodiment of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiment. In each drawing, the same members or elements are given the same reference numbers, and duplicate descriptions are omitted or simplified.

<Embodiment 1>
In this embodiment, an information processing device 1 that performs SLAM (Simultaneous Localization and Mapping) will be described. That is, the information processing device 1 of this embodiment matches feature points obtained from an image input from a camera, which is a measuring device 101, with feature points having three-dimensional coordinates included in map information, and generates and corrects map information while calculating the position and orientation of the camera. The calculated position and orientation are used to control a moving object.

In addition, route setting information uses waypoints consisting of positions and orientations set in the coordinate system of map information. Waypoint navigation is implemented to control the moving object so that it follows multiple waypoints in order.

In this embodiment, the information on the waypoint consists of three parameters: two-dimensional position coordinates and a directional angle value. The movement of the moving body is controlled so that the distance and attitude difference between the waypoint of interest and the calculated position and attitude are reduced. The moving body moves by repeating the process of moving to the next waypoint when the distance and attitude difference become equal to or less than a predetermined distance.

The moving object may be, for example, an AMR (Autonomous Mobile Robot), an AGV (Autonomous Guided Vehicle), an autonomous vehicle, a cleaning robot, a drone, etc. In this embodiment, the case where the invention is applied to an AGV will be described.

In this embodiment, an example is described in which the information processing device 1 is mounted on a moving body, but part or all of the information processing device 1 can also be provided on a server connected via a network. In addition, waypoints are set using absolute coordinates such as GPS in autonomous driving, but are set on a map defined by SLAM indoors, etc.

Route setting information can generally use coordinates if absolute location information, such as GPS, is available. On the other hand, in environments where GPS cannot be used, such as indoors, route setting information is set using coordinates in map information created by SLAM.

In SLAM, map information is generated while calculating the position and orientation of the camera. As a result, errors in the calculation of the camera's position and orientation become errors in the map information, and vice versa, and so on.

Errors in the camera's position and orientation and in the map information tend to increase as the distance traveled increases. Therefore, multiple map elements that are located at the same point in real space but are determined to be in different positions in the map information are integrated, and the map information is updated so that the entire map information is consistent.

In this embodiment, map information stores as map elements the position and orientation of the camera determined from images captured by the camera when creating the map, key frames that use these images as elements, and three-dimensional position information of feature points. Updating map information in this embodiment means updating the key frames and the three-dimensional positions of feature points.

When map information is updated, the three-dimensional positions of individual map elements (key frames and feature points) change. As a result, even if the camera is located at the same point in real space, the calculated position and orientation of the camera change before and after the map update. For this reason, if a moving object is controlled using updated map information to head toward a waypoint that was set before the map information was updated, the object will end up moving to a different position in real space before and after the map update.

For example, in a newly constructed factory, there may be cases where a SLAM map is created, routes are set, and operations are started, after which additional equipment is installed. Of course, this is not limited to this example; there are also cases where a SLAM map is created for an existing factory, and then equipment is added or removed or the layout is changed later.

In such cases, feature points of the newly installed equipment will be detected in the images captured by the camera. In order to estimate the position and orientation with high accuracy using these feature points, SLAM performs map optimization. When the map is optimized (corrected) to include the new feature points, the position coordinates and orientation values of the map elements will change.

When the position coordinates or orientation values of map elements change, the camera's position and orientation values calculated after updating the map information change, even if the camera is located at the same point in real space. For this reason, when an AGV is driven after updating the map information, the AGV moves so that its coordinates match those in the map information before the update, and so in real space the AGV will be driving in a different position than it was before the map information was updated.

As another example, when creating map information while moving an AGV, the vehicle may pause midway to obtain the camera's position and orientation at that time, and these coordinates may be set as waypoints.

When new map elements are added to the map information as the map is created and the map information is updated, the coordinates of the map elements also change. Therefore, even if the AGV is stopped again after updating the map information at a location in real space where it had been temporarily stopped during the update, a different camera position and orientation will be calculated after the map information is updated than before the map information was updated.

For this reason, if the AGV is moved toward a waypoint that was set during map creation using map information after creation is complete (after the map information is updated), the AGV will move to a location different from the location where it stopped temporarily during map creation. Of the above use cases, this embodiment describes a method for correcting waypoints in a configuration in which waypoints are set while a map is being created.

In order to accurately control a moving object to the same point in real space before and after updating the map information, it is necessary to update the waypoints when the map information is updated. In this embodiment, of the above two cases, a configuration example is described in which the waypoints set while creating the map information are updated in accordance with the update of the map information. In addition, a method is described in which the waypoints are updated based on the update amount of the keyframe as the amount of movement of the map elements when the map information is updated.

Figure 1 is a diagram explaining the concept of processing in embodiment 1. Figure 1 shows an example of a plane traveled by a camera viewed from above as it moves clockwise along a rectangular route, with points where the camera paused during map creation set as waypoints. Note that in reality, a mobile object equipped with a camera is moving.

E100, E110, and E120 show the shape of the map information. Furthermore, a triangle as shown in E101 indicates a key frame, which is a map element. Marks as shown in E102 indicate waypoints (black circles indicate the positions of waypoints, and arrows indicate the direction of the waypoints).

In E100, the map shape has been distorted due to accumulated errors, and there is a discontinuity in the bottom right of the map. In E110, the same point has been detected in the bottom right of the route, and the map has been updated (this updating of map information is called a loop close; details will be described later).

Although the distortion has been reduced by updating the map information, there is a discrepancy in coordinates between the map elements and the route points because the waypoints are defined at coordinates before the map information was updated. E120 shows the result of correcting the route points to match the updated map shape in this embodiment, and the positions and orientations of the map elements and route points are consistent.

E105, E115, and E125 are enlarged views of partial areas E104, E114, and E124 of E100, E110, and E120, respectively. E106 shows the position and orientation of the key frame before updating the map information, and E107 shows the waypoint. The dotted line in E108 indicates the relative position and orientation between the waypoint and the key frame.

E110 shows how key frame E106 has moved to E116 as a result of updating the map information. As a result of this map update, the relative position and orientation between the key frame and the waypoint becomes as shown in E118, and the distance has increased compared to E108.

E120 is the result of correcting the waypoint position so that the difference between the relative position relationship E108 before map correction and the relative position relationship E118 after map correction is reduced. This relative position is represented by the relative position relationship E128. The waypoint is updated to satisfy this relative position relationship E128. (The updated waypoint becomes E127.)

FIG. 2 is a functional block diagram showing the configuration of a mobile body 100 including an information processing device 1 according to embodiment 1. Note that some of the functional blocks shown in FIG. 2 are realized by causing a CPU, which serves as a computer included in each of the information processing device 1 and the mobile body 100, to execute a computer program stored in a memory, which serves as a storage medium.

However, some or all of these functions may be implemented using hardware. For hardware, a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP) may be used. Furthermore, the functional blocks shown in FIG. 2 do not have to be built into the same housing, and may be configured as separate devices connected to each other via signal paths.

In this embodiment, a mobile body system is described that moves the mobile body, generates map information, registers waypoints, and controls the mobile body 100. Note that the measuring device 101 is a camera that acquires image information.

As described above, the information processing device 1 updates the map information stored in the map information storage unit 10. In addition, along with the update of the map information, the information processing device 1 updates the waypoints stored in the route setting information storage unit 12. Then, the position and orientation calculation unit 103 of the moving body 100 calculates the position and orientation using an image input from a camera, which is the measurement device 101, and the map information stored in the map information storage unit 10.

Using the calculated position and orientation and the waypoints updated by the route setting information storage unit 12, the control unit 14 calculates control values for controlling the moving body, and drives tires or the like to move the moving body via a motor or the like, which is an actuator 105.

The measurement information input unit 102 is connected to the camera, which is the measurement device 101, and inputs images captured by the camera in time series (e.g., 60 frames per second) and outputs them to the position and orientation calculation unit 103. Here, the measurement information input unit 102 functions as a measurement information input means that inputs the measurement information input by the measurement device. Note that the camera as the measurement device 101 is assumed to acquire monochrome images, for example, but may also acquire color images.

The position and orientation calculation unit 103 acquires map information from the map information storage unit 10, and matches feature points detected from the image input by the measurement information input unit 102 with feature points included in the map information. Based on the correspondence between the matched feature points, the position and orientation of the measurement device 101 is calculated. Here, the position and orientation calculation unit 103 functions as a position and orientation calculation means that calculates the position and orientation of the measurement device 101 using the measurement information.

The calculated position and orientation are output to the control unit 104. In addition, the position and orientation and feature points calculated by the position and orientation calculation unit 103 and the image input by the measurement information input unit 102 are output to the map information update unit 11 and the route setting information generation unit 106.

The route setting information generating unit 106 (route setting information generating means) inputs the position and orientation of the measuring device 101 calculated by the position and orientation calculating unit 103. In addition, the user inputs whether or not to register a waypoint from an input interface (e.g., a touch panel, etc.) (not shown).

When a signal to register a via point is input, the route setting information generation unit 106 generates coordinates of the via point as route setting information based on the position and orientation calculated by the position and orientation calculation unit 103, and stores the generated coordinates of the via point in the route setting information storage unit 12.

The map information holding unit 10 holds map information for the position and orientation calculation unit 103 to calculate the position and orientation of the measuring device 101. It also holds feature points and position and orientation information calculated by the position and orientation calculation unit 103. The map information holding unit 10 functions as a map information holding means that executes a map information holding step for holding map information for calculating the position and orientation information of the measuring device mounted on the moving body.

The map information update unit 11 updates the map information stored in the map information storage unit 10. In this embodiment, the map is expanded using the feature points and position and orientation information calculated by the position and orientation calculation unit 103. At this time, it is determined whether there is a point on the map that is the same as the expanded area, and if there is a point, the entire map is corrected so that it is consistent.

The map information updated in this manner is stored in the map information storage unit 10. The updated map information is also output to the route setting information update unit 13. The map information update unit 11 functions as a map information update means that executes a map information update step that updates the map information.

The route setting information storage unit 12 stores the measurement information measured by the measuring device 101 in association with the route setting information (waypoints). That is, the route setting information (waypoints) input by the route setting information generation unit 106 is stored in association with the measurement information.

The route setting information storage unit 12 also outputs the waypoints it stores to the route setting information update unit 13 and the control unit 14. The route setting information storage unit 12 functions as a route setting information storage means that executes a route setting information storage step of storing route setting information including at least one of the position and the attitude of the moving body on the route of the moving body in association with map information.

The route setting information update unit 13 inputs the updated map information input by the map information update unit 11 and the waypoints stored in the route setting information storage unit 12, and updates the coordinates and azimuth angles of the waypoints in accordance with the map update. It also outputs the updated route setting information to the route setting information storage unit 12.

That is, the route setting information update unit 13 calculates the position coordinates or attitude value of the measurement device 101 using the map information updated by the map information update unit 11 and the measurement information stored in association with the route setting information, and updates the route setting information.

The route setting information update unit 13 functions as a route setting information update means that executes a route setting information update step that updates the route setting information. The route setting information update unit 13 also updates the route setting information so that the difference between a specific position or position/orientation in real space that is specified by the route setting information before updating the map information and a specific position or position/orientation that is specified by the route setting information after updating the map information is reduced.

The control unit 104 inputs the position and orientation of the measuring device 101 calculated by the position and orientation calculation unit 103 and the waypoints stored in the route setting information storage unit 12, calculates the control values of the moving body, and drives the actuator 105. The control unit 104 also has a built-in CPU, which executes computer programs stored in a memory (not shown) to perform various calculations and control other functional blocks.

The components of the information processing device 1 according to this embodiment are a map information storage unit 10, a map information update unit 11, a route setting information storage unit 12, and a route setting information update unit 13. The measurement information input unit 102, the position and orientation calculation unit 103, the route setting information generation unit 106, and the control unit 104 are not essential components of this information processing device 1, but may be provided within this information processing device 1.

In this embodiment, map information refers to data that holds feature points detected from an image, position and orientation information of the camera that captured the image in which the feature points were detected (hereinafter also referred to as the position and orientation of the image), and three-dimensional position information of image features as key frames, which are map elements. A feature point consists of two-dimensional coordinates (u, v) on the image and a feature amount for uniquely identifying a small patch around those coordinates.

Feature points are detected as points that indicate locations in the image where there is a large change in the brightness gradient, such as edges and corners. Position and orientation information refers to six parameters: three parameters that indicate the camera's position in world coordinates defined in real space, and three parameters that indicate the camera's orientation.

The six rotation parameters can also be expressed as a 3x3 rotation matrix. Methods for mutual conversion of rotation expressions are well known, so a detailed explanation is omitted. The three-dimensional position information of a feature point refers to three-dimensional coordinates (X, Y, Z) on the world coordinate system.

In this embodiment, as described above, the feature points and the position and orientation of the image in which they are detected are stored as a key frame. The camera position and orientation are calculated by matching the feature points included in the key frame with the feature points detected from the image and minimizing the deviation in the geometric correspondence.

ORB (Oriented FAST and Rotated BRIEF) features, which hold local patterns as binary features, are used as image features. The position and orientation of the measuring device 101 are calculated by solving the Perspective-n-Point Problem (PNP) from the correspondence between the three-dimensional coordinates of the feature points in the matched keyframe and the two-dimensional coordinates of the feature points in the image.

The position and orientation calculation and key frames can be obtained using a method such as that described in Raul et al. (Raul Mur-Artal et. al, ORB-SLAM: A Versatile and Accurate Monocular SLAM System. IEEE Transactions on Robotics) (hereafter referred to as the Raul et al. method).

The current position and orientation in controlling a moving object refers to three parameters: two-dimensional coordinates that represent the two-dimensional position of the calculated three-dimensional position and orientation of the measuring device 101 projected onto the two-dimensional plane on which the moving object is moving, and an angle that represents the direction.

To calculate the control value, all possible variations of the control value that minimize the Euclidean distance and angle between the current position and orientation of the moving object and the via point are calculated as candidate control values. Then, the Dynamic Window Approach (DWA) is used to select the optimal control value from the candidates.

FIG. 3 is a diagram showing an example of the hardware configuration of the information processing device 1 of the first embodiment. 31 is a CPU that controls various devices connected to a system bus 39. 32 is a ROM that stores the BIOS program and the boot program. 33 is a RAM that is used as the main storage device of the CPU 31.

34 is an external memory that stores computer programs to be processed by the information processing device 1. The input unit 35 performs processing related to the input of information, etc. via a keyboard, mouse, touch panel, etc. The display unit 36 outputs the results of calculations by the information processing device 1 to a display device of an external PC terminal in accordance with instructions from the CPU 31. The display device may be of any type, such as a liquid crystal display device, a projector, or an LED indicator.

37 is a communication interface that communicates information via a network. The communication interface can be Ethernet, USB, serial communication, wireless communication, or the like, and any type of interface can be used. Calculated position and orientation information and route setting information are exchanged with the position and orientation calculation unit 103, control unit 104, and the like via the communication interface 17. 38 is an I/O that inputs images from the measurement device 101.

Figure 4 is a flowchart for explaining an example of the operation of the information processing device 1 of embodiment 1. The operations of each step of the flowchart in Figure 4 are performed by the CPU serving as a computer of the information processing device 1 and the mobile object 100 executing a computer program stored in memory. The process described in Figure 4 is automatically started when the information processing device 1 starts up as the mobile object 100 is powered on.

In this embodiment, a case will be described in which all steps performed by the mobile object 100 are executed within the information processing device 1, but steps S101 to S105, etc. may be executed by an external device to the information processing device 1, and the information processing device 1 may perform the processing from S106 onwards.

In step S101, the information processing device 1 initializes the system. That is, the information processing device 1 reads a program from the external memory 34 and makes the information processing device 1 operable. In addition, map information and camera parameters are read from the external memory 34 into the RAM 33 as necessary. Once a series of initialization processes is completed, the process proceeds to step S102.

In step S102, the measurement information input unit 102 inputs an image captured by the camera, which is the measurement device 101. The input image is output to the position and orientation calculation unit 103, and the process proceeds to step S103.

In step S103, the position and orientation calculation unit 103 performs feature point matching to determine where in the image information input by the measurement information input unit 102 the feature points contained in the map information held by the map information holding unit 10 are located, and finds the correspondence between the feature points. Then, based on the correspondence, the position and orientation of the camera is calculated by solving the PNP problem. The calculated position and orientation is output to the control unit 104 and the route setting information generation unit 106. Once a series of calculations and registrations have been completed, the process proceeds to step S104.

In step S104, the route setting information generating unit 106 determines whether or not there is a request from the user to register a new waypoint. If there is no request to register a waypoint, the process proceeds to step S106, and if there is a request, the process proceeds to step S105.

In step S105, the route setting information generation unit 106 generates via points (route setting information) and outputs them to the route setting information storage unit 12, where they are stored (registered). Specifically, the three-dimensional position and orientation of the measurement device 101 calculated by the position and orientation calculation unit 103 is projected onto a two-dimensional plane on which the moving body moves, to calculate two-dimensional positions X and Y. Furthermore, the angle θ with the X-axis is calculated, and these three parameters are set as the coordinates and azimuth angle of the via points.

Furthermore, among the key frames, which are map elements stored in the map information storage unit 10, the key frame closest to the position and orientation of the measuring device 101 (nearest key frame) is selected, and the three-dimensional position and orientation of that key frame is similarly projected onto the two-dimensional plane on which the moving object moves. Then, a key frame ID for identifying this nearest key frame and the two-dimensional relative position and orientation between that key frame and the generated way point are stored in the route setting information storage unit 12 together with the way point.

In step S106, the map information update unit 11 updates the map information stored in the map information storage unit 10. First, a key frame is generated using the camera position and orientation, feature points, and image information calculated by the position and orientation calculation unit 103, and is registered in the map information storage unit 10.

Next, a Bag of Features (BoF) is calculated to calculate the similarity between images based on the image information contained in the registered keyframes. Then, the similarity between the calculated BoF and the BoF calculated from the keyframe images contained in the map information is calculated.

If the similarity is equal to or greater than a predetermined level, it is determined that the newly added key frame is at the same location as the existing key frame, and the process proceeds to step S107. If a key frame at the same location, i.e. with a similar BoF, is not found from the map information, it is determined that the map information will not be updated, and the process proceeds to step S109.

In step S107, the map information update unit 11 performs a loop-closing process to correct the entire map information so that the positions of the key frames included in the map information held by the map information holding unit 10 and the newly registered key frames are consistent. As described in FIG. 1, the loop-closing process is a process of propagating the error between two map elements throughout the entire map so that map elements (key frames) at the same point in real space are also at the same point on the map.

Since the above-mentioned method by Raul et al. can be used for the loop closing process, detailed explanation will be omitted. The map information update unit 11 outputs the change value of the position and orientation of the key frame accompanying the loop closing process to the route setting information update unit 13 together with a key frame ID that can uniquely identify the key frame. After completing a series of map information update processes, proceed to step S108.

In step S108, the route setting information update unit 13 corrects the waypoints stored in the route setting information storage unit 12 based on the change value of the position or attitude of the key frame updated by the map information update unit 11. That is, in step S108, the route setting information (waypoints) is updated so that at least one of the difference in relative position between the map element before the map information update and the route setting information, or the difference in relative attitude between the attitude of the map element before the map information update and the attitude of the route setting information is reduced.

Specifically, the key frame is searched for in the map information based on the key frame ID registered in association with the way point when the way point was generated, and the amount of movement of the key frame associated with the map information update is added to the coordinates of the route point to correct it. The corrected way point is stored in the route setting information storage unit 12.

In step S109, the control unit 104 calculates a control value for controlling the moving body so as to reduce the difference between the position and orientation calculated by the position and orientation calculation unit 103 and the position or position and orientation of the via point (route setting information) held by the route setting information holding unit 12. For the control, a DWA is used as described above.

The actuator 105 is then driven based on the calculated control value to move the moving body. Here, step S109 functions as a control step (control means) that calculates a control value for controlling the moving body using the position and orientation and route setting information.

In step S110, it is determined whether or not to terminate the system. In this embodiment, the system is terminated when the moving body 100 arrives at the last waypoint, or when the user inputs a command to terminate the system via an input unit (not shown). If the answer is No in step S110, the process returns to step S102, and position and orientation calculation, map information update, waypoint update, and control of the moving body are continued.

As described above, in the first embodiment, the waypoint correction process is performed according to the amount of movement of the key frame accompanying the map information update. By doing this, the waypoints and map information will be consistent even if map correction occurs in the SLAM process.

This makes it possible to reduce any decrease in accuracy with which a moving object moves through waypoints after map information is corrected. In addition, by automatically correcting waypoints, it is possible to significantly reduce the effort required for users to correct waypoints one by one.

<Modification 1>
In the first embodiment, the correction (update) of the waypoint accompanying the update of the map information is calculated from the amount of movement of the nearest keyframe (change in coordinates accompanying the update of the map information). However, as long as the coordinate relationship between the updated map element and the waypoint is consistent, the amount of movement of the feature point may be used. In other words, the amount of change in the coordinate of the feature point may be added to the coordinate of the waypoint. Furthermore, if the position of an object is stored in the map information, the amount of movement of the object may be used.

For example, by using a deep learning model that has been trained to find objects and their positions from images held in keyframes, the position of the object after the map is updated can be determined based on the position and orientation of the keyframes accompanying the map correction. In this case, if the relative positions of the object and the waypoints are stored in the map information beforehand, the waypoints can be corrected so that the relative position of the object on the updated map matches that before the update.

Note that the number of map elements associated with one waypoint is not limited to one, and may be multiple. In other words, multiple nearby keyframes may be selected, and the position and orientation of the waypoint may be calculated so that the weighted sum of the difference between the relative positions of those multiple keyframes and the waypoint and the relative positions of the multiple keyframes and the waypoint after the map information has been updated is minimized.

If the update pattern for waypoints maintains the relative positions with respect to multiple keyframes, multiple pieces of information can be used even if the positional accuracy of a particular keyframe deteriorates due to updates to map information. This reduces the impact of deterioration in the positional accuracy of keyframes, and further improves the accuracy of the position and orientation of waypoints.

It is also possible to change key frames to feature points and correct the waypoints so that the relative positions between multiple feature points and waypoints are maintained. In that case, even if feature points with low accuracy are linked to waypoints, outliers can be detected, for example, by majority vote, further improving the accuracy of the position and orientation of the waypoints.

In addition, the map element used as an index for updating the route setting information may not be fixed, but may be selected according to the situation. For example, the closest of the nearest key frame or the nearest feature point may be selected. In this way, the way point is updated according to the map element that is closer to the way point, making it possible to update the way point with higher accuracy.

In the first embodiment, the measurement device 101 is described as a monochrome camera, but as long as it can acquire images, it may be a color camera, a monocular camera, a stereo camera, a depth camera, etc.

In addition, in the first embodiment, the measurement device 101 is a camera, but any measurement device capable of implementing SLAM can be used, and no matter what kind of measurement device is used, the waypoints can be corrected (updated) based on the relative positional relationship between the SLAM map elements and the waypoints.

For example, 2D-LiDAR (Light Detection and Ranging) or 3D-LiDAR can be used. When correcting a 2D map using 2D-LiDAR, the surrounding 2D features measured at a certain time and the position and orientation at that time are used as nodes to perform loop closing processing.

This 2D-LiDAR loop closure process can be performed using a method such as that described in Wolfgang et al.'s paper (Wolfgang Hess et.al. Real-Time Loop Closure in 2D LIDAR SLAM (ICRA 2016)), and a detailed explanation will be omitted.

The waypoints can be corrected so that their relative positions to the nodes and two-dimensional features are maintained before and after map correction. When using 3D LiDAR, processing can be performed in the same way as 2D LiDAR if the surrounding three-dimensional features measured at a certain time and the position and orientation at that time are used as nodes. In this way, this embodiment can be applied regardless of the type of measurement device.

In SLAM using LiDAR, this embodiment can also be applied when shape features are used instead of feature points in an image. As such shape features, SHOT features (Signature of Histograms of Orientations) using a histogram of the product of no normal vectors around a certain point can be calculated. Alternatively, PFH features (Point Feature Histograms) using a histogram of the arrangement of points around a point of interest can be calculated.

<Modification 2>
In the first embodiment, the waypoints are corrected in accordance with the amount of movement of the keyframe accompanying the update of the map information. However, the present invention can also be applied to a configuration in which the measurement information of the measurement device 101 at the waypoints is stored, instead of storing the relative positions between the keyframes and the waypoints. In that case, if the position and orientation of the waypoints are calculated using the measurement information at the waypoints when the map information is updated, the coordinate relationship between the updated map and the waypoints will be consistent, allowing for highly accurate correction of the waypoints.

Figure 5 is a diagram for explaining variant example 2 of embodiment 1, and the specific configuration concept when a 2D-LiDAR that measures the circumferential distance from a sensor is used as the measurement device 101 is explained using Figure 5.
E200, E210, and E220 show the shapes of the map information. A 2D occupancy grid map is used as the map information, and the boundaries with objects are shown by solid lines (E201).

The waypoints are indicated by arrows E202, which show their positions and azimuth angles. In E200, the map shape is distorted due to accumulated errors. In E210, the same point is detected at the bottom right of the route, and the map is corrected. For the correction here, loop-closing processing is performed using, for example, the method of Wolfgang et al.

Although the distortion has been reduced by map correction, the waypoints are defined at coordinates before the map correction, and there is a discrepancy in coordinates between the map elements and the route points. E220 shows the result of correcting the route points to match the corrected map. The correction has reduced the discrepancy in coordinates between the map elements and the route points.

E205, E215, and E225 are enlarged views of E204, E214, and E224, which are partial areas of E200, E210, and E220, respectively. E206 shows the waypoint before map correction. E208 is a depth value measured by LiDAR, and its shape matches the occupancy grid E207, which represents the wall surface.

E210 shows how the occupancy grid E207, which represents a wall surface, has moved to E217 as a result of map correction. This map correction has resulted in an area E218 where the depth value measured by the LiDAR at the waypoint does not reach the wall surface, and an area E219 where the depth value passes through the wall surface. E220 shows the result (E226) of correcting the coordinates of the waypoint so that the depth value measured by the LiDAR at the waypoint matches the occupancy grid E217.

This type of correction can be performed, for example, using the ICP (Iterative Closest Point) algorithm used to calculate the position and orientation of points and points and surfaces. Alternatively, a method such as NDT (Normal Distribution Transform), which divides point cloud data into a grid and matches the normal distribution of the point cloud of each grid, can be used. This allows the waypoints and map elements to match, making it possible to control the moving object with greater precision.

When a camera is used for the measuring device 101, if image information captured by the camera at the waypoint and feature points detected from the image information are stored, the waypoint can be corrected by calculating the map information and the position and orientation of the waypoint, just as in the case of using LiDAR.

In addition, when updating route setting information, a method of correcting the way points described above, such as the relative position and orientation with respect to measurement information and map elements, may be selected according to the situation. For example, if there is a landscape on the map that matches the image attached to the route setting information, the way points may be updated using that and the map information, and if not, the way points may be updated from the amount of movement associated with updating the map information of the nearest key frame. In this way, even if the way points cannot be updated using one indicator, they can be updated using other indicators, increasing the stability of the way point updates.

<Modification 3>
In the first embodiment, the relative positions and orientations between the route setting information and the key frames are stored, and the waypoints are corrected according to the amount of movement of the key frames accompanying the update of the map information. However, it is sufficient if the amount of change in the map elements before and after the map correction can be known, even if the relative positions and orientations are not stored.

For example, a conversion table that spatially represents the amount of change in map elements may be used. That is, a conversion table of at least one of the position coordinates or the attitude values before and after a map update created based on the amount of update of the map elements accompanying the map information update may be held. Furthermore, the map information update unit 11 may update the route setting information based on the conversion table.

First, the space is divided into a grid, and the centroid coordinates and the relative position with the nearest map element are calculated. The ID and relative position of the nearest map element used at this time are stored for each grid. Next, after the map is corrected, the centroid coordinates of each grid are calculated again, and the relative value with the map element having the stored ID.

The coordinates of each grid and the amount of change in the relative position between map elements before and after map correction are stored in each cell. To update the coordinates of a waypoint, the nearest grid is selected from the coordinates of the waypoint, and the difference in relative position stored in that grid is added to the waypoint. By using this method, route setting information can be updated even if route setting information for an arbitrary point is input.

In addition, instead of selecting the nearest lattice point to the intermediate point, a weighted sum of the values of multiple lattices can be used. In that case, even if the intermediate point is set on the lattice boundary, the calculation can be performed with continuous values, so the route setting information can be updated with higher accuracy.

Figure 6 is a diagram for explaining variant 3 of embodiment 1, and shows the concept of a conversion table for updating route setting information in variant 2. E300 represents a conversion table in which the space containing map information is divided into a grid. E301 is the smallest unit cell of the conversion table.

The shape of the map information before the update is shown by the dotted line E311, and the shape of the map information after the update is shown by the solid line E312. E313 represents the change in the position and orientation of each cell as a result of propagating the change in the map shape (the change in the relative position between each grid coordinate and the map elements before and after the map correction described above) to each cell. Such cell change amounts E313 are stored.

In addition, it is not limited to a conversion table; as long as the spatial coordinate changes of map elements accompanying map information updates are known, the conversion formula may be expressed as a polynomial, for example. In that case, the number of parameters is reduced compared to a conversion table, making it possible to update waypoints with less memory.

<Modification 4>
In the first embodiment, the route setting information is a waypoint consisting of three-dimensional parameters indicating a position and a direction, but any geometric information that determines the position and direction of the route of a moving object may be used. For example, a pattern that uses waypoints consisting of six-dimensional parameters that indicate a position and a posture in space can be applied to a moving object that moves in the air or underwater, such as a drone.

In addition, if the control points for determining the spline curve are used as the route setting information, it will be possible to configure the system to control the moving object more smoothly. If a cost map in which the distance to an obstacle is used as the route setting information, it will be possible to apply the system to a configuration in which the moving object is controlled while maintaining a constant distance from the obstacle.

In this embodiment, the control uses the DWA method, but other control value patterns may be used. For example, control may be performed using a graph search approach, which is a method of calculating the variation of the control value at multiple future times, calculating the position and orientation (predicted trajectory) at multiple times, and selecting the control value that follows the predicted trajectory.

Alternatively, a certain area free of obstacles may be randomly sampled from within two-dimensional space from the current position, and the area around the sampled area may be randomly sampled using the sampled area as a node. In this case, a randomized approach may be used, for example.

When using these control methods, if the route setting information is updated, the control value can be recalculated, allowing map updates to be performed while moving. In this way, by applying this embodiment, it becomes possible to update route setting information with high accuracy even when map information is updated using any control method.

In the first embodiment, map information updates were performed using loop-close processing, but the method of this embodiment can be used for updates that change the position or orientation of map elements.

Map element updates can also be configured to correct route setting information in conjunction with the implementation of bundle adjustment, which optimizes key frames and feature points so that they match in three dimensions. Using bundle adjustment makes it possible to update maps with higher accuracy, and to update waypoints with higher accuracy as well.

As described in the first embodiment, the map information update unit 11 can update the map information at any timing, not just when a key frame, which is a map element, is added. For example, the map information may be updated at any timing by performing bundle adjustment or pose graph optimization so as to reduce the residual of the map stored in the map information storage unit 10.

In addition, the route setting information update unit 13 updates the route information in conjunction with the update of the map information. In this way, the route setting information can be updated even in a configuration in which map information and route setting are created on a small computer installed in a mobile body, and then the data is copied to a large computer such as a workstation to update the map.

In this embodiment, the information processing device 1 includes at least a map information storage unit 10, a map information update unit 11, a route setting information storage unit 12, and a route setting information update unit 13. However, for example, the configuration of the information processing device 1 may be provided in an external server connected to the mobile body via a network, and the map information and route setting information may be input to the information processing device 1 via the network.

In such a case, the information processing device 1 can perform the processes related to updating the map information and the route setting information on a separate computer from the mobile body, and the mobile body can use a small-resource computer that only performs the processes related to position and attitude calculation and control.

Also, as described in the first embodiment, the measurement information input unit 102 and the position and orientation calculation unit 103 may be configured to be included in the information processing device 1. In that case, even if the map information is updated using the position and orientation and feature point information input by the position and orientation calculation unit 103 when the arrangement of an object changes, for example, the route setting information can be updated so as to be consistent with the map information.

Furthermore, the information processing device 1 may be configured to include the measurement information input unit 102, the position and orientation calculation unit 103, and the route setting information generation unit 106. In that case, for example, when a route is set at the same time as creating a map, the route setting information can be updated so as to be consistent with the map information even if the map information is updated.

The route setting information generating unit 106 may also generate route setting information (waypoints) by associating the position and orientation calculated by the position and orientation calculating unit 103 with at least one of the map elements or the measurement values output by the measuring device 101, and store the information in the route setting information storage unit 12.

As such, without implementing this embodiment, it was necessary to create a map once, set the map not to be updated, and then set the route. However, with this embodiment, it is possible to set the route at the same time as creating the map, reducing the effort required for map creation and route setting, and enabling route setting information to be generated with high accuracy.

It is also possible to configure the system to store map information before and after the update. That is, when route setting information is input, the system may first link the map elements before the update with the route setting information, and then update the route setting information so that the relative positions and attitudes of the corresponding map elements after the map information is updated are consistent with the route setting information.

With such a configuration, it becomes possible to update route setting information generated with the coordinates of the map information before the update after updating the map information, even if the route setting information is not set before updating the map information. For example, if a display UI for route setting is created using a map that has already been created, it becomes possible to convert the newly set route setting information on that UI to the coordinates after the map is updated.

In addition, in the first embodiment, the route setting information is updated in conjunction with the update of the map information, but the update may be delayed as necessary. For example, the route setting information may be updated only when the control unit 104 uses the route setting information. In this way, the update process for the route setting information is performed only when the route setting information becomes necessary, so that even a computer with small computational resources can perform processing according to the load.

In addition, in the first embodiment, a configuration for updating route setting information has been described, but if the route setting information can be corrected in accordance with the map update, the information processing device 1 may be configured to output the update amount for updating the route setting information without updating the route setting information.

With this configuration, when an information processing device 1 implemented in a server receives a request from each AGV via the network to update the route setting information, the information processing device 1 can return only the amount of update to the route setting information via the network.

In the configuration of the first embodiment, for example, if the map elements and route setting information shown in FIG. 1 or FIG. 4 are displayed on the display unit 36, the user can easily associate and grasp the update of the map information with the route setting information. In addition, the user can select and modify the route setting information updated in this embodiment by using the input unit 35 including a keyboard, mouse, etc.

In addition, in the first embodiment, for example, map information may be created before the mobile object is brought in, and the map information and route setting information may be updated after the mobile object is brought in, or route setting information may be generated while creating the map information, and the route setting information may be updated in conjunction with the update of the map information.

The present embodiment can also be easily applied to other use cases that require updating route setting information in conjunction with updating map information. For example, it can be applied to a use case in which map information is updated as needed in a logistics warehouse where goods are brought in and out. Alternatively, this embodiment can be applied to updating map information, such as when updating a map according to appearance in an environment where the appearance of an image changes due to changes in the lighting environment, such as morning, noon, or night.

<Embodiment 2>
In the first embodiment, the route setting information is updated along with the update of the map information. In the second embodiment, a method applied to a case where new map information is generated by combining multiple pieces of map information will be described as one type of map information update.

That is, the route setting information set for each piece of map information is updated (corrected) so that it is consistent with the integrated map information. That is, in the second embodiment, multiple pieces of map information are combined, and the route setting information defined in the coordinate system of each of the multiple maps is updated to route setting information in the coordinate system of the combined map.

In this embodiment, we will explain a method for integrating the routes of camera-equipped AGVs that previously operated different routes using two different small maps, and enabling operations that allow them to travel between the two routes. Note that in this embodiment, as in the explanation of embodiment 1, the route setting information is assumed to be waypoints.

The configuration of the information processing device 1 in this embodiment is the same as the configuration of the information processing device 1 described in embodiment 1, so a description thereof will be omitted. What differs from embodiment 1 is that the map information storage unit 10 stores multiple pieces of map information, and the map information update unit 11 integrates the multiple pieces of map information.

The route setting information storage unit 12 also stores waypoints linked to each piece of map information stored in the map information storage unit 10, and the route setting information update unit 13 updates the coordinates of the waypoints to match the integrated map information.

Figure 7 is a flowchart showing an example of the operation of the device of embodiment 2. The operation of each step in the flowchart in Figure 7 is performed by the CPU, which serves as the computer of the information processing device 1, executing a computer program stored in memory. The input maps and the coordinates of the waypoints associated with each are updated. The process described in Figure 7 begins when an instruction is given to start the map synthesis process. Note that steps with the same reference numbers as those in Figure 4 perform the same operation, and therefore will not be described.

In step S201, one piece of map information is input from a map input unit (not shown) and registered in the map information storage unit 10. In addition, a map ID is issued to distinguish between multiple pieces of input map information, and is linked to the map information and stored. Once the map information has been registered, the process proceeds to step S202.

In step S202, the waypoints linked to the map information input in step S201 are input from the route setting information generation unit 106 and registered (stored) in the route setting information storage unit 12. The route setting information is assigned a map ID, a key frame ID for identifying the nearest key frame, and a relative position with respect to the nearest key frame. Once this series of processes is completed, the process proceeds to step S203.

In step S203, if all map elements to be integrated have been input, proceed to step S204; if not, proceed to S201 and input another map.

In step S204, the map information update unit 11 integrates the maps registered in the map information storage unit 10. Specifically, it searches for similar key frames between the maps using the BoF detected from the key frames included in the map information, calculates a rigid body transformation that makes their positional relationships consistent, and converts the map information so that it is expressed in a common coordinate system. Then, it performs pose graph optimization to correct local distortions in the two maps.

The rigid body transformation and pose graph optimization involved in integrating these maps can be performed using the method described in, for example, Tong et al.'s VINS-Mono: A Robust and Versatile Monocular Visual-Internal State Estimator, IEEE ToR 2018. Therefore, a detailed explanation will be omitted.

The integrated map information is stored in the map information storage unit 10, and the process proceeds to step S205. In step S205, the route setting information update unit 13 updates the coordinates of the transit points stored in the route setting information storage unit 12.

Specifically, the integrated value of the relative position and orientation assigned to the via point is registered as a new via point coordinate for the map ID assigned to the via point and the position and orientation of the key frame having the key frame ID. This is performed for all via points. The updated via points are stored in the route setting information storage unit 12, and the process ends.

As described above, in the second embodiment, when multiple pieces of map information are integrated, each way point before the integration is updated to match the position and orientation of the associated key frame. In this way, it is possible to reduce the decrease in accuracy with which a moving object moves through way points after the map information is integrated. Furthermore, when the map information is integrated, the way points can be updated with high accuracy without the user's effort.

<Modification 1>
In the second embodiment, rigid body transformation and pose graph optimization are used to correct the map information, but any method for integrating maps may be used, and other methods may also be used.

For example, if only rigid body transformation is performed, it can be executed in a short time, in which case the waypoints can be updated just by accumulating the rigid body transformation parameters to them. Also, bundle adjustment can be performed after rigid body transformation and pose graph optimization. This improves the accuracy of the map information and the route setting information.

As described in the modified example of the first embodiment, the camera serving as the measuring device 101 of the second embodiment may be a monochrome camera, a color camera, a monocular camera, a stereo camera, a depth camera, or the like. In addition, any sensor capable of identifying the position may be used, and therefore 2D-LiDAR, 3D-LiDAR, or the like may also be used.

The information processing device 1 of embodiment 2 can be mounted on a moving body and configured to receive map information from another moving body (such as an AGV). In this way, an AGV can be automatically operated without hassle along a route that another AGV has been operating. It also leads to a reduction in the work time required for map creation and route setting, such as setting routes while creating maps with multiple AGVs.

In the above embodiment, an example of applying the present invention to an autonomous moving body has been described. However, the moving body in this embodiment is not limited to an autonomous moving body such as an AGV (Autonomous Guided Vehicle) or an AMR (Autonomous Mobile Robot). In addition, it may be used as a driving assistance body even if it does not move completely autonomously.

The moving object may be any moving device that moves, such as an automobile, train, ship, airplane, robot, or drone. The information processing device 1 of the embodiment may or may not be partially mounted on the moving object. The present invention may also be applied to cases where a moving object is remotely controlled.

The present invention has been described in detail above based on a preferred embodiment, but the present invention is not limited to the above embodiment, and various modifications are possible based on the spirit of the present invention, and are not excluded from the scope of the present invention. This embodiment includes the following combinations.

(Configuration 1) An information processing device comprising: a map information holding means for holding map information for calculating position and orientation information of a measuring device mounted on a moving body; a map information update means for updating the map information; a route setting information holding means for holding route setting information including at least one of the position or orientation of the moving body on the route of the moving body in association with the map information; and a route setting information update means for updating the route setting information so as to reduce a difference between a specific position or position and orientation in real space defined by the route setting information before the map information update means updates the map information and the specific position or position and orientation defined by the route setting information after the map information update means updates the map information.

(Configuration 2) The information processing device according to Configuration 1, characterized in that the route setting information update means updates the route setting information so as to reduce at least one of the difference in relative position between the map element before the update of the map information and the route setting information, or the difference in relative attitude between the attitude of the map element before the update of the map information and the attitude of the route setting information.

(Configuration 3) The information processing device according to configuration 1 or 2, characterized in that the route setting information storage means stores the measurement information measured by the measurement device in association with the route setting information, and the route setting information update means calculates the position coordinates or attitude value of the measurement device using the map information updated by the map information update means and the measurement information stored in association with the route setting information, and updates the route setting information.

(Configuration 4) The information processing device according to any one of configurations 1 to 3, characterized in that the map information update means holds a conversion table of at least one of position coordinates or attitude values before and after a map update, which is created based on the update amount of map elements accompanying the map information update, and the map information update means updates the route setting information based on the conversion table.

(Configuration 5) The information processing device according to any one of configurations 1 to 4, wherein the map information update means combines a plurality of pieces of map information, and further updates the route setting information defined in the coordinate system of each of the plurality of maps to the route setting information in the coordinate system of the combined map.

(Configuration 6) Further, a route setting information generating means for generating the route setting information is provided,
6. The information processing apparatus according to any one of configurations 1 to 5, wherein the route setting information storage means stores the route setting information generated by the route setting information generation means.

(Configuration 7) Further, a measurement information input means for inputting measurement information input by the measurement device;
a position and orientation calculation means for calculating the position and orientation of the measuring apparatus using the measurement information;
7. The information processing device according to any one of configurations 1 to 6, comprising:

(Configuration 8) The information processing device according to Configuration 6, characterized in that the route setting information generating means generates the route setting information by associating the position and orientation calculated by the position and orientation calculating means with at least one of map elements or measurement values output by the measuring device, and stores the route setting information in the route setting information storing means.

(Configuration 9) The information processing device according to any one of configurations 1 to 8, further comprising a control means for calculating a control value for controlling the moving body using the position and orientation and the route setting information.

(Method) An information processing method comprising: a map information holding step for holding map information for calculating position and orientation information of a measuring device mounted on a moving body; a map information updating step for updating the map information; a route setting information holding step for holding route setting information, which is geometric information for determining at least one of the position or orientation of the moving body on the route of the moving body, in association with the map information; and a route setting information updating step for updating the route setting information so as to reduce a difference between a specific position or position and orientation in real space defined by the route setting information before updating the map information in the map information updating step and the specific position or position and orientation defined by the route setting information after updating the map information in the map information updating step.

(Program) A computer program for controlling the information processing device described in any one of the configurations or each means of the mobile system described in configuration 9 by a computer.

In order to realize some or all of the control in this embodiment, a computer program that realizes the functions of the above-mentioned embodiment may be supplied to an information processing device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, etc.) in the information processing device or the like may read and execute the program. In this case, the program and the storage medium on which the program is stored constitute the present invention.

1: Information processing device 10: Map information storage unit 11: Map information update unit 12: Route setting information storage unit 13: Route setting information update unit,
100: moving body 101: measuring device 102: measurement information input unit 103: position and orientation calculation unit 104: control unit 105: actuator 106: route setting information generation unit,

Claims (11)

A map information storage means for storing map information for calculating position and orientation information of a measuring device mounted on a moving object;
a map information update means for updating the map information;
a route setting information holding means for holding route setting information including at least one of a position and an attitude of the moving body on a route of the moving body in association with the map information;
a route setting information updating means for updating the route setting information so that a difference between a specific position or a position and orientation in real space defined by the route setting information before the map information updating means updates the map information and the specific position or the position and orientation defined by the route setting information after the map information updating means updates the map information decreases;
13. An information processing device comprising: The information processing device according to claim 1, characterized in that the route setting information update means updates the route setting information so as to reduce at least one of the difference in relative position between the map element before the update of the map information and the route setting information, or the difference in relative attitude between the attitude of the map element before the update of the map information and the attitude of the route setting information. the route setting information storage means stores measurement information measured by the measurement device in association with the route setting information;
2. The information processing device according to claim 1, wherein the route setting information update means calculates a position coordinate or an attitude value of the measurement device by using the map information updated by the map information update means and the measurement information stored in association with the route setting information, and updates the route setting information. the map information update means holds a conversion table of at least one of position coordinates and attitude values before and after a map update, the conversion table being created based on an update amount of a map element accompanying the map information update;
2. The information processing apparatus according to claim 1, wherein said map information update means updates said route setting information based on said conversion table. The information processing device according to claim 1, characterized in that the map information update means combines a plurality of the map information, and further updates the route setting information defined in the coordinate system of each of the plurality of maps to the route setting information in the coordinate system of the combined map. The system further includes a route setting information generating means for generating the route setting information,
2. The information processing apparatus according to claim 1, wherein said route setting information holding means holds said route setting information generated by said route setting information generating means. Further, a measurement information input means for inputting measurement information input by the measurement device;
a position and orientation calculation means for calculating the position and orientation of the measuring apparatus using the measurement information;
2. The information processing apparatus according to claim 1, further comprising: The information processing device according to claim 6, characterized in that the route setting information generating means generates the route setting information by associating the position and orientation calculated by the position and orientation calculating means with at least one of map elements or measurement values output by the measuring device, and stores the route setting information in the route setting information storing means. The mobile body system according to claim 1, further comprising a control means for calculating a control value for controlling the mobile body using the position and orientation and the route setting information. A map information storage step for storing map information for calculating position and orientation information of a measuring device mounted on a moving object;
a map information updating step of updating the map information;
a route setting information storage step of storing route setting information, which is geometric information for determining at least one of a position and an attitude of the moving body on a route of the moving body, in association with the map information;
a route setting information updating step of updating the route setting information so that a difference between a specific position or position and orientation in real space defined by the route setting information before updating the map information in the map information updating step and the specific position or position and orientation defined by the route setting information after updating the map information in the map information updating step decreases;
13. An information processing method comprising: A computer program for controlling the information processing device according to any one of claims 1 to 8 or each means of the mobile body system according to claim 9 by a computer.

Images