WO2022190662A1 - 情報処理装置、および情報処理方法、並びにプログラム - Google Patents

情報処理装置、および情報処理方法、並びにプログラム Download PDF

Info

Publication number
WO2022190662A1
WO2022190662A1 PCT/JP2022/001992 JP2022001992W WO2022190662A1 WO 2022190662 A1 WO2022190662 A1 WO 2022190662A1 JP 2022001992 W JP2022001992 W JP 2022001992W WO 2022190662 A1 WO2022190662 A1 WO 2022190662A1
Authority
WO
WIPO (PCT)
Prior art keywords
map
information
node
elevator
analysis unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/001992
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
佑允 高橋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Priority to JP2023505177A priority Critical patent/JPWO2022190662A1/ja
Publication of WO2022190662A1 publication Critical patent/WO2022190662A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions

Definitions

  • the present disclosure relates to an information processing device, an information processing method, and a program. More specifically, the present invention relates to an information processing apparatus, an information processing method, and a program for creating a map that can be used for traveling of a moving body such as a robot.
  • map information and destination information to grasp the surrounding conditions.
  • map information that enables them to select a safe travel route without colliding with obstacles is necessary.
  • a map on which the positions of obstacles and the like can be confirmed is called an environmental map, for example.
  • SLAM Simultaneous localization and mapping
  • SLAM is known as a representative process for creating an environment map and estimating a self-location.
  • SLAM is a process of executing self-position estimation processing (localization) and environment map creation processing (mapping) in parallel using, for example, information acquired by a sensor such as a camera, such as a photographed image of the robot's running environment.
  • a map is created (mapping), which is a so-called environmental map that enables grasping of object positions and the like.
  • robot position is also estimated (localized).
  • SLAM processing analyzes the object positions at various positions where the robot that creates the map has moved, and for example, by integrating the object positions at each position, creates an environment map corresponding to each floor, which is a map for each floor. can be done.
  • SLAM is a process that executes self-position estimation processing (localization) and environment map creation processing (mapping) in parallel using information acquired by sensors such as cameras.
  • SLAM is not limited to visual SLAM that uses camera-captured images as described above, but there are various other methods.
  • LiDAR SLAM that uses LiDAR (Light Detection and Ranging), which is a sensor that measures the distance to an obstacle by laser light.
  • a process of analyzing the positions of surrounding objects and the like from images captured by cameras at various positions (x, y, z) is performed.
  • the environment map on the first floor and the environment map on the second floor must be created as separate and independent maps.
  • the environmental maps for each floor can be created individually, but the plurality of created environmental maps do not record data indicating the relationship with other environmental maps. Therefore, even if an attempt is made to move a robot on the first floor to a destination on the second floor using environment maps created for each floor, for example, a remote operation by an operator or the like is required to move the robot from the first floor to the second floor. etc. is necessary.
  • Patent Document 1 Japanese Unexamined Patent Application Publication No. 2012-215959 discloses a configuration in which a robot travels a plurality of floors and travels.
  • This patent document 1 discloses a security robot that can move and run on a plurality of floors.
  • the configuration described in this document is a configuration in which the robot is moved according to one security route preset by the user, and cannot be applied to a configuration in which the starting point or destination is changed.
  • JP 2012-215959 A International publication WO2020/085135
  • the present disclosure has been made, for example, in view of the above-described problems. It is an object of the present invention to provide an information processing device, an information processing method, and a program for creating a map and map structure information that enable the vehicle to travel to the ground.
  • map structure information that records the relationship between each map when creating a plurality of independent environmental maps, such as floor-by-floor environmental maps.
  • a first aspect of the present disclosure includes: a map generation unit that generates a map using sensor acquisition information from a sensor attached to a mobile object; a map structure analysis unit that generates map structure information indicating relationships between the plurality of maps generated by the map generation unit; The map structure analysis unit When the map generating unit finishes generating the first map and generates a new second map, The node position of the first map transition node that is the map creation end node of the first map, the node position of the second map transition node that is the map creation start node of the second map, and the node position of these two map transition nodes as map transition node information in a map structure information storage unit.
  • a second aspect of the present disclosure is An information processing method executed in an information processing device, a map generation step in which the map generation unit generates a map using sensor acquisition information from a sensor attached to the mobile object; a map structure information recording step in which the map structure analysis unit generates map structure information indicating the relationship between the plurality of maps generated by the map generation unit and records the map structure information in the map structure information storage unit; The map structure analysis unit, in the map structure information recording step, When the map generating unit finishes generating the first map and generates a new second map, The node position of the first map transition node that is the map creation end node of the first map, the node position of the second map transition node that is the map creation start node of the second map, and the node position of these two map transition nodes as map transition node information in a map structure information storage unit.
  • a third aspect of the present disclosure is A program for executing information processing in an information processing device, a map generation step of causing a map generation unit to generate a map using sensor acquisition information from a sensor attached to a mobile object; causing a map structure analysis unit to execute a map structure information recording step of generating map structure information indicating relationships between a plurality of maps generated by the map generation unit and recording the map structure information in a map structure information storage unit;
  • the program in the map structure information recording step that is executed by the map structure analysis unit, When the map generating unit finishes generating the first map and generates a new second map, The node position of the first map transition node that is the map creation end node of the first map, the node position of the second map transition node that is the map creation start node of the second map, and the node position of these two map transition nodes as map transition node information in the map structure information storage unit.
  • the program of the present disclosure is, for example, a program that can be provided in a computer-readable format to an information processing device or computer system capable of executing various program codes via a storage medium or communication medium.
  • processing according to the program is realized on the information processing device or computer system.
  • a system is a logical collective configuration of a plurality of devices, and the devices of each configuration are not limited to being in the same housing.
  • a map generation unit that generates a map using sensor-obtained information from a sensor attached to a moving body, and map structure information that indicates the relationship between a plurality of maps generated by the map generation unit is generated. It has a map structure analysis part.
  • the map structure analysis unit determines the node position of the first map transition node, which is the map generation end node of the first map, and the first map transition node.
  • the node position of the second map transition node which is the map creation start node of the second map, is recorded in the map structure information storage unit as map transition node information.
  • the map generating unit is instructed to store the first map in the map information storage unit and to start generating the second map.
  • FIG. 5 is a diagram illustrating an example of processing for creating an environment map by a robot; It is a figure explaining an example of the data stored in the map information memory
  • FIG. 4 is a diagram for explaining data stored in a map information storage unit within a storage unit of a robot by map creation processing for two floors; It is a figure explaining the problem of map creation processing accompanying elevator movement.
  • FIG. 4 is a diagram describing an outline of processing of the present disclosure
  • FIG. 4 is a diagram describing an outline of processing of the present disclosure
  • FIG. 4 is a diagram describing an outline of processing of the present disclosure
  • FIG. 4 is a diagram describing an outline of processing of the present disclosure
  • FIG. 4 is a diagram describing an outline of processing of the present disclosure
  • FIG. 4 is a diagram describing an outline of processing of the present disclosure
  • FIG. 4 is a diagram illustrating data stored in a storage unit of a robot (information processing device) through processing according to the present disclosure
  • 4 is a diagram illustrating a correspondence relationship between data stored in a map structure information storage unit 40 and data stored in a map information storage unit 30.
  • FIG. FIG. 3 is a diagram illustrating an overview of processing of the first embodiment of the information processing apparatus of the present disclosure
  • FIG. 3 is a diagram illustrating an overview of processing of the first embodiment of the information processing apparatus of the present disclosure
  • 1 is a diagram illustrating a configuration example of an information processing apparatus according to Example 1 of the present disclosure
  • FIG. 4 is a diagram illustrating data stored in a map information storage unit of the information processing apparatus according to Example 1 of the present disclosure
  • FIG. 4 is a diagram illustrating data stored in a map structure information storage unit of the information processing apparatus according to the first embodiment of the present disclosure
  • FIG. 4 is a diagram showing a flowchart describing a processing sequence executed by the information processing apparatus according to the first embodiment of the present disclosure
  • FIG. 4 is a diagram showing a flowchart describing a processing sequence executed by the information processing apparatus according to the first embodiment of the present disclosure
  • FIG. 3 is a sequence diagram describing a processing sequence executed by the information processing apparatus according to the first embodiment of the present disclosure
  • FIG. 3 is a sequence diagram describing a processing sequence executed by the information processing apparatus according to the first embodiment of the present disclosure
  • FIG. 3 is a sequence diagram describing a processing sequence executed by the information processing apparatus according to the first embodiment of the present disclosure
  • FIG. 10 is a diagram illustrating an outline of processing of the second embodiment of the information processing apparatus of the present disclosure
  • FIG. 10 is a diagram illustrating an outline of processing of the second embodiment of the information processing apparatus of the present disclosure
  • FIG. 10 is a diagram illustrating an outline of processing of the second embodiment of the information processing apparatus of the present disclosure
  • FIG. 7 is a diagram illustrating a configuration example of an information processing apparatus according to a second embodiment of the present disclosure
  • FIG. 10 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus according to a second embodiment of the present disclosure
  • FIG. 10 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus according to a second embodiment of the present disclosure
  • FIG. 12 is a diagram illustrating an overview of processing of the third embodiment of the information processing apparatus of the present disclosure
  • FIG. 12 is a diagram illustrating an overview of processing of the third embodiment of the information processing apparatus of the present disclosure
  • FIG. 11 is a diagram illustrating a configuration example of an information processing apparatus according to a third embodiment of the present disclosure
  • FIG. 11 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus according to Example 3 of the present disclosure
  • FIG. 12 is a diagram illustrating an overview of processing of the fourth embodiment of the information processing apparatus of the present disclosure
  • FIG. 12 is a diagram illustrating a configuration example of an information processing apparatus according to a fourth embodiment of the present disclosure
  • FIG. 11 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus according to Example 4 of the present disclosure
  • FIG. 11 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus according to Example 4 of the present disclosure
  • FIG. 12 is a diagram illustrating an outline of processing of the fifth embodiment of the information processing apparatus of the present disclosure
  • FIG. 13 is a diagram illustrating a configuration example of an information processing apparatus according to Example 5 of the present disclosure
  • FIG. 11 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus according to Example 5 of the present disclosure
  • FIG. 11 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus according to Example 5 of the present disclosure
  • FIG. 12 is a diagram illustrating an outline of processing of the sixth embodiment of the information processing apparatus of the present disclosure
  • FIG. 13 is a diagram illustrating data stored in a map structure information storage unit of an information processing apparatus according to Example 6 of the present disclosure
  • FIG. 13 is a diagram illustrating data stored in a map structure information storage unit of an information processing apparatus according to Example 6 of the present disclosure
  • FIG. 11 is a diagram illustrating a configuration example of an information processing apparatus according to a sixth embodiment of the present disclosure
  • FIG. 13 is a diagram illustrating data stored in a map structure information storage unit of an information processing apparatus according to Example 6 of the present disclosure
  • FIG. 11 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus according to Example 6 of the present disclosure
  • FIG. 11 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus according to Example 6 of the present disclosure
  • FIG. 12 is a diagram illustrating an outline of processing of the seventh embodiment of the information processing apparatus of the present disclosure
  • FIG. 12 is a diagram illustrating an outline of processing of the seventh embodiment of the information processing apparatus of the present disclosure
  • FIG. 12 is a diagram illustrating an outline of processing of the seventh embodiment of the information processing apparatus of the present disclosure
  • FIG. 12 is a diagram illustrating an outline of processing of the seventh embodiment of the information processing apparatus of the present disclosure
  • FIG. 12 is a diagram illustrating an outline of processing of the seventh embodiment of the information processing apparatus of the present disclosure
  • FIG. 11 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus
  • FIG. 12 is a diagram illustrating a configuration example of an information processing apparatus according to a seventh embodiment of the present disclosure
  • FIG. 14 is a diagram illustrating data stored in a map structure information storage unit of an information processing apparatus according to Example 7 of the present disclosure
  • FIG. 14 is a diagram illustrating data stored in a map structure information storage unit of an information processing apparatus according to Example 7 of the present disclosure
  • FIG. 14 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus according to Example 7 of the present disclosure
  • FIG. 14 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus according to Example 7 of the present disclosure
  • FIG. 14 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus according to Example 7 of the present disclosure
  • Example 14 is a sequence diagram illustrating a processing sequence executed by an information processing apparatus according to Example 7 of the present disclosure; It is a figure explaining the example of composition of the information processing system of this indication. It is a figure explaining the hardware structural example of the information processing apparatus of this indication.
  • Embodiment 2 Embodiment in which a plurality of maps and map structure information are generated by analyzing an image captured by a camera to detect elevator movement 3-3.
  • Embodiment 3 Embodiment in which elevator movement is detected using user input information and multiple maps and map structure information are generated 3-4.
  • Embodiment 4 An embodiment in which a plurality of maps and map structure information are generated by analyzing communication data with a communication unit installed in an elevator boarding/alighting unit to detect elevator movement.
  • Embodiment 5 Regarding an embodiment of returning to the floor where the map has already been created and updating the created map.
  • Embodiment 6 Regarding an embodiment in which the generated map is switched according to the inclination angle of the robot running surface6.
  • Map information is required.
  • a map on which the positions of obstacles and the like can be confirmed is called an environmental map, for example.
  • FIG. 1 is a diagram showing an execution example of SLAM (simultaneous localization and mapping) processing, which is representative processing for creating an environment map and estimating a self-location.
  • SLAM simultaneous localization and mapping
  • the robot 10 shown in FIG. 1 runs on the first floor (1F) 11, which is the planned floor for map creation.
  • the robot 10 is equipped with sensors such as a camera and LiDAR (Light Detection and Ranging), and moves on the first floor (1F) 11 while acquiring sensor detection information.
  • sensors such as a camera and LiDAR (Light Detection and Ranging), and moves on the first floor (1F) 11 while acquiring sensor detection information.
  • FIG. 1 shows the processing when the robot 10 sequentially moves a plurality of nodes 12a to 12d on the first floor, that is, nodes 1A to 1B to nodes 1C to 1D (Ev_1f) to create a map.
  • the robot 10 records, for example, the three-dimensional positions (x, y, z) of the nodes 1A and 12a and the yaw angle (Yaw) indicating the direction of the robot as node information 13a of the nodes 1A and 12a in a storage unit inside the robot. do. Furthermore, sensor acquisition information acquired by the nodes 1A and 12a, that is, surrounding object position information acquired by sensors such as cameras and LiDARs is recorded.
  • sensors such as LiDAR detect the distance to objects around the robot, and by setting point groups according to this distance, it is possible to construct a map consisting of point groups according to the shape of the object.
  • the robot 10 acquires node information (x, y, z, yaw) consisting of the positions of the nodes and the direction of the robot on the nodes for each of a plurality of nodes (1A to 1D) on the travel route. and stores it in the storage unit, and obtains surrounding point group information (object position information) for each node and stores it in the storage unit.
  • node information x, y, z, yaw
  • FIG. 2 is a diagram explaining the data stored in the map information storage unit 15 in the storage unit of the robot by the map creation process shown in FIG.
  • the map information storage unit 15 stores map correspondence data 15p and 1F map actual data 15q.
  • (P1) 1F map correspondence data is recorded in the map correspondence data 15p.
  • (P11) Source frame 1F map (P12) 1F map node information Consists of these data.
  • the source frame is acquisition destination information of the map data (actual map data), and in this case, the 1F map.
  • 1F map node information is a recording area for the node information described with reference to FIG. That is, node information (x, y, z, yaw) consisting of the position (x, y, z) of each node and the orientation (yaw) of the robot is recorded as target frame data.
  • this 1F map node information (x, y, z, yaw) is data according to a coordinate system unique to the 1F map.
  • the 1F map coordinate system is, for example, a coordinate system set with the generation start point of the 1F map as the origin.
  • each target frame of the (P12) 1F map node information of the map correspondence data 15p corresponds to the position (x, y, z) on the 1F map actual data 15q and the orientation (yaw) of the robot. do.
  • the point group information at each node of the 1F map actual data 15q that is, the position of objects such as obstacles Information can be obtained.
  • the robot 10 that creates the environment map stores the map-corresponding data consisting of the position (x, y, z) of the node to which the robot has moved and the orientation (yaw) of the robot in the map information storage unit 15, and the map-corresponding data.
  • Actual map data composed of point cloud information (object position information) that can be acquired based on the recorded data (target frame) is recorded.
  • the robot 10 or other robots refer to the map correspondence data 15p and the 1F actual map data 15q to autonomously run on the first floor (1F) 11 shown in FIG. 1 without colliding with obstacles. becomes possible.
  • a point cloud (PC) corresponding to the object position generated based on the object distance information, which is the LiDAR detection information, is set.
  • the object distance information which is the LiDAR detection information
  • the environment map is not limited to such point cloud data, and may be a map using various representation formats such as voxel representation and a primitive shape group using object detection information.
  • SLAM processing analyzes the object positions around the movement route of the robot that creates the map, and integrates them to create a floor-by-floor map. environment map can be created.
  • a process of analyzing the positions of surrounding objects and the like from images captured by cameras at various positions (x, y, z) is performed.
  • FIG. 3 shows the first floor 11 and the second floor 21 . Each floor can be moved using an elevator.
  • the rightmost node 1D (EV_1f) on the first floor 11 shown in FIG. 3 corresponds to the elevator position on the first floor.
  • the rightmost node 2A (EV_2f) on the second floor 21 shown in FIG. 3 corresponds to the elevator position on the second floor.
  • the robot 10 operates on the first floor 11, as described above with reference to FIGS.
  • a map is created by sequentially moving nodes 1D (Ev_1f).
  • the robot 10 After moving to the second floor (2F) 21 using the elevator, the robot 10 starts moving from the elevator position corresponding node 2A (EV_2F) on the second floor (2F) 21 and moves to the second floor (2F). 21 start the environmental map creation process.
  • the robot 10 runs on the second floor (2F) 21, which is the planned floor for map creation.
  • the robot 10 sequentially moves a plurality of nodes on the second floor, that is, node 2A (EV_2F) through node 2B through node 2C through node 2D to create a map.
  • the robot 10 acquires node information (x, y, z, yaw) for a plurality of nodes (2A to 2D) on the travel route and stores it in the storage unit, and also acquires point groups around each node.
  • Information object position information is acquired and stored in the storage unit.
  • FIG. 4 is a diagram explaining data stored in the map information storage unit 15 in the storage unit of the robot by the map creation process for the two floors shown in FIG.
  • the map information storage unit 15 stores map-corresponding data 15p, 1F actual map data 15q1, and 2F actual map data 15q2 in the same manner as described above with reference to FIG. .
  • the map-corresponding data 15p includes: (P1) 1F map corresponding data (P2) 2F map corresponding data, These are recorded separately.
  • (P11) source frame and (P21) source frame are acquisition destination information of map data (actual map data), and in this case, they are the 1F map and 2F map, respectively.
  • (P12) 1F map node information and (P22) 2F map node information are recording areas for the node information described with reference to FIG. That is, node information (x, y, z, yaw) consisting of the position (x, y, z) of each node and the orientation (yaw) of the robot is recorded as target frame data.
  • (P12) 1F map node information (x, y, z, yaw) is data according to the coordinate system unique to the 1F map.
  • the 1F map coordinate system is, for example, a coordinate system set with the generation start point of the 1F map as the origin.
  • (P22) 2F map node information (x, y, z, yaw) is data according to the coordinate system unique to the 2F map.
  • the 2F map coordinate system is, for example, a coordinate system set with the starting point of generation of the 2F map as the origin.
  • the actual map data is configured as 1F actual map data 15q1 and 2F actual map data 15q2, that is, two separate environment map data.
  • the map information storage unit 15 does not record any relationship between these two environmental maps. As described above, when the robot 10 moves in an elevator, it is impossible to analyze the direction and amount of movement of the elevator from information acquired by sensors such as cameras and LiDAR. Therefore, self-position estimation and environment map creation processing cannot be continued, and environment map creation and self-position estimation processing as SLAM processing are interrupted. environmental maps become separate and independent maps, and their relationships cannot be recorded at all.
  • the environment map can be created individually for each floor, but the created environment maps do not have data indicating the relationship with other environment maps. Therefore, even if an attempt is made to move a robot on the first floor to a destination on the second floor by using the created environment maps for each floor, the information for moving from the first floor to the second floor cannot be obtained from the map information. can't get As a result, there arises a problem that, for example, remote operation by an external operator or the like is required.
  • the robot 10 uses the map correspondence data 15q, the 1F actual map data 15q1, and the 2F actual map data 15q2, which are created by the robot 10 and stored in the map information storage unit 15, to determine whether the robot 10 is on the first floor.
  • the following unknown points occur.
  • remote operation by an operator or the like is required.
  • the present disclosure has been made, for example, in view of the above-described problems. Allows it to run to the ground.
  • map structure information that records the relationship between each map is recorded.
  • the information processing apparatus of the present disclosure is installed in a robot that performs processing for creating an environment map by, for example, SLAM processing.
  • it may be configured to be installed in a server or the like that can communicate with the robot.
  • the robot uses an information processing device having a data processing unit and a storage unit that perform environment map creation processing, map structure information generation processing, and storage processing of these generated data in a storage unit. It is an example of processing in the case of a configuration having
  • FIGS. 6 to 10 show an outline of the processing sequence when the robot 10 creates a plurality of individual environment maps. Specifically, first, create an environment map (1F map) of the first floor (1F), then move to the second floor (2F) by elevator, and then move to the second floor (2F) environment map (2F map) map) is shown. Each processing of the processing steps (S01) to (S06) shown in FIGS. 6 to 10 will be described in order.
  • Step S01 First, in step S01, the robot 10 starts creating a map of the first floor. As described above with reference to FIG. 1, a map is created while setting a plurality of nodes at the movement positions on the first floor.
  • step S02 the robot 10 ends map creation of the first floor. As previously described with reference to FIG. 1, robot 10 finishes mapping the ground floor when it reaches the node corresponding to the elevator location on the ground floor.
  • Step S03 Next, in step S03 shown in FIG. 7, the robot 10 stores the map of the first floor (1F map) in the map information storage unit.
  • the robot 10 stores 1F map corresponding data 30p1 and 1F map actual data 30q1 in the map information storage unit 30 within the robot.
  • the 1F map correspondence data 30p1 is composed of the following data, as shown in FIG. (1)
  • Source frame information 1F map (2) 1F map node information (3) 1F map origin position information (x, y) (4) 1F map resolution information
  • Source frame information is acquisition destination information of map data (actual map data), and in this case, it is the 1F map.
  • 1F map node information is a recording area for node information similar to that described with reference to FIG. That is, node information (x, y, z, yaw) consisting of the position (x, y, z) of each node and the orientation (yaw) of the robot is recorded as target frame data.
  • 1F map origin position information (x, y,) is a coordinate position indicating the origin position of the 1F map in an external coordinate system, for example, a coordinate system such as world coordinates. Although only the two-dimensional coordinate position (x, y) is shown in this example, the three-dimensional coordinate position (x, y, z) may be recorded.
  • 1F map resolution information is information indicating the resolution of the 1F map data 30q1.
  • the map information storage unit 30 stores the actual 1F map data generated by the robot 10 in the 1F actual map data 30q1.
  • the 1F map actual data 30q1 is the same data as the data described above with reference to FIGS. That is, for example, map data (environmental map) representing objects around the robot travel route as point group information (object position information) is stored.
  • map data environment map
  • the environment map is not limited to point cloud data, and may be a map using various representation formats such as voxel representation and primitive shape group using object detection information.
  • Step S04 Next, in step S04 shown in FIG. 8, the robot 10 uses an elevator to move from the first floor (1F) to the second floor (2F).
  • the map structure information storage unit 40 provided in the information processing device in the robot stores the relational information between the two maps, that is, the generated 1F map and the newly generated map. Record the information related to the 2F map.
  • map structure information storage unit 40 records the following information.
  • Map transition node attributes (2) Map transition node information (3) Movement amount data between map transition nodes
  • Map transition node attribute is a field for recording attributes related to nodes before and after switching the environment map.
  • the map transition nodes are two nodes, one for the first floor (1F) elevator position and the other for the second floor (2F) elevator position.
  • Map transition node information records the node information of two map transition nodes, the node at the elevator position on the first floor (1F) and the node at the elevator position on the second floor (2F).
  • This node information is similar to the map correspondence data 15p in the map information storage unit 15p described above with reference to FIG. That is, for each of the two map transition nodes, a source frame (1F map or 2F map), which is the acquisition destination information of the map data (actual map data); Node information (x, y, z, yaw) consisting of the position (x, y, z) of each node and the orientation (yaw) of the robot Record these data.
  • Target frame before map transition EV_1F: (x, y, z, yaw)
  • This data is node information about the node of the elevator position on the first floor (1F), and is data according to the coordinate system unique to the 1F map.
  • the 1F map coordinate system is, for example, a coordinate system set with the generation start point of the 1F map as the origin.
  • Target frame after map transition EV_2F: (x, y, z, yaw)
  • This data is node information about the node of the elevator position on the second floor (2F), and is data according to the coordinate system specific to the 2F map.
  • the 2F map coordinate system is, for example, a coordinate system set with the starting point of generation of the 2F map as the origin.
  • the map structure information storage unit 40 records "(3) movement amount data between map transition nodes". Note that the (3) map transition node-to-map movement amount data is data that is optionally recorded, not essential recording data.
  • the movement amount data between map transition nodes is the movement amount between two map transition nodes. That is, the movement amount data between the nodes of the two map transition nodes, the node of the elevator position on the first floor (1F) and the node of the elevator position on the second floor (2F), is recorded.
  • the amount of movement is recorded for each element of (x, y, z, yaw), for example. i.e. Movement amount in x direction (rel_x) Movement amount in y direction (rel_y) Movement amount in z direction (rel_z) Movement amount of yaw angle (rel_yaw) These element-by-element movement amounts are recorded.
  • movement between two map transition nodes is movement only in the vertical direction (z direction).
  • (3) movement amount data between map transition nodes is not limited to recording as movement amount data (rel_x, rel_y, rel_z, rel_yaw) for each element as described above. It may be configured to record the floor.
  • Movement amount data between map transition nodes "1st floor (1F) -> 2nd floor (2F)”
  • Movement amount data between map transition nodes "2nd floor (2F) -> 5th floor (5F)”
  • Map structure information is generated and recorded as information indicating the relationship. That is, as shown in FIG. 8, the following information is recorded in the map structure information storage unit 40.
  • Map transition node attributes (2) Map transition node information (3) Movement amount data between map transition nodes
  • Step S05 the robot 10 starts creating a new environment map, that is, the environment map of the second floor (2F), runs on the second floor (2F),
  • step S06 shown in FIG. 10 the creation of the environmental map of the second floor (2F) ends.
  • map information storage unit 30 of the robot 10 stores map correspondence data 30p2 and 2F map actual data 30q2 as shown in FIG.
  • the map correspondence data 30p2 generated by the process of creating the environment map of the second floor (2F) is composed of the following data, as shown in FIG. (1)
  • Source frame information 2F map (2) 2F map node information (3) 2F map origin position information (x, y) (4) 2F map resolution information
  • Source frame information is acquisition destination information of map data (actual map data), and in this case, it is the 2F map.
  • 2F map node information is a recording area for node information similar to that described with reference to FIG. That is, node information (x, y, z, yaw) consisting of the position (x, y, z) of each node and the orientation (yaw) of the robot is recorded as target frame data.
  • 2F map origin position information (x, y,) is a coordinate position indicating the origin position of the 2F map in an external coordinate system, for example, a coordinate system such as world coordinates. Although only the two-dimensional coordinate position (x, y) is shown in this example, the three-dimensional coordinate position (x, y, z) may be recorded.
  • 2F map resolution information is information indicating the resolution of the 2F map data 30q2.
  • the map information storage unit 30 stores the 2F map data generated by the robot 10 in the 2F map actual data 30q2 in addition to the 2F map corresponding data 30p2.
  • the 2F map actual data 30q2 is the same data as the data described above with reference to FIGS. That is, for example, map data (environmental map) representing objects around the robot travel route as point group information (object position information) is stored.
  • map data environment map
  • the environment map is not limited to point cloud data, and may be a map using various representation formats such as voxel representation and primitive shape group using object detection information.
  • map correspondence data 30p, 1F actual map data 30q1, and 2F actual map data 30q2 are recorded.
  • the map correspondence data 30p includes the following data. (P1) 1F map corresponding data (P2) 2F map corresponding data, These are recorded separately.
  • (P1) 1F map correspondence data is correspondence data for the 1F map actual data 30q1 previously described with reference to FIG. 7, and is composed of the following data.
  • (P11) Source frame information 1F map (P12) 1F map node information (P13) 1F map origin position information (x, y) (P14) 1F map resolution information
  • (P11) source frame information is acquisition destination information of map data (actual map data), and is the 1F map.
  • (P12) 1F map node information is a recording area for node information similar to that described with reference to FIG. That is, node information (x, y, z, yaw) consisting of the position (x, y, z) of each node and the orientation (yaw) of the robot is recorded as target frame data.
  • the (P2) 2F map corresponding data is corresponding data for the 2F map actual data 30q2 described above with reference to FIG. 10, and is composed of the following data.
  • Source frame information 2F map (P22) 2F map node information (P23) 2F map origin position information (x, y) (P24) 2F map resolution information
  • Source frame information is acquisition destination information of map data (actual map data), and in this case, it is the 2F map.
  • node information (x, y, z, yaw) consisting of the position (x, y, z) of each node and the orientation (yaw) of the robot is recorded as target frame data.
  • (P12) 1F map node information (x, y, z, yaw) is data according to the coordinate system unique to the 1F map.
  • the 1F map coordinate system is, for example, a coordinate system set with the generation start point of the 1F map as the origin.
  • (P22) 2F map node information (x, y, z, yaw) is data according to the coordinate system unique to the 2F map.
  • the 2F map coordinate system is, for example, a coordinate system set with the starting point of generation of the 2F map as the origin.
  • the 1F actual map data 30q1 and the 2F actual map data 30q2 are individual environment map data, and as described above, are configured by map data, such as point cloud data, that can identify object positions on each floor.
  • map structure information 40 is recorded in the storage unit of the robot 10 .
  • the map structure information 40 records the following information. (1) Map transition node attributes (2) Map transition node information (3) Movement amount data between map transition nodes
  • Map transition node attribute Elevator
  • Map transition node information The following data are recorded as map transition node information.
  • map structure information 40 The following information recorded in the map structure information 40: (1) Map transition node attributes (2) Map transition node information (3) Movement amount data between map transition nodes By referring to these pieces of information, the relationship between two individual environment maps can be analyzed.
  • node information about all nodes is recorded as the map-corresponding data 30p of the map information storage unit 30.
  • the node information is node information (x, y, z, yaw) including the position (x, y, z) of each node and the orientation (yaw) of the robot at each node.
  • the map structure information storage unit 40 stores node information about transition nodes between two pieces of map data, that is, 1F map data and 2F map data. i.e. (a) Node information of node 1D (EV_1f), which is a transition node on the first floor (1F) (b) Node information of node 2A (EV_2f), which is a transition node on the second floor (2F) These node information (x) , y, z, yaw) are recorded.
  • map structure information storage unit 40 stores two pieces of map data, that is, movement amount information between transition nodes between the 1F map data and the 2F map data, that is, (c) Movement amount information between transition nodes (rel_x, rel_y, rel_z, rel_yaw) is recorded.
  • the autonomous mobile robot attempting to move from the node 1A on the first floor to the node 2D on the second floor performs the following processing. By executing these in sequence, it becomes possible to move from the node 1A on the first floor to the node 2D on the second floor.
  • the robot analyzes the amount of movement by the elevator.
  • a specific example of this elevator movement amount analysis processing will be described later.
  • the robot 10 reads the map structure information stored in the map structure information storage unit 40, that is, (1) Map transition node attributes (2) Map transition node information (3) Movement amount data between map transition nodes From among these, "(3) Movement amount data between map transition nodes" is acquired and analyzed by the robot. Judge if it matches the elevator movement amount.
  • the node information 2A (EV_2f) of the elevator stop position on the second floor is also recorded as 2F map corresponding data stored in the map corresponding data 30p of the map information storage unit 30, that is, "(P22) 2F map node information". and the robot acquires this node information.
  • the robot uses the data stored in the map information storage unit 30 and the data stored in the map structure information storage unit 40 to use a plurality of different environment maps without using instructions from the outside such as an operator. Autonomous driving becomes possible.
  • Embodiment 1 An embodiment in which elevator movement is detected using detection information from an air pressure sensor and an IMU (inertial measurement unit) to generate a plurality of maps and map structure information.
  • Embodiment 3 Example of detecting elevator movement using user input information and generating multiple maps and map structure information (Implementation)
  • Example 4 An embodiment that analyzes communication data with a communication unit installed in an elevator boarding/alighting unit, detects elevator movement, and generates a plurality of maps and map structure information.
  • the information processing apparatus of the present disclosure is, for example, an information processing apparatus incorporated in a mobile object such as a robot, or an information processing apparatus that communicates with a mobile object such as a robot, and executes SLAM processing. It is an information processing device that performs map creation, self-position estimation processing, and the like.
  • a representative example of the information processing apparatus an embodiment using an information processing apparatus incorporated in a moving object such as a robot will be described.
  • Example 1 Embodiment 1 Embodiment in Which Elevator Movement is Detected Using Detection Information from an Air Pressure Sensor and an IMU (Inertial Measurement Unit) to Generate Multiple Maps and Map Structure Information
  • Example 1 an example will be described in which a plurality of maps and map structure information are generated by detecting elevator movement using detection information from an air pressure sensor and an IMU (inertial measurement unit).
  • Example 1 is an example in which elevator movement is detected using an air pressure sensor and information detected by an IMU (inertial measurement unit).
  • IMU intial measurement unit
  • FIG. 13 is a diagram explaining the details of the process of detecting elevator movement using detection information from an IMU (inertial measurement unit).
  • the graph shown in FIG. 13 is a graph showing changes in the speed of the elevator during this elevator movement.
  • the horizontal axis indicates time (sec), and the vertical axis indicates elevator speed (m/s).
  • Time t0 is the time when the elevator in which the robot 10 has boarded starts rising from the first floor (1F).
  • the elevator then accelerates from time t0 to time t1, gradually increasing the upward speed. Acceleration is stopped when the speed of the elevator reaches Va, and then the elevator rises at a constant speed (Va) until time t2. After that, deceleration is started at time t2, and the speed becomes 0 at time t3. That is, the elevator stops.
  • This time t3 is the time when the elevator reaches the second floor (2F).
  • the speed change in the graph shown in FIG. 13 is data that can be detected by an IMU (inertial measurement unit) attached to the robot 10 . That is, if the robot 10 is equipped with an IMU (inertial measurement unit) as a sensor, it is possible to detect whether the robot 10 is ascending or descending. However, since the distance between floors varies depending on the structure of the building, it is difficult to accurately determine the number of floors to which the vehicle has moved based on this speed data alone.
  • the robot 10 is equipped with an IMU (inertial measurement unit) as a sensor, and is equipped with an atmospheric pressure sensor for detecting the atmospheric pressure for determining the number of floors to move.
  • the robot 10 analyzes the detected value of the air pressure sensor to determine the number of floors to move.
  • FIG. 14 A specific example of the process of determining the number of floors to be moved by analyzing the detected value of the air pressure sensor will be described with reference to FIG. 14 .
  • an elevator configured in a multi-level building ascends or descends, it passes through each floor of the building.
  • the space in which the floors are configured becomes a sparse space, and the space between the floors becomes a dense space.
  • the space between floors is above the ceiling of the lower floor and under the floor of the upper floor, and since there are steel frames, concrete, various pipes, wiring, etc., it becomes a dense space.
  • the graph shown on the right side of FIG. 14 is a graph showing a specific example of changes in atmospheric pressure when the elevator moves upward from the first floor (1F) to the third floor (3F).
  • the horizontal axis is time t (sec), and the vertical axis is atmospheric pressure (Pa).
  • the elevator From time t0 to t1, the elevator is in a state of being stopped on the first floor, then rises to time t6, and shows changes in atmospheric pressure until it stops on the third floor. The atmospheric pressure will decrease as the elevator rises.
  • FIG. 15 is a block diagram showing a configuration example of the information processing apparatus 100 of the first embodiment.
  • This information processing device 100 is mounted inside a robot 10 that creates a map.
  • a part of the configuration other than the sensors and the like may be set in a device capable of communicating with the robot 10, such as a server.
  • the information processing apparatus 100 includes an atmospheric pressure sensor 101, an IMU (inertial measurement unit) 102, a wheel encoder 103, a LiDAR 104, an elevator movement analysis unit 105, a map generation unit 106, a map structure analysis unit 107, map information It has a storage unit 108 and a map structure information storage unit 109 .
  • IMU intial measurement unit
  • the atmospheric pressure sensor 101 detects atmospheric pressure and outputs atmospheric pressure information 111 as a detected value to the elevator movement analysis unit 105 .
  • An IMU (inertial measurement unit) 102 detects the acceleration and angular velocity of the robot 10 and outputs the detected acceleration and angular velocity 112 to an elevator movement analysis unit 105 and a map generation unit 106 .
  • the wheel encoder 103 measures the amount of rotation of the wheels of the robot, calculates the amount of movement of the robot, and outputs the calculated amount of movement 113 to the map generator 106 .
  • the LiDAR 104 calculates the object distance around the robot based on the laser light, generates point group information 114 indicating the object position based on the calculated result, and outputs the point group information 114 to the map generation unit 106 .
  • the elevator movement analysis unit 105 receives atmospheric pressure information 111 input from the atmospheric pressure sensor 101 and the acceleration and angular velocity 112 of the robot 10 from the IMU (inertial measurement unit) 102, and uses the elevator based on these input information. Elevator detection information indicating whether or not a moving movement occurs, and the amount of elevator movement corresponding to information on the number of floors moved by the elevator are analyzed. The analysis information generated by the elevator movement analysis unit 105 , that is, the elevator detection information & elevator movement amount 115 is output to the map structure analysis unit 107 .
  • the map generating unit 106 inputs the acceleration and angular velocity 112 of the robot 10 from the IMU (inertial measurement unit) 102, inputs the movement amount 113 of the robot from the wheel encoder 103, and furthermore, from the LiDAR 104, calculates the object positions around the robot. Input the point cloud information 114 shown.
  • the map generation unit 106 utilizes these pieces of input information to generate an environment map showing object positions and the like around the travel route of the robot 10, and further performs self-position estimation. For example, the SLAM process using the LiDAR detection information is performed, and the environment map generation process and the self-position estimation process are performed together.
  • a generated map 118 that is an environmental map generated by the map generation unit 106 and other map-related information are stored in the map information storage unit 108 .
  • Estimated self-position 116 calculated by map generation unit 106 is output to map structure analysis unit 107 .
  • the processing of storing the generated map 106 in the map information storage unit 108 in the map generation unit 106 and the processing of generating a new map are executed based on the input of the processing system example from the map structure analysis unit 107 . That is, it is executed based on the input of the "map storage instruction, new map creation instruction 117" output from the map structure analysis unit 107 shown in FIG.
  • the map structure analysis unit 107 receives analysis information from the elevator movement analysis unit 105 , that is, elevator detection information & elevator movement amount 115 . Furthermore, the estimated self-position 116 is input from the map generator 106 . Based on these pieces of information, the map structure analysis unit 107 generates map structure information indicating the relationship between the plurality of maps generated by the map generation unit 106. FIG. That is, the map structure information 119 shown in FIG. 15 is generated, and the generated map-related information is stored in the map structure information storage unit 109 .
  • the map structure analysis unit 107 further includes a map generated by the map generation unit 106 based on the elevator detection information & elevator movement amount 115 input from the elevator movement analysis unit 105 and the estimated self-position 116 input from the map generation unit 106.
  • a map storage instruction is output to the map generation unit 106 at the timing for storing the map in the map information storage unit 108 or at the determined storage timing. It also determines the timing of new map creation by the map generation unit 106, and instructs the map generation unit 106 to start creating the new map at the determined new map creation timing.
  • the map information storage unit 108 stores maps and map-related information generated by the map generation unit 106 .
  • the map structure information storage unit 109 stores map structure information 119 generated by the map structure analysis unit 107 .
  • FIG. 16 Details of the information stored in the map information storage unit 108 and the information stored in the map structure information storage unit 109 will be described with reference to FIGS. 16 and 17.
  • FIG. 16 Details of the information stored in the map information storage unit 108 and the information stored in the map structure information storage unit 109 will be described with reference to FIGS. 16 and 17.
  • FIG. 16 Details of the information stored in the map information storage unit 108 and the information stored in the map structure information storage unit 109 will be described with reference to FIGS. 16 and 17.
  • FIG. 16 shows an example of information stored in the map information storage unit 108 by the map generation unit 106. As shown in FIG. 16
  • the information stored in the map information storage unit 108 by the map generation unit 106 includes map correspondence data 108p and actual map data 108q as shown in FIG.
  • the map correspondence data 108p is composed of the following data, as shown in FIG. (1) Source frame information (2) Map node information (3) Map origin position information (x, y) (4) Map resolution information
  • Source frame information is acquisition destination information of map data (actual map data), and is, for example, a 1F map, a 2F map, or the like.
  • Map node information is a recording area for node information similar to that described with reference to FIG. That is, node information (x, y, z, yaw) consisting of the position (x, y, z) of each node and the orientation (yaw) of the robot is recorded as target frame data.
  • Map origin position information is a coordinate position indicating the origin position of the map in an external coordinate system, for example, a coordinate system such as world coordinates. Although only the two-dimensional coordinate position (x, y) is shown in this example, the three-dimensional coordinate position (x, y, z) may be recorded.
  • Map resolution information is information indicating the resolution of the actual map data 108q.
  • the actual map data 108q stores actual map data generated by the information processing apparatus 100 (robot 10).
  • the actual map data 108q is data similar to the data described above with reference to FIGS. That is, for example, map data (environmental map) representing objects around the robot travel route as point group information (object position information) is stored.
  • map data environment map
  • the environment map is not limited to point cloud data, and may be a map using various representation formats such as voxel representation and primitive shape group using object detection information.
  • FIG. 17 shows an example of information stored in the map structure information storage unit 109 by the map structure information analysis unit 107.
  • map structure information storage unit 109 records the following information.
  • Map transition node attributes (2) Map transition node information (3) Movement amount data between map transition nodes
  • Map transition node information records the node information of each map before and after the map switch. Specifically, the map creation end node that is the final node (map creation end node) of the map that has been created, the map creation start node that is the first node of the map to start creation, the positions of these transition nodes, etc. Record node information, including
  • the node information of two map transition nodes is recorded.
  • a source frame (1F map or 2F map) which is the acquisition destination information of the map data (actual map data);
  • Node information (x, y, z, yaw) consisting of the position (x, y, z) of each node and the orientation (yaw) of the robot
  • Target frame before map transition EV_1F: (x, y, z, yaw)
  • This data is node information about the node of the elevator position on the first floor (1F), and is data according to the coordinate system unique to the 1F map.
  • the 1F map coordinate system is, for example, a coordinate system set with the generation start point of the 1F map as the origin.
  • Target frame after map transition EV_2F: (x, y, z, yaw)
  • This data is node information about the node of the elevator position on the second floor (2F), and is data according to the coordinate system specific to the 2F map.
  • the 2F map coordinate system is, for example, a coordinate system set with the starting point of generation of the 2F map as the origin.
  • the map structure information storage unit 109 records "(3) movement amount data between map transition nodes".
  • the (3) map transition node-to-map movement amount data is data that is optionally recorded, not essential recording data.
  • the movement amount data between map transition nodes is the amount of movement between two map transition nodes, that is, the node at the elevator position on the first floor (1F) and the node at the elevator position on the second floor (2F). Inter-node movement data for one map transition node is recorded.
  • the amount of movement is recorded for each element of (x, y, z, yaw). i.e. Movement amount in x direction (rel_x) Movement amount in y direction (rel_y) Movement amount in z direction (rel_z) Movement amount of yaw angle (rel_yaw) These element-by-element movement amounts are recorded.
  • movement between two map transition nodes is movement only in the vertical direction (z direction).
  • FIG. 18 is a flowchart explaining the processing sequence executed by the elevator movement analysis unit 105 of the information processing apparatus 100 of the first embodiment shown in FIG.
  • the processing according to the flow described below is executed in accordance with a program stored in the storage unit of an information processing device in a mobile object such as a robot or an information processing device capable of communicating with a mobile object such as a robot. It is possible to For example, it can be performed as program execution processing by a processor such as a CPU having a program execution function. Processing of each step of the flow shown in FIG. 18 will be described below.
  • Step S101 First, in step S101, the elevator movement analysis unit 105 receives an IMU detection value, such as the acceleration of the robot, from the IMU (inertial measurement unit) 102 .
  • an IMU detection value such as the acceleration of the robot
  • step S102 the elevator movement analysis unit 105 analyzes the acceleration input from the IMU (inertial measurement unit) 102 and determines whether vertical acceleration has been detected. If acceleration in the vertical direction is detected, the process proceeds to step S103. If it is not detected, the process returns to step S101 and repeats the processes from step S101 onward.
  • IMU intial measurement unit
  • Step S103 When vertical acceleration is detected in step S102, the elevator movement analysis unit 105 notifies the map structure analysis unit 107 of detection of robot movement by the elevator in step S103.
  • Step S104 the elevator movement analysis unit 105 receives atmospheric pressure information from the atmospheric pressure sensor 101 and analyzes the inputted atmospheric pressure information.
  • step S105 Next, in step S ⁇ b>105 , the elevator movement analysis unit 105 determines whether movement between floors has been detected as an analysis result of the atmospheric pressure information input from the atmospheric pressure sensor 101 .
  • This hierarchical movement determination process is a determination process based on whether or not the phenomenon described above with reference to FIG. 14 is detected. That is, the determination is made according to whether or not a disturbance of the initial atmospheric change that occurs according to the density of the space through which the elevator passes has been detected. If the disturbance of the first atmospheric change occurs, it is determined that the movement between floors has been performed.
  • step S106 the elevator movement analysis unit 105 notifies the map structure analysis unit 107 of the analysis information in step S105, that is, detection information of floor movement obtained from the analysis result of the atmospheric pressure information input from the atmospheric pressure sensor 101. do.
  • step S107 the elevator movement analysis unit 105 receives an IMU detection value, such as the acceleration of the robot, from the IMU (inertial measurement unit) 102 .
  • step S108 the elevator movement analysis unit 105 analyzes the acceleration input from the IMU (inertial measurement unit) 102 and determines whether deceleration in the vertical direction has been detected. If vertical deceleration is detected, the process proceeds to step S109. If not detected, the process returns to step S107 and repeats the process of step S107.
  • IMU intial measurement unit
  • Step S109 When the deceleration of the speed in the vertical direction is detected in step S108, the elevator movement analysis unit 105 notifies the map structure analysis unit 107 that the elevator has stopped moving the robot in step S109.
  • the map structure analysis unit 107 successively inputs information such as the start of elevator movement of the robot, floor movement, movement stop, etc., from the elevator movement analysis unit 105 .
  • information such as the start of elevator movement of the robot, floor movement, movement stop, etc.
  • the map structure analysis unit 107 for inputting these pieces of information will be described with reference to the flowchart shown in FIG. The processing from step S121 onward in the flow shown in FIG. 19 will be sequentially described.
  • Step S121 First, in step S121, the map structure analysis unit 107 determines whether or not an elevator movement detection notification has been input from the elevator movement analysis unit 105.
  • FIG. 1 the map structure analysis unit 107 determines whether or not an elevator movement detection notification has been input from the elevator movement analysis unit 105.
  • step S121 This is the notification input confirmation process in step S103 of the flow shown in FIG. If it is determined in step S121 that the elevator movement detection notification from the elevator movement analysis unit 105 has been input, the process proceeds to step S122.
  • Step S122 If the map structure analysis unit 107 determines in step S121 that an elevator movement detection notification has been received from the elevator movement analysis unit 105, the map structure analysis unit 107 executes the process of step S122.
  • map structure analysis unit 107 acquires the elevator position on the created map before the elevator movement, and executes a process of recording it in the map structure information storage unit 109 .
  • This information corresponds to the following information in "(2) Map transition node information" recorded in the map structure information storage unit 109 described above with reference to FIG. (2a) Source frame before map transition Target frame before map transition
  • step S123 the map structure analysis unit 107 outputs to the map generation unit 106 an instruction to store the created map before moving to the elevator.
  • the map structure analysis unit 107 in step S123, An instruction to save the map of the first floor (1F) created before moving to the elevator is output to the generation unit 106 .
  • the map generation unit 106 stores the created map in the map information storage unit 108 according to the instruction from the map structure analysis unit 107 . For example, a map of the first floor (1F) created before moving by the elevator is stored in the map information storage unit 108 .
  • Step S124 the map structure analysis unit 107 determines whether or not a floor movement notification by elevator has been input from the elevator movement analysis unit 105 .
  • step S106 This is the notification input confirmation process in step S106 of the flow shown in FIG. If it is determined in step S124 that an elevator floor movement detection notification from the elevator movement analysis unit 105 has been input, the process proceeds to step S125.
  • Step S125 When the map structure analysis unit 107 determines in step S124 that the floor movement detection notification by the elevator from the elevator movement analysis unit 105 has been input, the process of step S125 is executed.
  • the map structure analysis unit 107 counts the number of floor movement detection notifications from the elevator movement analysis unit 105 in step S125 to determine the number of floors moved.
  • the number of floor movement detection notifications from the elevator movement analysis unit 105 is one, it is determined that the robot has moved one floor by the elevator. If the number of floor movement detection notifications from the elevator movement analysis unit 105 is two, it is determined that the robot has moved two floors by the elevator.
  • step S126 the map structure analysis unit 107 determines whether or not an elevator stop detection notification has been input from the elevator movement analysis unit 105 . If it is determined that the elevator stop detection notification has been input, the process proceeds to step S127. If the elevator stop detection notification has not been input, the process of step S125 is continued.
  • Step S127 If the map structure analysis unit 107 determines in step S126 that the elevator stop detection notification has been input from the elevator movement analysis unit 105, the process of step S127 is executed.
  • the map structure analysis unit 107 stores the following in the map structure information storage unit 109 in step S127.
  • "(3) Movement amount data between map transition nodes” "floor before elevator movement ⁇ floor after elevator movement”
  • This map transition node-to-node movement amount data is recorded. This is the data recorded in the map structure information storage unit 109 previously described with reference to FIG.
  • step S128 the map structure analysis unit 107 notifies the map generation unit 106 of an instruction to create a new map on the floor after moving by the elevator.
  • the map generation unit 106 Upon receiving this notification from the map structure analysis unit 107, the map generation unit 106 starts creating a new map on the floor after the elevator has moved.
  • Step S129 the map structure analysis unit 107 acquires the elevator position on the created map after the elevator movement, and executes processing for recording it in the map structure information storage unit 109.
  • FIG. 1 the map structure information storage unit 109.
  • Map transition node information recorded in the map structure information storage unit 109 described above with reference to FIG. (2b) Source frame after map transition Target frame after map transition
  • the robot 10 creates a map of the 1st floor (1F), gets on the elevator from the 1st floor (1F), moves to the 2nd floor (2F) by the elevator, and creates a map of the 2nd floor (2F).
  • the above information (2b) becomes the following information.
  • Source frame after map transition 2F
  • Target frame after map transition EV_2F: (x, y, z, yaw)
  • FIGS. 20 to 22 show elevator movement analysis unit 105, map structure analysis unit 107, and map generation unit when information processing apparatus 100 (robot 10) creates two maps before and after elevator movement.
  • 106 is a sequence diagram for explaining the sequence of data processing executed in each unit and data transfer processing between units. The processing of each step in this sequence diagram will be described below.
  • Step S151 First, in step S151, the map generating unit 106 of the information processing device 100 (robot 10) starts generating map 1.
  • FIG. For example, SLAM processing is started from the initial position of the first floor (1F), and map creation of the first floor (1F) is started.
  • Step S152 when the elevator movement analysis unit 105 detects the vertical acceleration of the robot 10 based on the acceleration information input from the IMU (inertial measurement unit) 102, the map structure analysis unit 107 detects the robot movement caused by the elevator. Notify when movement is detected.
  • Step S153 When the map structure analysis unit 107 receives the detection notification of the robot movement by the elevator from the elevator movement analysis unit 105, in step S153, the map structure analysis unit 107 inquires of the map generation unit 106 about the current self-position.
  • Step S154 In response to a self-location inquiry from the map structure analysis unit 107, the map generation unit 106 transmits the self-position calculated in the SLAM processing to the map structure analysis unit 107.
  • this self-position is the self-position in the coordinate system of the map 1 whose preparation is started in step S151, and corresponds to the elevator position.
  • the map 1 started to be created in step S151 is the map of the 1st floor (1F)
  • the map structure analysis unit 107 acquires the elevator position on the created map before the elevator movement, and executes a process of recording it in the map structure information storage unit 109 .
  • the map structure analysis unit 107 records the self position acquired from the map generation unit 106, ie, the elevator position, in the map structure information storage unit 109.
  • the map structure analysis unit 107 acquires the elevator position on the created map before the elevator moves, and executes a process of recording it as map structure information.
  • the recorded information corresponds to the following information in "(2) map transition node information" recorded in the map structure information storage unit 109 described with reference to FIG. (2a) Source frame before map transition Target frame before map transition
  • step S156 the map structure analysis unit 107 outputs to the map generation unit 106 an instruction to store the created map before the elevator movement.
  • This process corresponds to the process of step S123 described above with reference to the flow of FIG.
  • the map structure analysis unit 107 in step S156, An instruction to save the map of the first floor (1F) created before moving to the elevator is output to the generation unit 106 .
  • Map generation unit 106 stores the created map in map information storage unit 108 in accordance with the instruction from map structure analysis unit 107 in step S157. For example, a map of the first floor (1F) created before moving by the elevator is stored in the map information storage unit 108 .
  • step S158 the elevator movement analysis unit 105 outputs an elevator floor movement detection notification.
  • This process corresponds to the process of step S106 in the flow shown in FIG. 17 described above.
  • Step S159 the map structure analysis unit 107 determines the number of floors moved based on the floor movement detection notification from the elevator movement analysis unit 105.
  • step S125 of the flow shown in FIG. 19 This process is in contrast to the process of step S125 of the flow shown in FIG. 19 described above. For example, if the number of floor movement detection notifications from the elevator movement analysis unit 105 is one, it is determined that the robot has moved one floor by the elevator. If the number of floor movement detection notifications from the elevator movement analysis unit 105 is two, it is determined that the robot has moved two floors by the elevator.
  • Step S160 Next, in step S160, the elevator movement analysis unit 105 notifies the map structure analysis unit 107 that the elevator has stopped.
  • This process is a process corresponding to the process of step S109 in the flow of FIG. 18 described above.
  • the elevator movement analysis unit 105 detects deceleration in the vertical direction based on the input from the IMU (inertial measurement unit) 102, the elevator movement analysis unit 105 notifies the map structure analysis unit 107 of detection of the elevator stop.
  • Step S161 when the map structure analysis unit 107 receives the elevator stop notification from the elevator movement analysis unit 105, it analyzes the elevator movement amount and records the analyzed movement amount in the map structure information storage unit 109 in step S161.
  • the movement amount information recorded in the map structure information storage unit 109 is the recorded information described above with reference to FIG. (1) Map transition node attribute (2) Map transition node information (3) Movement amount data between map transition nodes This corresponds to "movement amount data between map transition nodes" in these pieces of information.
  • “(3) Movement amount data between map transition nodes” is the amount of movement between two map transition nodes in two maps generated by the robot 10, for example, the node at the elevator position on the first floor (1F) and the node on the second floor. It is the movement amount data between the two map transition nodes of the node of the elevator position of (2F).
  • the movement amount may be recorded for each element of (x, y, z, yaw). i.e. Movement amount in x direction (rel_x) Movement amount in y direction (rel_y) Movement amount in z direction (rel_z) Movement amount of yaw angle (rel_yaw) These element-by-element movement amounts may be recorded.
  • Step S162 the map structure analysis unit 107 instructs the map generation unit 106 to create a new map on the floor after moving by the elevator.
  • step S163 the map generation unit 106 starts to generate a new map (map 2) in response to the input of the new map generation instruction from the map structure analysis unit 107.
  • step S164 the map structure analysis unit 107 inquires of the map generation unit 106 about the current self-position.
  • Step S165 In response to a self-location inquiry from the map structure analysis unit 107, the map generation unit 106 transmits the self-position calculated in the SLAM processing to the map structure analysis unit 107.
  • this self-position is the self-position in the coordinate system of the map 2 whose creation is started in step S163, and corresponds to the elevator position after movement by the elevator.
  • the map 2 started to be created in step S163 is the map of the second floor (2F)
  • the map structure analysis unit 107 records the self-position acquired from the map generation unit 106, that is, the elevator position, in the map structure information storage unit 109.
  • the map structure analysis unit 107 acquires the elevator position on the created map after the elevator movement and executes processing for recording it as map structure information.
  • the recorded information corresponds to the following information in "(2) map transition node information" recorded in the map structure information storage unit 109 described with reference to FIG. (2b) Source frame after map transition Target frame after map transition
  • the robot 10 creates a map of the 1st floor (1F), gets into the elevator from the 1st floor (1F), moves in the elevator, and after arriving at the 2nd floor (2F), moves to the 2nd floor (2F).
  • the above information (2b) becomes the following information.
  • Source frame after map transition 2F
  • Target frame after map transition EV_2F: (x, y, z, yaw)
  • the map structure information storage unit 109 is configured to record information clarifying the relationship between a plurality of maps.
  • Map transition node attribute (2) Map transition node information (3) Movement amount data between map transition nodes These data are recorded. By recording these data, it becomes possible to analyze the relationship between a plurality of individual maps recorded in the map information storage unit 108 .
  • autonomous driving using a plurality of different environmental maps can be achieved without using external instructions such as an operator. can be performed by the robot.
  • Example 2 An embodiment in which a plurality of maps and map structure information are generated by analyzing an image captured by a camera to detect elevator movement.
  • a second embodiment is an embodiment that analyzes an image captured by a camera, detects elevator movement, and generates a plurality of maps and map structure information.
  • Figures 23 to 25 are diagrams explaining the details of the process of analyzing the captured image of the camera and detecting the movement of the elevator.
  • the robot 10 that performs the map creation process first creates a map of the first floor (1F) and arrives in front of the elevator on the first floor (1F). This state is the state shown in FIG. Next, as shown in FIG. 23(b), the robot 10 gets into an elevator with an open door on the floor where map creation has been completed, for example, the first floor (1F).
  • This state is the state shown in FIG.
  • the robot 10 photographs the state inside the elevator with a camera attached to the robot 10.
  • the elevator then begins to ascend or descend to move to the next mapping floor. For example, if the current point is on the first floor (1F) and the next mapping floor is on the second floor (2F), then start climbing to move to the second floor (2F), for example.
  • This state is the state shown in FIG.
  • the robot 10 photographs the floor display section in the elevator with a camera attached to the robot 10 .
  • the data processing unit of the information processing device in the robot 10 analyzes the captured image of the floor number display unit in the elevator to confirm the number of floors to which the elevator is moved.
  • the elevator then reaches the next mapping floor, for example the second floor (2F), and the door is opened.
  • This state is the state shown in FIG.
  • a data processing unit of the information processing device in the robot 10 analyzes an image of the interior of the elevator and detects an open space.
  • an image captured by a camera attached to the robot 10 and a point cloud map generated using LiDAR detection information are analyzed to detect elevator movement, and a plurality of maps and map structure information are generated. Generate.
  • FIG. 26 is a block diagram showing a configuration example of the information processing apparatus 120 of the second embodiment.
  • This information processing device 120 is mounted inside the robot 10 that creates a map.
  • a part of the configuration other than the sensors and the like may be set in a device capable of communicating with the robot 10, such as a server.
  • the information processing device 120 includes a camera 121, an IMU (inertial measurement unit) 102, a wheel encoder 103, a LiDAR 104, an elevator movement analysis unit 105, a map generation unit 106, a map structure analysis unit 107, and a map information storage. 108 and a map structure information storage unit 109 .
  • Components other than camera 121 that is, IMU (inertial measurement unit) 102, wheel encoder 103, LiDAR 104, elevator movement analysis unit 105, map generation unit 106, map structure analysis unit 107, map information storage unit 108, map structure information storage A unit 109 has the same configuration as the information processing apparatus 100 of the first embodiment described above with reference to FIG.
  • the captured image information 125 of the camera 121 and the point cloud information 114 generated by the LiDAR 104 are input to the elevator movement analysis unit 105 .
  • the camera 121 captures an image around the robot and outputs the captured image, that is, captured image information 125 shown in FIG.
  • the LiDAR 104 calculates the object distance around the robot based on the laser light, generates point cloud information 114 indicating the object position based on the calculated result, and outputs it to the elevator movement analysis unit 105 and the map generation unit 106.
  • the elevator movement analysis unit 105 receives the captured image information 125 generated by the camera 121 and the point group information 114 generated by the LiDAR 104, and based on these input information, determines whether movement using the elevator has occurred. and the elevator movement amount corresponding to the information on the number of floors moved by the elevator are analyzed.
  • the elevator movement analysis unit 105 can analyze, for example, the photographed image information 125 generated by the camera 121 to analyze the elevator boarding/alighting of the robot and the number of elevator boarding/alighting floors.
  • the analysis information generated by the elevator movement analysis unit 105 that is, the elevator detection information & elevator movement amount 115 is output to the map structure analysis unit 107 .
  • the processing by the elevator movement analysis unit 105 is significantly different from that in the first embodiment.
  • the elevator movement analysis unit 105 uses the captured image information 125 generated by the camera 121 and the point cloud information 114 generated by the LiDAR 104 to detect elevator detection information as to whether or not movement using the elevator has occurred, Elevator travel amount corresponding to information on the number of floors traveled by the elevator is analyzed.
  • the elevator movement analysis unit 105 when the robot 10 enters the elevator and the elevator is closed, the elevator movement analysis unit 105 generates captured image information 125 generated by the camera 121, That is, it detects that the robot 10 has entered the closed space by analyzing the captured image of the interior of the elevator.
  • the camera 121 attached to the robot 10 photographs the floor display section inside the elevator.
  • the elevator movement analysis unit 105 analyzes captured image information 125 generated by the camera 121 . In other words, the number of floors to which the elevator moves is confirmed by analyzing an image of the floor number display section in the elevator.
  • the door is opened when the elevator reaches the next mapping floor, for example, the second floor (2F).
  • the elevator movement analysis unit 105 analyzes the captured image information 125 generated by the camera 121 to detect an open space.
  • an image captured by a camera attached to the robot 10 and a point cloud map generated using LiDAR detection information are analyzed to detect elevator movement, and a plurality of maps and map structure information are generated. Generate.
  • FIG. 27 is a flowchart for explaining the processing sequence executed by the elevator movement analysis unit 105 of the information processing apparatus 120 of the second embodiment shown in FIG.
  • the processing according to the flow described below is executed in accordance with a program stored in the storage unit of an information processing device in a mobile object such as a robot or an information processing device capable of communicating with a mobile object such as a robot. It is possible to For example, it can be performed as program execution processing by a processor such as a CPU having a program execution function. Processing of each step of the flow shown in FIG. 27 will be described below.
  • Step S201 First, in step S201, the elevator movement analysis unit 105 receives the point cloud information 114 generated by the LiDAR 104 and analyzes the input point cloud information.
  • the LiDAR 104 calculates the object distance around the robot based on the laser beam, generates the point group information 114 indicating the object position based on the calculated result, and outputs it to the elevator movement analysis unit 105.
  • step S202 the elevator movement analysis unit 105 determines whether or not it has detected that the robot 10 has entered a closed space as the analysis result of the point cloud information 114 input from the LiDAR 104 .
  • step S203 If a closed space is detected, proceed to step S203. If it is not detected, the process returns to step S201 and repeats the processing from step S201 onwards.
  • Steps S203-S204 When a closed space is detected in step S202, the elevator movement analysis unit 105 analyzes the camera-captured image and determines whether or not it is inside an elevator in step S203. For example, it analyzes whether or not the captured image has a feature amount inside the elevator. If it is determined that the vehicle is inside the elevator based on the image captured by the camera, the process proceeds to step S205.
  • Step S205 the elevator movement analysis unit 105 analyzes the image captured by the camera, detects the floor display part from the captured image, and confirms the current floor and the floor movement.
  • step S206 the elevator movement analysis unit 105 notifies the map structure analysis unit 107 of detection of robot movement by the elevator.
  • step S207 the elevator movement analysis unit 105 receives the point cloud information 114 generated by the LiDAR 104 and analyzes the input point cloud information.
  • the LiDAR 104 calculates the object distance around the robot based on the laser beam, generates the point group information 114 indicating the object position based on the calculated result, and outputs it to the elevator movement analysis unit 105.
  • step S208 the elevator movement analysis unit 105 determines whether an open space has been detected from the analysis result of the point group information 114 input from the LiDAR 104.
  • Step S209 the elevator movement analysis unit 105 analyzes the photographed image information 125 generated by the camera 121 in step S209. In other words, the number of floors to which the elevator moves is confirmed by analyzing an image of the floor number display section in the elevator.
  • Step S210 Next, in step S210, the elevator movement analysis unit 105 notifies the map structure analysis unit 107 that the elevator has stopped moving the robot.
  • the map structure analysis unit 107 successively inputs information such as the start of elevator movement of the robot, floor movement, movement stop, etc., from the elevator movement analysis unit 105 .
  • information such as the start of elevator movement of the robot, floor movement, movement stop, etc.
  • the sequence of processing executed by the map structure analysis unit 107 for inputting these pieces of information will be described with reference to the flowchart shown in FIG. Processing after step S221 in the flow shown in FIG. 28 will be described in order.
  • Step S221 First, in step S221, the map structure analysis unit 107 determines whether or not an elevator movement detection notification has been input from the elevator movement analysis unit 105.
  • FIG. 1 the map structure analysis unit 107 determines whether or not an elevator movement detection notification has been input from the elevator movement analysis unit 105.
  • step S206 This is the notification input confirmation process in step S206 of the flow shown in FIG. If it is determined in step S221 that the elevator movement detection notification from the elevator movement analysis unit 105 has been input, the process proceeds to step S222.
  • Step S222 If the map structure analysis unit 107 determines in step S221 that an elevator movement detection notification has been received from the elevator movement analysis unit 105, the map structure analysis unit 107 executes the process of step S222.
  • map structure analysis unit 107 acquires the elevator position on the created map before the elevator movement, and executes a process of recording it in the map structure information storage unit 109 .
  • This information corresponds to the following information in "(2) Map transition node information" recorded in the map structure information storage unit 109 described above with reference to FIG. (2a) Source frame before map transition Target frame before map transition
  • step S223 the map structure analysis unit 107 outputs to the map generation unit 106 an instruction to store the created map before moving to the elevator.
  • the map structure analysis unit 107 In step S223, An instruction to save the map of the first floor (1F) created before moving to the elevator is output to the generation unit 106 .
  • the map generation unit 106 stores the created map in the map information storage unit 108 according to the instruction from the map structure analysis unit 107 . For example, a map of the first floor (1F) created before moving by the elevator is stored in the map information storage unit 108 .
  • Step S224 the map structure analysis unit 107 determines whether or not a notification of elevator floor movement stoppage and stop floor number has been input from the elevator movement analysis unit 105 or not.
  • step S224 This is the notification input confirmation process in step S210 of the flow shown in FIG. If it is determined in step S224 that the notification of the elevator floor movement stop and the number of floors to be stopped from the elevator movement analysis unit 105 has been input, the process proceeds to step S225.
  • Step S225 If the map structure analysis unit 107 determines in step S224 that the elevator movement analysis unit 105 has input the notification of the floor movement stop by the elevator and the number of floors to stop, the map structure analysis unit 107 executes the process of step S225.
  • step S226 the map structure analysis unit 107 notifies the map generation unit 106 of an instruction to create a new map on the floor after moving by the elevator.
  • the map generation unit 106 Upon receiving this notification from the map structure analysis unit 107, the map generation unit 106 starts creating a new map on the floor after the elevator has moved.
  • Step S227 the map structure analysis unit 107 acquires the elevator position on the created map after the elevator movement, and executes a process of recording it in the map structure information storage unit 109.
  • FIG. 1 the map structure information storage unit 109.
  • Map transition node information recorded in the map structure information storage unit 109 described above with reference to FIG. (2b) Source frame after map transition Target frame after map transition
  • the robot 10 creates a map of the 1st floor (1F), gets on the elevator from the 1st floor (1F), moves to the 2nd floor (2F) by the elevator, and creates a map of the 2nd floor (2F).
  • the above information (2b) becomes the following information.
  • Source frame after map transition 2F
  • Target frame after map transition EV_2F: (x, y, z, yaw)
  • an image captured by a camera attached to the robot 10 and a point cloud map generated using LiDAR detection information are analyzed to detect elevator movement, and a plurality of maps and map structure information are generated. Generate.
  • Example 3 Embodiment 3 Embodiment of Detecting Elevator Movement Using User Input Information and Generating Multiple Maps and Map Structure Information
  • Example 3 an example in which elevator movement is detected using user input information and a plurality of maps and map structure information are generated will be described.
  • Example 3 is an example in which user input information is used to detect elevator movement and generate multiple maps and map structure information.
  • 29 to 30 are diagrams explaining the details of the process of detecting elevator movement using user input information.
  • the robot 10 that performs the map creation process first creates a map of the first floor (1F) and arrives in front of the elevator on the first floor (1F). This state is the state shown in FIG. Next, as shown in FIG. 29(b), the robot 10 gets into an elevator with an open door on the floor where map creation has been completed, for example, the first floor (1F).
  • the user When a user such as an operator observes the movement of the robot 10 and confirms that the robot 10 has boarded the elevator, the user inputs that the robot 10 has boarded the elevator via the UI section of the information processing device.
  • the input information from the user may be directly input to the UI unit of the robot 10, but may also be input via a UI unit such as a controller capable of communicating with the robot 10.
  • This state is the state shown in FIG.
  • a user such as an operator observes the movement of the robot 10, confirms that the robot 10 has entered the elevator, closed the elevator doors, and started moving. Enters that the elevator in which is boarded has started to move.
  • the input information from the user may be directly input to the UI unit of the robot 10, but may also be input via a UI unit such as a controller capable of communicating with the robot 10.
  • the elevator then reaches the next mapping floor, for example the second floor (2F), and the door is opened.
  • This state is the state shown in FIG.
  • the elevator in which the robot 10 has boarded is displayed via the UI section of the information processing device. Enter information such as the fact that the vehicle stopped, the number of floors it stopped, and the fact that the door was opened.
  • the input information from the user may be directly input to the UI unit of the robot 10, but may also be input via a UI unit such as a controller capable of communicating with the robot 10.
  • the third embodiment is an embodiment in which the movement of the elevator is detected using the user input information of the user who observes the movement of the robot 10, and multiple maps and map structure information are generated.
  • FIG. 31 is a block diagram showing a configuration example of the information processing device 130 of the third embodiment.
  • This information processing device 130 is mounted inside the robot 10 that creates a map.
  • a part of the configuration other than the sensors and the like may be set in a device capable of communicating with the robot 10, such as a server.
  • the information processing device 130 includes a UI unit 131, an inertial measurement unit (IMU) 102, a wheel encoder 103, a LiDAR 104, a map generation unit 106, a map structure analysis unit 107, a map information storage unit 108, a map structure It has an information storage unit 109 .
  • IMU inertial measurement unit
  • IMU intial measurement unit
  • wheel encoder 103 wheel encoder
  • LiDAR 104 map generation unit
  • map structure analysis unit 107 map information storage unit 108
  • map structure information storage unit 109 map structure information storage unit 109
  • user input information 135 input via the UI unit 131 is input to the map structure analysis unit 107.
  • the user input information 135 input via the UI unit 131 is, as described above with reference to FIGS. 29 and 30, (a) that the robot 10 got into the elevator (b) that the elevator into which the robot 10 got into started to move (c) that the elevator into which the robot 10 got into stopped and the number of floors to stop These pieces of information.
  • the map structure analysis unit 107 analyzes these pieces of user input information, analyzes that the robot 10 is moving using an elevator, analyzes the number of floors to which the robot 10 is moving, and analyzes the map structure described above with reference to FIG. It performs processing such as generating structure information.
  • Step S301 First, in step S ⁇ b>301 , the map structure analysis unit 107 determines whether or not the notification of the robot entering the elevator and the movement start has been input from the UI unit 131 . If so, the process proceeds to step S302.
  • Step S302 When the map structure analysis unit 107 determines in step S301 that the notification of the robot getting into the elevator and the movement start has been input from the UI unit 131, the map structure analysis unit 107 executes the process of step S302.
  • map structure analysis unit 107 acquires the elevator position on the created map before the elevator movement, and executes a process of recording it in the map structure information storage unit 109 .
  • This information corresponds to the following information in "(2) Map transition node information" recorded in the map structure information storage unit 109 described above with reference to FIG. (2a) Source frame before map transition Target frame before map transition
  • step S303 the map structure analysis unit 107 outputs to the map generation unit 106 an instruction to store the created map before the elevator movement.
  • the map structure analysis unit 107 in step S123, An instruction to save the map of the first floor (1F) created before moving to the elevator is output to the generation unit 106 .
  • the map generation unit 106 stores the created map in the map information storage unit 108 according to the instruction from the map structure analysis unit 107 . For example, a map of the first floor (1F) created before moving by the elevator is stored in the map information storage unit 108 .
  • Step S304 Next, in step S ⁇ b>304 , the map structure analysis unit 107 determines whether or not notification of the elevator stop and the number of floors to be moved has been input from the UI unit 131 . If it is determined that the notification has been input, the process proceeds to step S305.
  • Step S305 If the map structure analysis unit 107 determines in step S304 that the notification of the elevator stop and the number of floors to be moved has been input from the UI unit 131, the map structure analysis unit 107 executes the process of step S305.
  • the map structure analysis unit 107 stores the following in the map structure information storage unit 109 in step S305.
  • "(3) Movement amount data between map transition nodes” "floor before elevator movement ⁇ floor after elevator movement”
  • This map transition node-to-node movement amount data is recorded. This is the data recorded in the map structure information storage unit 109 previously described with reference to FIG.
  • step S306 the map structure analysis unit 107 notifies the map generation unit 106 of an instruction to create a new map on the floor after moving by the elevator.
  • the map generation unit 106 Upon receiving this notification from the map structure analysis unit 107, the map generation unit 106 starts creating a new map on the floor after the elevator has moved.
  • Step S307 the map structure analysis unit 107 acquires the elevator position on the created map after the elevator movement, and executes processing for recording it in the map structure information storage unit 109.
  • FIG. 1 the map structure information storage unit 109.
  • Map transition node information recorded in the map structure information storage unit 109 described above with reference to FIG. (2b) Source frame after map transition Target frame after map transition
  • the robot 10 creates a map of the 1st floor (1F), gets on the elevator from the 1st floor (1F), moves to the 2nd floor (2F) by the elevator, and creates a map of the 2nd floor (2F).
  • the above information (2b) becomes the following information.
  • Source frame after map transition 2F
  • Target frame after map transition EV_2F: (x, y, z, yaw)
  • the third embodiment uses user input information from a user who observes the movement of the robot 10 to detect elevator movement and generate multiple maps and map structure information.
  • Example 4 Embodiment in which Elevator Movement is Detected by Analyzing Communication Data with a Communication Unit Installed in an Elevator Boarding/Alighting Section, and Multiple Maps and Map Structure Information are Generated
  • Example 4 an example will be described in which communication data with a communication section installed in an elevator boarding/alighting section is analyzed, elevator movement is detected, and a plurality of maps and map structure information are generated.
  • a fourth embodiment is an embodiment that analyzes communication data with a communication section installed in an elevator boarding/alighting section, detects elevator movement, and generates a plurality of maps and map structure information.
  • a data communication unit 145 is provided at the entrance/exit of the elevator.
  • a data communication unit 145 at the entrance/exit of the elevator communicates with the information processing device of the robot 10 . For example, it notifies the robot 10 that it is an elevator entrance and exit, current floor information, and the like.
  • the robot 10 confirms that it is an elevator doorway and the current floor based on the information received from the data communication unit 145 of the elevator doorway. Based on this confirmation result, the elevator movement of the robot 10 is analyzed to generate a plurality of maps and map structure information.
  • FIG. 34 is a block diagram showing a configuration example of the information processing device 140 of the fourth embodiment.
  • This information processing device 140 is mounted inside the robot 10 that creates a map.
  • a part of the configuration other than the sensors and the like may be set in a device capable of communicating with the robot 10, such as a server.
  • the information processing device 140 includes a communication unit 141, an inertial measurement unit (IMU) 102, a wheel encoder 103, a LiDAR 104, an elevator movement analysis unit 105, a map generation unit 106, a map structure analysis unit 107, map information It has a storage unit 108 and a map structure information storage unit 109 .
  • IMU inertial measurement unit
  • Components other than the communication unit 141 that is, IMU (inertial measurement unit) 102, wheel encoder 103, LiDAR 104, elevator movement analysis unit 105, map generation unit 106, map structure analysis unit 107, map information storage unit 108, map structure information
  • the storage unit 109 has the same configuration as the information processing apparatus 100 of the first embodiment described above with reference to FIG.
  • the reception information 142 of the communication unit 141 and the point cloud information 114 generated by the LiDAR 104 are input to the elevator movement analysis unit 105.
  • the communication unit 141 receives information transmitted by the communication unit 145 installed at the elevator doorway described with reference to FIG. For example, it receives from the communication unit 145 the fact that it is an elevator entrance/exit, the current floor information, and the like. Communication unit 141 outputs reception information 142 to elevator movement analysis unit 105 .
  • the LiDAR 104 calculates the object distance around the robot based on the laser light, generates point cloud information 114 indicating the object position based on the calculated result, and outputs it to the elevator movement analysis unit 105 and the map generation unit 106.
  • the elevator movement analysis unit 105 inputs the reception information 142 received by the communication unit 141 and the point group information 114 generated by the LiDAR 104, and based on these input information, determines whether movement using the elevator has occurred. and the elevator movement amount corresponding to the information on the number of floors moved by the elevator are analyzed.
  • the elevator movement analysis unit 105 can analyze, for example, the reception information 142 received by the communication unit 141 to analyze the elevator boarding/alighting of the robot and the number of elevator boarding/alighting floors.
  • the analysis information generated by the elevator movement analysis unit 105 that is, the elevator detection information & elevator movement amount 115 is output to the map structure analysis unit 107 .
  • the processing by the elevator movement analysis unit 105 is significantly different from that in the first embodiment.
  • the elevator movement analysis unit 105 uses the reception information 142 received by the communication unit 141 from the communication unit 145 installed at the elevator entrance and the point group information 114 generated by the LiDAR 104 to determine whether movement using the elevator occurs. Elevator detection information indicating whether or not the elevator is moving and the amount of elevator movement corresponding to information on the number of floors moved by the elevator are analyzed.
  • FIG. 35 is a flowchart for explaining the processing sequence executed by the elevator movement analysis unit 105 of the information processing apparatus 140 of the fourth embodiment shown in FIG.
  • the processing according to the flow described below is executed in accordance with a program stored in the storage unit of an information processing device in a mobile object such as a robot or an information processing device capable of communicating with a mobile object such as a robot. It is possible to For example, it can be performed as program execution processing by a processor such as a CPU having a program execution function. Processing of each step of the flow shown in FIG. 35 will be described below.
  • Step S401 First, the elevator movement analysis unit 105 determines whether input information from the communication unit 141 has been detected in step S401. If detected, the process proceeds to step S402.
  • Step S402 the elevator movement analysis unit 105 acquires elevator detection information and current floor information from the information input from the communication unit 141 in step S402.
  • step S403 the elevator movement analysis unit 105 receives the point cloud information 114 generated by the LiDAR 104 and analyzes the input point cloud information.
  • the LiDAR 104 calculates the object distance around the robot based on the laser beam, generates the point group information 114 indicating the object position based on the calculated result, and outputs it to the elevator movement analysis unit 105.
  • Step S404 the elevator movement analysis unit 105 determines whether or not it has detected that the robot 10 has entered a closed space as the analysis result of the point group information 114 input from the LiDAR 104.
  • Step S405 If a closed space is detected in step S404, the elevator movement analysis unit 105 receives an IMU detection value, such as robot acceleration, from the IMU (inertial measurement unit) 102 in step S405.
  • an IMU detection value such as robot acceleration
  • step S406 the elevator movement analysis unit 105 analyzes the acceleration input from the IMU (inertial measurement unit) 102, and determines whether vertical acceleration has been detected. If vertical acceleration is detected, the process proceeds to step S407. If not detected, the process returns to step S405, and the processes from step S405 onward are repeated.
  • IMU intial measurement unit
  • Step S407 the elevator movement analysis unit 105 notifies the map structure analysis unit 107 of detection of floor movement of the robot by the elevator.
  • step S408 the elevator movement analysis unit 105 receives the point cloud information 114 generated by the LiDAR 104 and analyzes the input point cloud information.
  • the LiDAR 104 calculates the object distance around the robot based on the laser beam, generates the point group information 114 indicating the object position based on the calculated result, and outputs it to the elevator movement analysis unit 105.
  • step S409 the elevator movement analysis unit 105 determines whether an open space has been detected from the analysis result of the point group information 114 input from the LiDAR 104.
  • step S410 If an open space is detected, proceed to step S410. If not detected, the process returns to step S408, and the processing from step S408 onward is repeated.
  • Step S410 the elevator movement analysis unit 105 acquires the current floor, that is, the number of the floor reached by the elevator movement, based on the input information from the communication unit 141.
  • FIG. 1 the elevator movement analysis unit 105 acquires the current floor, that is, the number of the floor reached by the elevator movement, based on the input information from the communication unit 141.
  • step S411 the elevator movement analysis unit 105 notifies the map structure analysis unit 107 of the detection of the robot's movement stop by the elevator and the number of floors to stop.
  • the map structure analysis unit 107 successively inputs information such as the start of elevator movement of the robot, floor movement, movement stop, etc., from the elevator movement analysis unit 105 .
  • information such as the start of elevator movement of the robot, floor movement, movement stop, etc.
  • the sequence of processing executed by the map structure analysis unit 107 for inputting these pieces of information will be described with reference to the flowchart shown in FIG. Processing after step S421 in the flow shown in FIG. 36 will be described in order.
  • Step S421 First, in step S421, the map structure analysis unit 107 determines whether or not an elevator movement detection notification has been input from the elevator movement analysis unit 105.
  • FIG. 1 the map structure analysis unit 107 determines whether or not an elevator movement detection notification has been input from the elevator movement analysis unit 105.
  • step S406 This is the notification input confirmation process in step S406 of the flow shown in FIG. If it is determined in step S421 that the elevator movement detection notification from the elevator movement analysis unit 105 has been input, the process proceeds to step S422.
  • Step S422 If the map structure analysis unit 107 determines in step S421 that an elevator movement detection notification has been received from the elevator movement analysis unit 105, the map structure analysis unit 107 executes the process of step S422.
  • map structure analysis unit 107 acquires the elevator position on the created map before the elevator movement, and executes a process of recording it in the map structure information storage unit 109 .
  • This information corresponds to the following information in "(2) Map transition node information" recorded in the map structure information storage unit 109 described above with reference to FIG. (2a) Source frame before map transition Target frame before map transition
  • step S423 the map structure analysis unit 107 outputs to the map generation unit 106 an instruction to save the created map before moving to the elevator.
  • the robot 10 creates a map of the 1st floor (1F), gets into the elevator from the 1st floor (1F), and starts moving in the elevator, the map structure analysis unit 107, in step S423, An instruction to save the map of the first floor (1F) created before moving to the elevator is output to the generation unit 106 .
  • the map generation unit 106 stores the created map in the map information storage unit 108 according to the instruction from the map structure analysis unit 107 . For example, a map of the first floor (1F) created before moving by the elevator is stored in the map information storage unit 108 .
  • step S424 the map structure analysis unit 107 determines whether or not the elevator movement analysis unit 105 has input an elevator stop and movement floor notification. If it is determined that the notification has been input, the process proceeds to step S425.
  • Step S425) If the map structure analysis unit 107 determines in step S424 that the elevator movement analysis unit 105 has input the elevator stop and movement floor notification, the map structure analysis unit 107 executes the process of step S425.
  • the map structure analysis unit 107 stores the following in the map structure information storage unit 109 in step S425.
  • "(3) Movement amount data between map transition nodes” "floor before elevator movement ⁇ floor after elevator movement”
  • This map transition node-to-node movement amount data is recorded. This is the data recorded in the map structure information storage unit 109 previously described with reference to FIG.
  • step S426 the map structure analysis unit 107 notifies the map generation unit 106 of an instruction to create a new map on the floor after moving by the elevator.
  • the map generation unit 106 Upon receiving this notification from the map structure analysis unit 107, the map generation unit 106 starts creating a new map on the floor after the elevator has moved.
  • Step S427) the map structure analysis unit 107 acquires the elevator position in the created map after the elevator movement, and executes processing for recording it in the map structure information storage unit 109.
  • FIG. 1 the map structure information storage unit 109.
  • Map transition node information recorded in the map structure information storage unit 109 described above with reference to FIG. (2b) Source frame after map transition Target frame after map transition
  • the robot 10 creates a map of the 1st floor (1F), gets on the elevator from the 1st floor (1F), moves to the 2nd floor (2F) by the elevator, and creates a map of the 2nd floor (2F).
  • the above information (2b) becomes the following information.
  • Source frame after map transition 2F
  • Target frame after map transition EV_2F: (x, y, z, yaw)
  • the fourth embodiment analyzes communication data with the communication unit installed in the elevator boarding/alighting unit, detects elevator movement, and generates multiple maps and map structure information.
  • Example 5 Returning to the floor where the map has been created and updating the created map]
  • Example 5 an example of returning to the floor on which the map has already been created and updating the created map will be described.
  • FIG. 37(a) the robot 10 moves to the second floor (2F) by elevator after creating the map of the first floor (1F), and completes the creation of the map of the second floor (2F). state.
  • the robot 10 creates two floor-by-floor maps in accordance with the processing of the first embodiment described above, stores the created maps in the map information storage unit 108, and stores the map structure information in the map structure information storage unit 109. is stored in
  • the robot 10 uses the elevator to return from the second floor (2F) to the first floor (1F). Furthermore, as shown in FIG. 37(c), the robot 10 moves on the first floor (1F) and creates a map of the first floor (1F) again.
  • the fifth embodiment is a process that is applied when the robot 10 creates a map again for an area in which a map that has already been created exists.
  • the robot 10 when the robot 10 creates a map again for an area where a created map already exists, the robot 10 acquires the created 1F map_V1 with the position information of the map switching position as the initial position. Then, the 1F map_V2 is generated by the process of updating the acquired map.
  • either the 1F map_V1 or the 2F map_V1 may be used for updating.
  • FIG. 38 is a block diagram showing a configuration example of the information processing apparatus 150 of the fifth embodiment.
  • This information processing device 150 is mounted inside the robot 10 that creates a map.
  • a part of the configuration other than the sensors and the like may be set in a device capable of communicating with the robot 10, such as a server.
  • the information processing device 150 includes an air pressure sensor 101, an IMU (inertial measurement unit) 102, a wheel encoder 103, a LiDAR 104, an elevator movement analysis unit 105, a map generation unit 106, a map structure analysis unit 107, map information It has a storage unit 108 and a map structure information storage unit 109 . These have the same configuration as the information processing apparatus 100 of the first embodiment described above with reference to FIG.
  • a difference from the first embodiment is notification data 151 that the map structure analysis unit 107 outputs to the map generation unit 106 .
  • the map structure analysis unit 107 instructs the map generation unit 106 to (a) map storage instructions; (b) new map creation instructions; In addition to these instructions, (c) Initial position at the start of new map creation and instruction to read created map These notifications are executed. The above (c) is used when performing the map update process described with reference to FIG.
  • FIG. 39 is a flowchart for explaining the processing sequence executed by the elevator movement analysis unit 105 of the information processing apparatus 150 of the fifth embodiment shown in FIG.
  • the processing according to the flow described below is executed in accordance with a program stored in the storage unit of an information processing device in a mobile object such as a robot or an information processing device capable of communicating with a mobile object such as a robot. It is possible to For example, it can be performed as program execution processing by a processor such as a CPU having a program execution function. Processing in each step of the flow shown in FIG. 39 will be described below.
  • Step S501 First, in step S501, the elevator movement analysis unit 105 receives an IMU detection value, such as the acceleration of the robot, from the IMU (inertial measurement unit) 102 .
  • an IMU detection value such as the acceleration of the robot
  • Step S502 the elevator movement analysis unit 105 analyzes the acceleration input from the IMU (inertial measurement unit) 102 and determines whether vertical acceleration has been detected. If acceleration in the vertical direction is detected, the process proceeds to step S503. If not detected, the process returns to step S501 to repeat the processes from step S501 onward.
  • IMU intial measurement unit
  • Step S503 When vertical acceleration is detected in step S502, the elevator movement analysis unit 105 notifies the map structure analysis unit 107 of detection of robot movement by the elevator in step S503.
  • step S504 the elevator movement analysis unit 105 receives atmospheric pressure information from the atmospheric pressure sensor 101 and analyzes the inputted atmospheric pressure information.
  • step S505 the elevator movement analysis unit 105 determines whether movement between floors has been detected as an analysis result of the atmospheric pressure information input from the atmospheric pressure sensor 101.
  • This hierarchical movement determination process is a determination process based on whether or not the phenomenon described above with reference to FIG. 14 is detected. That is, the determination is made according to whether or not a disturbance of the initial atmospheric change that occurs according to the density of the space through which the elevator passes has been detected. If the disturbance of the first atmospheric change occurs, it is determined that the movement between floors has been performed.
  • step S506 the elevator movement analysis unit 105 notifies the map structure analysis unit 107 of the analysis information in step S505, that is, detection information of floor movement obtained from the analysis result of the atmospheric pressure information input from the atmospheric pressure sensor 101. do.
  • Step S507 the elevator movement analysis unit 105 receives an IMU detection value, such as the acceleration of the robot, from the IMU (inertial measurement unit) 102 .
  • step S508 the elevator movement analysis unit 105 analyzes the acceleration input from the IMU (inertial measurement unit) 102 and determines whether deceleration in the vertical direction has been detected. If vertical deceleration is detected, the process proceeds to step S509. If not detected, the process returns to step S507 and repeats the processing of step S507.
  • IMU intial measurement unit
  • Step S509 When the deceleration of the speed in the vertical direction is detected in step S508, the elevator movement analysis unit 105 notifies the map structure analysis unit 107 that the elevator has stopped moving the robot in step S509.
  • the map structure analysis unit 107 successively inputs information such as the start of elevator movement of the robot, floor movement, movement stop, etc., from the elevator movement analysis unit 105 .
  • information such as the start of elevator movement of the robot, floor movement, movement stop, etc.
  • the map structure analysis unit 107 for inputting these pieces of information will be described with reference to the flowchart shown in FIG. Processing after step S521 in the flow shown in FIG. 40 will be described in order.
  • Step S521 First, in step S521, the map structure analysis unit 107 determines whether or not an elevator movement detection notification has been input from the elevator movement analysis unit 105.
  • FIG. 1 the map structure analysis unit 107 determines whether or not an elevator movement detection notification has been input from the elevator movement analysis unit 105.
  • step S503 This is the notification input confirmation process in step S503 of the flow shown in FIG. If it is determined in step S521 that an elevator movement detection notification from the elevator movement analysis unit 105 has been input, the process proceeds to step S522.
  • Step S522 If the map structure analysis unit 107 determines in step S521 that an elevator movement detection notification has been received from the elevator movement analysis unit 105, the map structure analysis unit 107 executes the process of step S522.
  • map structure analysis unit 107 acquires the elevator position on the created map before the elevator movement, and executes a process of recording it in the map structure information storage unit 109 .
  • This information corresponds to the following information in "(2) Map transition node information" recorded in the map structure information storage unit 109 described above with reference to FIG. (2a) Source frame before map transition Target frame before map transition
  • step S523 the map structure analysis unit 107 outputs to the map generation unit 106 an instruction to save the created map before moving to the elevator.
  • the robot 10 creates a map of the 1st floor (1F), gets into the elevator from the 1st floor (1F), and starts moving in the elevator, the map structure analysis unit 107, in step S523, An instruction to save the map of the first floor (1F) created before moving to the elevator is output to the generation unit 106 .
  • the map generation unit 106 stores the created map in the map information storage unit 108 according to the instruction from the map structure analysis unit 107 . For example, a map of the first floor (1F) created before moving by the elevator is stored in the map information storage unit 108 .
  • Step S524 the map structure analysis unit 107 determines whether or not a floor movement notification by elevator has been input from the elevator movement analysis unit 105.
  • step S506 This is the notification input confirmation process in step S506 of the flow shown in FIG. If it is determined in step S524 that an elevator floor movement detection notification has been input from the elevator movement analysis unit 105, the process proceeds to step S525.
  • Step S525) If the map structure analysis unit 107 determines in step S524 that the floor movement detection notification by the elevator has been input from the elevator movement analysis unit 105, the map structure analysis unit 107 executes the process of step S525.
  • the map structure analysis unit 107 counts the number of floor movement detection notifications from the elevator movement analysis unit 105 in step S525 to determine the number of floors moved.
  • the number of floor movement detection notifications from the elevator movement analysis unit 105 is one, it is determined that the robot has moved one floor by the elevator. If the number of floor movement detection notifications from the elevator movement analysis unit 105 is two, it is determined that the robot has moved two floors by the elevator.
  • step S526 the map structure analysis unit 107 determines whether an elevator stop detection notification has been input from the elevator movement analysis unit 105 or not. If it is determined that the elevator stop detection notification has been input, the process proceeds to step S527. If the elevator stop detection notification has not been input, the process of step S525 is continued.
  • Step S527) If it is determined in step S526 that the map structure analysis unit 107 has received an elevator stop detection notification from the elevator movement analysis unit 105, the process of step S527 is executed.
  • the map structure analysis unit 107 stores the following information in the map structure information storage unit 109 in step S527.
  • "(3) Movement amount data between map transition nodes” "floor before elevator movement ⁇ floor after elevator movement”
  • This map transition node-to-node movement amount data is recorded. This is the data recorded in the map structure information storage unit 109 previously described with reference to FIG.
  • step S528 the map structure analysis unit 107 determines whether the floor on which the elevator has stopped is a new floor without a created map.
  • This determination process is executed by referring to the stored map data of the map information storage unit 108 and the stored data of the map structure information storage unit 109.
  • Step S529) If it is determined in step S528 that the floor on which the elevator has stopped is a new floor without a created map, the process proceeds to step S529.
  • step S529 the map structure analysis unit 107 notifies the map generation unit 106 of an instruction to create a new map on the floor after the elevator has moved.
  • the map generation unit 106 Upon receiving this notification from the map structure analysis unit 107, the map generation unit 106 starts creating a new map on the floor after the elevator has moved.
  • Step S530 the map structure analysis unit 107 acquires the elevator position in the created map after the elevator movement, and executes processing for recording it in the map structure information storage unit 109.
  • Map transition node information recorded in the map structure information storage unit 109 described above with reference to FIG. (2b) Source frame after map transition Target frame after map transition
  • the robot 10 creates a map of the 1st floor (1F), gets on the elevator from the 1st floor (1F), moves to the 2nd floor (2F) by the elevator, and creates a map of the 2nd floor (2F).
  • the above information (2b) becomes the following information.
  • Source frame after map transition 2F
  • Target frame after map transition EV_2F: (x, y, z, yaw)
  • Step S531 On the other hand, if it is determined in step S528 that the floor on which the elevator has stopped is the floor on which the created map exists, the process proceeds to step S531.
  • step S531 the map structure analysis unit 107 notifies the map generation unit 106 of the initial position and instructs it to update the created map.
  • the map generation unit 106 acquires the created map stored in the map information storage unit 108 in response to the input of this instruction, and performs update processing using the acquired created map.
  • either one of the 1F map_V1 and the 2F map_V1 may be used for updating.
  • Example 6 Regarding an embodiment in which the generated map is switched according to the inclination angle of the robot running surface
  • FIG. 41 shows an example of a running surface on which the robot 10 creates a map while running.
  • the running surface shown in FIG. 41 is a substantially horizontal surface from x1 to x2, and an inclined surface from x2 to x3, that is, a slope having an inclination angle. Furthermore, x3 to x4 are substantially horizontal planes.
  • the map to be generated is switched at the point where the inclination angle of the running surface is changed in this way.
  • the robot 10 generates the following three maps. (1) Horizontal map 1 from x1 to x2 (2) Map 2 for slopes from x2 to x3 (3) Horizontal map 3 from x3 to x4 Robot 10 generates these three maps.
  • the robot 10 When the generated map is switched, the robot 10 records map structure information indicating the relationship between the maps before and after switching in the map structure information storage unit.
  • map structure information An example of the map structure information that the robot 10 records in the map structure information storage unit will be described with reference to FIGS. 42 and 43.
  • FIG. 42 and 43 An example of the map structure information that the robot 10 records in the map structure information storage unit will be described with reference to FIGS. 42 and 43.
  • the robot 10 (p) complete generation of map 1 corresponding to horizontal planes from x1 to x2; (q) start generating map 2 corresponding to slopes from x2 to x3;
  • the map structure information A records the following information in the same manner as described above with reference to FIG. (1) Map transition node attributes (2) Map transition node information (3) Movement amount data between map transition nodes
  • Map transition node attribute is a field for recording attributes related to nodes before and after switching of a generated map (environmental map).
  • Rotation amount), and a roll corresponding to the rotation amount around the axis (X-axis) in the traveling direction of the robot are also recorded.
  • the movement amount data between map transition nodes is the amount of movement between two map transition nodes, that is, the map transition node 1e of map 1, the map transition node 2s of map 2, and the movement between these two map transition nodes. Quantitative data are recorded.
  • the amount of movement is recorded for each element of (x, y, z, roll, pitch, yaw). i.e. Movement amount in x direction (rel_x) Movement amount in y direction (rel_y) Movement amount in z direction (rel_z) Movement amount of roll angle (rel_roll) Movement amount of pitch angle (rel_pitch) Movement amount of yaw angle (rel_yaw) These element-by-element movement amounts are recorded.
  • the pitch of the map transition node 1e of the map 1 is 0deg
  • the pitch of the map transition node 2s of the map 2 is 30deg.
  • rel_pitch the amount of movement of the pitch angle
  • the robot 10 (r) completing the generation of map 2 corresponding to slopes from x2 to x3; (s) start generating map 3 corresponding to horizontal planes from x3 to x4;
  • An example of the map structure information B recorded in the map structure information storage unit 109 is shown.
  • Map transition node attributes (2) Map transition node information (3) Movement amount data between map transition nodes
  • Map transition node attribute is a field for recording attributes related to nodes before and after switching of a generated map (environmental map).
  • a source frame (map 2 or map 3), which is the acquisition destination information of the map data (actual map data);
  • Node information (x, y, z, roll, pitch, yaw) consisting of the position (x, y, z) of each node and the amount of rotation (roll, pitch, yaw) around three axes (XYZ axes)
  • the map transition node 2e of the map 2 is on an inclined surface.
  • the inclination is 30 degrees
  • the movement amount data between map transition nodes is the amount of movement between two map transition nodes, that is, the map transition node 2e of map 2, the map transition node 3s of map 3, and the movement between these two map transition nodes. Quantitative data are recorded.
  • the pitch of the map transition node 2e of the map 2 is 30deg, and the pitch of the map transition node 3s of the map 3 is 0deg.
  • FIG. 44 is a block diagram showing a configuration example of the information processing device 160 of the sixth embodiment.
  • This information processing device 160 is mounted inside the robot 10 that creates a map.
  • a part of the configuration other than the sensors and the like may be set in a device capable of communicating with the robot 10, such as a server.
  • the information processing device 160 includes an IMU (inertial measurement unit) 102, a wheel encoder 103, a LiDAR 104, an inclination analysis unit 161, a map generation unit 106, a map structure analysis unit 107, a map information storage unit 108, It has a map structure information storage unit 109 .
  • IMU intial measurement unit
  • Components other than the tilt angle analysis unit 161 that is, an IMU (inertial measurement unit) 102, a wheel encoder 103, a LiDAR 104, an elevator movement analysis unit 105, a map generation unit 106, a map structure analysis unit 107, a map information storage unit 108, and a map
  • the structural information storage unit 109 has the same configuration as the information processing apparatus 100 of the first embodiment described above with reference to FIG.
  • the IMU (inertial measurement unit) 102 detects the acceleration and angular velocity of the robot 10 and outputs the detected values of acceleration and angular velocity 112 to the tilt angle analysis unit 161 and the map generation unit 106 .
  • the tilt angle analysis unit 161 detects the tilt of the robot 10, that is, the tilt angle of the robot running surface, based on the acceleration and angular velocity 112 input from the IMU (inertial measurement unit) 102 .
  • the tilt of the robot 10 detected by the tilt angle analysis unit 161 that is, the tilt angle change detection information indicating the tilt angle of the robot running surface and the tilt angle information 162 are output to the map structure analysis unit 107 .
  • the map structure analysis unit 107 uses the tilt angle change detection information and the tilt angle information 162 input from the tilt angle analysis unit 161 to generate the map structure information 119 indicating the relationship between the two maps, and stores the map structure information. Stored in unit 109 . That is, the map structure information 119 as described above with reference to FIGS. 42 and 43 is generated and stored in the map structure information storage unit 109.
  • FIG. 109 the map structure information 119 as described above with reference to FIGS. 42 and 43 is generated and stored in the map structure information storage unit 109.
  • Map transition node attribute (2) Map transition node information (3) Movement amount data between map transition nodes
  • map transition node information for each of the two map transition nodes, a source frame (map 1 or map 2), which is information on where to obtain map data (actual map data); Node information (x, y, z, roll, pitch, yaw) consisting of the position (x, y, z) of each node and the amount of rotation (roll, pitch, yaw) around three axes (XYZ axes) These data are recorded.
  • Movement amount data between map transition nodes records movement amount data between two map transition nodes. for example, Movement amount in x direction (rel_x) Movement amount in y direction (rel_y) Movement amount in z direction (rel_z) Movement amount of roll angle (rel_roll) Movement amount of pitch angle (rel_pitch) Movement amount of yaw angle (rel_yaw) These element-by-element movement amounts are recorded.
  • IMU detection values ie, robot acceleration and angular velocity information
  • IMU intial measurement unit
  • Step S603 If the amount of pitch change per unit time is equal to or greater than the threshold, the tilt angle analysis unit 161 causes the map structure analysis unit 107 to notify the map structure analysis unit 107 of tilt angle change detection information and tilt angle information in step S603. conduct.
  • Step S621 First, in step S ⁇ b>621 , the map structure analysis unit 107 determines whether or not notification of tilt angle change detection information and tilt angle information has been input from the tilt angle analysis unit 161 .
  • step S603 This is the notification input confirmation process in step S603 of the flow shown in FIG. If it is determined in step S621 that notification of tilt angle change detection information and tilt angle information has been input from the tilt angle analysis unit 161, the process proceeds to step S622.
  • Step S622 If the map structure analysis unit 107 determines in step S621 that the tilt angle change detection information and the tilt angle information from the tilt angle analysis unit 161 have been input, it executes the process of step S622.
  • the map structure analysis unit 107 acquires the position and inclination (pitch) of the map transition node, which is the final node (map creation end node) in the created map before the inclination angle change, and stores the information in the map structure information storage unit. 109 is executed.
  • This information corresponds to the following information in "(2) map transition node information" recorded in the map structure information storage unit 109 described above with reference to FIG. (2a) Source frame before map transition Target frame before map transition
  • the map structure analysis unit 107 determines the position and inclination (pitch) of the map transition node, which is the final node (map creation end node) in the created map before the inclination angle change, and furthermore, the created map after the inclination angle change. Acquire the position and inclination (pitch) of the map transition node, which is the map creation start node, in the map structure information storage unit 109, (3) Execute the process of recording the amount of movement between map transition nodes.
  • the amount of movement is recorded for each element of (x, y, z, roll, pitch, yaw). i.e. Movement amount in x direction (rel_x) Movement amount in y direction (rel_y) Movement amount in z direction (rel_z) Movement amount of roll angle (rel_roll) Movement amount of pitch angle (rel_pitch) Movement amount of yaw angle (rel_yaw)
  • movement amount in x direction (rel_x) Movement amount in y direction (rel_y) Movement amount in z direction (rel_z) Movement amount of roll angle (rel_roll) Movement amount of pitch angle (rel_pitch) Movement amount of yaw angle (rel_yaw)
  • step S623 the map structure analysis unit 107 outputs to the map generation unit 106 an instruction to store the created map before the inclination angle is changed.
  • the map generation unit 106 stores the created map in the map information storage unit 108 according to the instruction from the map structure analysis unit 107 .
  • the horizontal floor map 1 created before the inclination angle is changed is stored in the map information storage unit 108 .
  • Step S624 the map structure analysis unit 107 notifies the map generation unit 106 of an instruction to create a new map on the floor after the tilt angle has been changed.
  • the map generation unit 106 Upon receiving this notification from the map structure analysis unit 107, the map generation unit 106 starts creating a new map on the floor after the tilt angle has been changed.
  • step S625 the map structure analysis unit 107 acquires the position and inclination angle of the map transition node, which is the map creation start node in the created map after the inclination angle change, and records them in the map structure information storage unit 109. to run.
  • This information corresponds to the following information in "(2) map transition node information" recorded in the map structure information storage unit 109 described above with reference to FIG. (2b) Source frame after map transition Target frame after map transition
  • the sixth embodiment is an embodiment in which the generated map is switched according to the inclination angle of the robot's running surface. Not only the position of the map transition node to be switched but also the pitch corresponding to the inclination angle of the map transition node are recorded. Furthermore, it is configured to record the tilt angle change amount (rel
  • Example 7 Regarding an embodiment in which the generated map is switched according to the availability of GPS signals.
  • FIG. 48 shows an example of a running surface on which the robot 10 creates a map while running.
  • the robot 10 runs in the section x1 to x2 inside the building. In the building, it is difficult to receive the transmission signal from the GPS satellite 40, and the vehicle runs without using the GPS position information.
  • the robot 10 leaves the building at x2 and travels in the section from x2 to x3.
  • the section from x2 to x3 is outdoors, and the transmission signal from the GPS satellite 40 can be received. Therefore, the robot 10 runs using the GPS position information in this outdoor section x2-x3.
  • the robot 10 After that, the robot 10 re-enters another building at x3 and travels within the building from x3 to x4. It is difficult to receive the transmission signal from the GPS satellite 40 in this section from x3 to x4. Therefore, the robot 10 runs without using the GPS position information in this intra-building section x3-x4.
  • the map to be generated is switched at the point where the availability of GPS position information is changed as described above.
  • the robot 10 generates the following three maps. (1) Map 1 of GPS location information unusable section (GPS signal strength low section) from section x1 to x2 (2) Map 2 of the section x2 to x3 where GPS location information can be used (high GPS signal strength section) (3) Map 3 of GPS location information unusable section (GPS signal strength low section) from section x3 to x4 Robot 10 generates these three maps.
  • the robot 10 When the generated map is switched in this way, the robot 10 records the GPS position information in the map structure information storage unit 109 if the GPS position information is available in the map structure information storage unit.
  • FIG. 50 shows a robot travel route similar to FIGS. 48 and 49.
  • Map 1 of GPS location information unusable section (GPS signal strength low section) from section x1 to x2 (2) Map 2 of the section x2 to x3 where GPS location information can be used (high GPS signal strength section) (3) Map 3 of GPS location information unusable section (GPS signal strength low section) from section x3 to x4 Robot 10 generates these three maps.
  • the robot 10 When executing such map creation processing, the robot 10 stores the GPS position information in the map structure information storage unit 109 if the GPS position information is available at the timing of switching the created map from map 1 to map 2, for example. Record.
  • map transition node 1e which is the final node (map creation end node) of map 1 shown in FIG.
  • the GPS position information is recorded in the map structure information storage unit 109 for the map transition node 2s, which is the map creation start node of the map 2, which is the GPS position information usable section.
  • the movement from the map transition node 2e, which is the final node (map creation end node) of the map 2 shown in FIG. Switching from the section to the unavailable section occurs.
  • the GPS position information is recorded in the map structure information storage unit 109 for the map transition node 2e, which is the final node (map creation end node) of the map 2, which is the GPS position information usable section.
  • the GPS position information is recorded in the map structure information storage unit 109 as node information.
  • FIG. 51 is a block diagram showing a configuration example of the information processing device 170 of the seventh embodiment.
  • This information processing device 170 is mounted inside the robot 10 that creates a map.
  • a part of the configuration other than the sensors and the like may be set in a device capable of communicating with the robot 10, such as a server.
  • the information processing device 170 includes a GPS signal receiving unit 171, a magnetic sensor 172, a GPS switching determination unit 173, an IMU (inertial measurement unit) 102, a wheel encoder 103, a LiDAR 104, a map generation unit 106, a map structure It has an analysis unit 107 , a map information storage unit 108 , and a map structure information storage unit 109 .
  • Components other than the GPS signal reception unit 171, the magnetic sensor 172, and the GPS switching determination unit 173, namely, the IMU (inertial measurement unit) 102, the wheel encoder 103, the LiDAR 104, the map generation unit 106, the map structure analysis unit 107, and the map information storage A unit 108 and a map structure information storage unit 109 have the same configuration as the information processing apparatus 100 of the first embodiment described above with reference to FIG.
  • a GPS signal 175 received by the GPS signal receiver 171 and posture information 176 indicating the posture of the robot, which is detected by the magnetic sensor 172 , are input to the GPS switching determination unit 173 .
  • the GPS switching determination unit 173 uses a GPS reception signal 175 input from the GPS signal reception unit 171 and attitude information 176 indicating the attitude of the robot input from the magnetic sensor 172 to perform self-position estimation using GPS position information and autonomous robot movement. Determine whether or not to run.
  • a mode in which self-position estimation and autonomous travel are performed using GPS position information is referred to as "outdoor mode", and a mode in which self-position estimation and autonomous travel are not performed using GPS position information is referred to as “indoor mode”.
  • indoor mode in which self-position estimation using GPS position information and autonomous driving are not performed, self-position estimation and autonomous driving are performed based only on SLAM processing.
  • the GPS switching determination unit 173 uses a GPS reception signal 175 input from the GPS signal reception unit 171 and attitude information 176 indicating the attitude of the robot input from the magnetic sensor 172 to perform self-position estimation using GPS position information and autonomous robot movement. It is determined whether or not to run, and the mode is determined according to the determination result.
  • the GPS switching determination unit 173 outputs a mode switching instruction and GPS position information according to the determined mode information to the map generation unit 106 and the map structure analysis unit 107 . This is the “mode switching instruction & GPS position information 177” shown in FIG.
  • the map generation unit 106 refers to the “mode switching instruction & GPS position information 177 ” input from the GPS switching determination unit 173 to switch the mode. That is, mode switching is performed between an “outdoor mode” in which self-position estimation and autonomous driving are performed using GPS position information, and an “indoor mode” in which self-position estimation and autonomous driving are not performed using GPS position information.
  • the map structure analysis unit 107 refers to the "mode switching instruction & GPS position information 177" input from the GPS switching determination unit 173, and generates and records data to be recorded in the map structure information storage unit 109.
  • FIG. 52 An example of map structure information recorded by the robot 10 in the map structure information storage unit will be described with reference to FIGS. 52 and 53.
  • FIG. 52 An example of map structure information recorded by the robot 10 in the map structure information storage unit will be described with reference to FIGS. 52 and 53.
  • the robot 10 (1) complete the generation of the map 1 of the GPS position information unusable section (GPS signal strength low section) of the section x1 to x2, (2) Start generating map 2 for the GPS location information available section (GPS signal strength high section) of section x2 to x3, An example of the map structure information recorded in the map structure information storage unit 109 is shown.
  • map structure information As for the map structure information, the following information is recorded in the same manner as described with reference to FIG. (1) Map transition node attributes (2) Map transition node information (3) Movement amount data between map transition nodes
  • Map transition node attribute is a field for recording attributes related to nodes before and after switching of a generated map (environmental map).
  • the target frame after map transition 2 s is a node for which GPS location information is available, so GPS location information is recorded.
  • the movement amount data between map transition nodes is the amount of movement between two map transition nodes, that is, the map transition node 1e of map 1, the map transition node 2s of map 2, and the movement between these two map transition nodes. Quantitative data are recorded.
  • the amount of movement is recorded for each element of (x, y, z, yaw). i.e. Movement amount in x direction (rel_x) Movement amount in y direction (rel_y) Movement amount in z direction (rel_z) Movement amount of yaw angle (rel_yaw) These element-by-element movement amounts are recorded.
  • the robot 10 (1) complete the generation of the map 2 of the GPS position information available section (GPS signal strength high section) of the section x2 to x3, (2) Start generating map 3 for the section x3 to x4 where GPS location information is not available (low GPS signal strength section), An example of the map structure information recorded in the map structure information storage unit 109 is shown.
  • map structure information As for the map structure information, the following information is recorded in the same manner as described with reference to FIG. (1) Map transition node attributes (2) Map transition node information (3) Movement amount data between map transition nodes
  • Map transition node attribute is a field for recording attributes related to nodes before and after switching of a generated map (environmental map).
  • the movement amount data between map transition nodes is the amount of movement between two map transition nodes, that is, the map transition node 2e of map 2, the map transition node 3s of map 3, and the movement between these two map transition nodes. Quantitative data are recorded.
  • the amount of movement is recorded for each element of (x, y, z, yaw). i.e. Movement amount in x direction (rel_x) Movement amount in y direction (rel_y) Movement amount in z direction (rel_z) Movement amount of yaw angle (rel_yaw) These element-by-element movement amounts are recorded.
  • Step S701 the GPS switching determination unit 173 analyzes the strength of the GPS signal received by the GPS signal reception unit 171 in step S701.
  • Step S702 the GPS switching determination section 173 determines whether or not the strength of the GPS signal received by the GPS signal reception section 171 is equal to or greater than a predetermined specified threshold.
  • step S703 If the GPS signal strength is greater than or equal to the specified threshold, the process proceeds to step S704.
  • Step S703 If the GPS signal strength is greater than or equal to the specified threshold, the GPS switching determination section 173 executes the following process in step S703.
  • the GPS switching determination unit 173 transmits the GPS position information based on the "outdoor mode" setting request signal for self-position estimation and autonomous driving using the GPS position information and the received signal from the GPS satellite to the map generation unit 106 and the map. Output to the structural analysis unit 107 .
  • Step S704 On the other hand, when the GPS signal strength is not equal to or greater than the specified threshold value, the GPS switching determination section 173 executes the following process in step S704.
  • GPS switching determination section 173 outputs to map generation section 106 and map structure analysis section 107 an “indoor mode” setting request signal in which self-position estimation using GPS position information and autonomous driving are not performed. At the same time, the output of the GPS positional information based on the signals received from the GPS satellites to the map generation unit 106 and the map structure analysis unit 107 is stopped.
  • Step S721 First, in step S721, the map structure analysis unit 107 determines whether or not the GPS switching determination unit 173 has input a “setting change request from indoor mode to outdoor mode”. If so, the process from step S722 onwards is executed.
  • Step S722 If the map structure analysis unit 107 determines in step S721 that the GPS switching determination unit 173 has input a "request to change the setting from the indoor mode to the outdoor mode", the map structure analysis unit 107 executes the process of step S722.
  • the map structure analysis unit 107 acquires the position of the map transition node, which is the final node (map creation end node) of the created map in the GPS disabled state before the map transition, and records it in the map structure information storage unit 109. Execute the process.
  • This information corresponds to the following information in "(2) map transition node information" recorded in the map structure information storage unit 109 described above with reference to FIG. (2a)
  • Source frame before map transition map 1 (GPS signal unavailable)
  • Target frame before map transition 1e: (x, y, z, yaw)
  • step S723 the map structure analysis unit 107 outputs to the map generation unit 106 an instruction to store the created map before the map transition.
  • the map generation unit 106 stores the created map in the map information storage unit 108 according to the instruction from the map structure analysis unit 107 .
  • the map 1 created in the GPS unavailable state is stored in the map information storage unit 108 .
  • step S724 the map structure analysis unit 107 notifies the map generation unit 106 of an instruction to create a new map under the GPS enabled state.
  • the map generation unit 106 Upon receiving this notification from the map structure analysis unit 107, the map generation unit 106 starts creating a new map in the GPS enabled state.
  • step S725 the map structure analysis unit 107 acquires the position of the map transition node, which is the map creation start node in the created map in the GPS enabled state, and executes processing for recording it in the map structure information storage unit 109. . GPS position information is also recorded.
  • map transition node information recorded in the map structure information storage unit 109 described above with reference to FIG. (2b)
  • Source frame map 2 after map transition GPS signal available
  • Target frame before map transition 2s: (x, y, z, yaw) & node 2s (latitude, longitude)
  • the map structure analysis unit 107 determines the amount of movement between nodes before and after the map transition, that is, the position of the map transition node, which is the final node (map creation end node) in the created map in the GPS unavailable state, and the orientation (yaw direction) of the robot. ) and the position of the map transition node, which is the map creation start node in the created map in the GPS enabled state, and the orientation (yaw) of the robot are acquired, and stored in the map structure information storage unit 109, (3) Execute the process of recording as the amount of movement between map transition nodes.
  • the amount of movement is recorded for each element of (x, y, z, yaw). i.e. Movement amount in x direction (rel_x) Movement amount in y direction (rel_y) Movement amount in z direction (rel_z) Movement amount of yaw angle (rel_yaw) These element-by-element movement amounts are recorded.
  • Step S741 First, in step S ⁇ b>741 , the map structure analysis unit 107 determines whether or not a “request for setting change from outdoor mode to indoor mode” has been input from the GPS switching determination unit 173 . If so, the process from step S742 onwards is executed.
  • Step S742 If the map structure analysis unit 107 determines in step S741 that the GPS switching determination unit 173 has input a "request to change the setting from the outdoor mode to the indoor mode", the map structure analysis unit 107 executes the process of step S742.
  • the map structure analysis unit 107 acquires the position of the map transition node, which is the final node (map creation end node) of the created map in the GPS enabled state before the map transition, and records it in the map structure information storage unit 109. Execute the process. GPS position information is also recorded.
  • map transition node information recorded in the map structure information storage unit 109 described above with reference to FIG. (2a)
  • Source frame before map transition map 2 (GPS signal available)
  • Target frame before map transition 2e: (x, y, z, yaw) & node 2e (latitude, longitude)
  • step S743 the map structure analysis unit 107 outputs to the map generation unit 106 an instruction to store the created map before the map transition.
  • the map generation unit 106 stores the created map in the map information storage unit 108 according to the instruction from the map structure analysis unit 107 .
  • the map 2 created in the GPS usable state is stored in the map information storage unit 108 .
  • step S744 the map structure analysis unit 107 notifies the map generation unit 106 of an instruction to create a new map under the GPS unavailable state.
  • the map generation unit 106 Upon receiving this notification from the map structure analysis unit 107, the map generation unit 106 starts creating a new map in the GPS disabled state.
  • step S745 the map structure analysis unit 107 acquires the position of the map transition node, which is the map creation start node in the map created in the GPS disabled state, and executes processing for recording it in the map structure information storage unit 109. . Note that the GPS position information cannot be obtained and is therefore not recorded.
  • This information corresponds to the following information in "(2) map transition node information" recorded in the map structure information storage unit 109 described above with reference to FIG. (2b)
  • Source frame map 3 after map transition GPS signal available
  • Target frame before map transition 3s: (x, y, z, yaw)
  • the map structure analysis unit 107 determines the amount of movement between nodes before and after the map transition, that is, the position of the map transition node, which is the final node (map creation end node) in the created map in the GPS enabled state, and the orientation (yaw) of the robot. and the difference between the position of the map transition node, which is the map creation start node, and the direction (yaw) of the robot in the created map in the GPS unavailable state, and store it in the map structure information storage unit 109, (3) Execute the process of recording as the amount of movement between map transition nodes.
  • the amount of movement is recorded for each element of (x, y, z, yaw). i.e. Movement amount in x direction (rel_x) Movement amount in y direction (rel_y) Movement amount in z direction (rel_z) Movement amount of yaw angle (rel_yaw) These element-by-element movement amounts are recorded.
  • the map to be generated is switched at a point where the availability of GPS location information is changed. Furthermore, when GPS position information is available, the GPS position information is also recorded as information indicating node positions in the map information storage unit and the map structure information storage unit.
  • the information processing device that performs the above-described processing was described as being installed in a robot that performs environment map creation processing such as SLAM processing.
  • the processing device may be configured to be installed in a server or the like that can communicate with the robot.
  • sensors for acquiring information around the robot are attached to the robot, the sensor acquisition information is transmitted to the server via the communication network, and the server performs processing using the sensor acquisition information according to the above-described embodiment. may be configured.
  • an information processing system 200 in which the robot 10 and the robot management server 210 are connected via a communication network.
  • the robot 10 transmits sensor acquisition information to the robot management server 210 .
  • the robot management server 210 uses sensor acquisition information received from the robot 10 to perform data processing according to the above-described embodiments.
  • a configuration using such an information processing system 200 may be employed.
  • the sensors of the robot 10 include the atmospheric pressure sensor 101, the IMU (inertial measurement unit) 102, the wheel encoder 103, the LiDAR 104, the camera 121, the communication unit 141, the GPS signal receiving unit 171, and the magnetic sensor 172 described in each embodiment. included.
  • the UI unit 131 the one attached to the robot 10 may be used, or a user terminal other than the robot, such as a smartphone, may be used. It is preferable that the user terminal, the robot 10, and the robot management server 210 are set to be able to communicate with each other.
  • the information processing device is installed inside the robot 10 .
  • it may be configured in the server as described above.
  • a CPU (Central Processing Unit) 301 functions as a data processing section that executes various processes according to programs stored in a ROM (Read Only Memory) 302 or a storage section 308 . For example, the process according to the sequence described in the above embodiment is executed.
  • a RAM (Random Access Memory) 303 stores programs and data executed by the CPU 301 . These CPU 301 , ROM 302 and RAM 303 are interconnected by a bus 304 .
  • the CPU 301 is connected to an input/output interface 305 via a bus 304.
  • the input/output interface 305 includes various switches, a keyboard, a touch panel, a mouse, a microphone, and status data of various sensors 321 such as a user input unit, a camera, and LiDAR GPS.
  • An input unit 306 such as an acquisition unit and an output unit 307 such as a display and a speaker are connected.
  • the output unit 307 also outputs driving information to the driving unit 322 of the mobile device (robot).
  • the CPU 301 receives commands, situation data, and the like input from the input unit 306 , executes various processes, and outputs processing results to the output unit 307 , for example.
  • a storage unit 308 connected to the input/output interface 305 includes, for example, a hard disk, and stores programs executed by the CPU 301 and various data.
  • a communication unit 309 functions as a transmission/reception unit for data communication via a network such as the Internet or a local area network, and communicates with an external device.
  • a GPU Graphics Processing Unit
  • a drive 310 connected to the input/output interface 305 drives a removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory such as a memory card to record or read data.
  • a removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory such as a memory card to record or read data.
  • the technique disclosed in this specification can take the following configurations.
  • a map generation unit that generates a map using sensor acquisition information from a sensor attached to a mobile object; a map structure analysis unit that generates map structure information indicating relationships between the plurality of maps generated by the map generation unit; The map structure analysis unit When the map generating unit finishes generating the first map and generates a new second map, The node position of the first map transition node that is the map creation end node of the first map, the node position of the second map transition node that is the map creation start node of the second map, and the node position of these two map transition nodes as map transition node information in a map structure information storage unit.
  • the map generating unit The generated map is stored in the map information storage unit, and node position information used for map generation is recorded in the map information storage unit,
  • the node positions of the two map transition nodes recorded in the map structure information storage unit by the map structure analysis unit are data at the same positions as the node positions recorded in the map information storage unit by the map generation unit.
  • the map structure analysis unit When performing processing for recording the node positions of the two map transition nodes as map transition node information in the map structure information storage unit, The information processing apparatus according to (1) or (2), wherein the map generation unit is instructed to store the first map in the map information storage unit and to start generating the second map.
  • the map structure analysis unit Further, as the map transition node information, (1) recording yaw angle information indicating the direction of the moving object at the first map transition node and yaw angle information indicating the direction of the moving object at the second map transition node in the map structure information storage unit; (3) The information processing device according to any one of the above.
  • the map structure analysis unit any one of (1) to (5) of recording, in the map structure information storage unit, a map transition node attribute indicating that an elevator has been moved between the first map transition node and the second map transition node; The information processing device described.
  • the map structure analysis unit Information according to any one of (1) to (6), wherein a map transition node attribute indicating that the first map transition node and the second map transition node have different running surface inclinations is recorded in the map structure information storage unit. processing equipment.
  • the map structure analysis unit The method according to any one of (1) to (7), wherein a map transition node attribute indicating that the first map transition node and the second map transition node have different GPS availability states is recorded in the map structure information storage unit.
  • Information processing equipment Information processing equipment.
  • the map structure analysis unit The information processing apparatus according to any one of (1) to (8), wherein the amount of movement of the moving object between the first map transition node and the second map transition node is recorded in the map structure information storage unit.
  • the map structure analysis unit As the amount of movement of the moving body between the first map transition node and the second map transition node, inter-node change data corresponding to the xyz axes of the moving body, or changes in orientation and inclination of the moving body.
  • the information processing apparatus according to any one of (1) to (9), wherein at least one of the inter-node angle change data shown and the data is recorded in the map structure information storage unit.
  • the information processing device further includes: having an elevator movement analysis unit, The elevator movement analysis unit Analyzing whether or not the moving body has boarded the elevator based on the sensor acquisition information of the sensor attached to the moving body, and notifying the map structure analysis unit of the analysis result according to any one of (1) to (10). information processing equipment.
  • the map structure analysis unit When analysis information indicating that the moving object has entered the elevator is input from the elevator movement analysis unit, outputting an instruction to the map generation unit to store the generated map in the map information storage unit;
  • analysis information indicating that the elevator in which the moving object has boarded has stopped is input from the elevator movement analysis unit,
  • the information processing apparatus according to (11), which outputs an instruction to create a new map to the map generation unit.
  • the elevator movement analysis unit (11) or (12) analyzing the number of floors moved by the elevator boarded by the moving body based on the acquired information from the atmospheric pressure sensor attached to the moving body, and notifying the map structure analysis unit of the analysis result;
  • the information processing device according to .
  • the elevator movement analysis unit According to (11) or (12), analyzing the moving floor number of the elevator boarded by the moving object based on the photographed image of the camera, which is the sensor attached to the moving object, and notifying the map structure analysis unit of the analysis result. information processing equipment.
  • the elevator movement analysis unit The communication unit attached to the moving object inputs the received information received from the transmitting unit installed at the elevator doorway, analyzes the elevator getting on and off of the moving object and the number of elevator getting on and off floors, and uses the analysis result as the map structure analysis.
  • the information processing apparatus according to (11) or (12), which notifies the department.
  • the map structure analysis unit The information processing apparatus according to any one of (1) to (10), which analyzes user input information from a user observing the moving object and records an analysis result in the map structure information storage unit.
  • the information processing device further includes: having a tilt angle analysis unit, The tilt angle analysis unit When it is detected that the moving body has moved to a running surface with a different tilt angle based on the sensor-acquired information of the sensor attached to the moving body, the tilt angle change detection result including the tilt angle information is sent to the map structure analysis unit.
  • the information processing apparatus according to any one of (1) to (10), which notifies.
  • the information processing device further includes: Having a GPS switching determination unit, The GPS switching determination unit, Based on the signal received by the GPS signal receiving unit attached to the mobile object, it is determined whether or not to use the GPS position information for confirming the position of the mobile object, and a mode switching instruction and acquisition are performed according to the determination result.
  • the information processing apparatus according to any one of (1) to (10), which outputs the GPS position information thus obtained to the map structure analysis unit.
  • An information processing method executed in an information processing device a map generation step in which the map generation unit generates a map using sensor acquisition information from a sensor attached to the mobile object; a map structure information recording step in which the map structure analysis unit generates map structure information indicating the relationship between the plurality of maps generated by the map generation unit and records the map structure information in the map structure information storage unit;
  • the map structure analysis unit in the map structure information recording step, When the map generating unit finishes generating the first map and generates a new second map, The node position of the first map transition node that is the map creation end node of the first map, the node position of the second map transition node that is the map creation start node of the second map, and the node position of these two map transition nodes as map transition node information in the map structure information storage unit.
  • a program for executing information processing in an information processing device a map generation step of causing a map generation unit to generate a map using sensor acquisition information from a sensor attached to a mobile object; causing a map structure analysis unit to execute a map structure information recording step of generating map structure information indicating relationships between a plurality of maps generated by the map generation unit and recording the map structure information in a map structure information storage unit;
  • the program in the map structure information recording step that is executed by the map structure analysis unit, When the map generating unit finishes generating the first map and generates a new second map, The node position of the first map transition node that is the map creation end node of the first map, the node position of the second map transition node that is the map creation start node of the second map, and the node position of these two map transition nodes as map transition node information in the map structure information storage unit.
  • a program recording the processing sequence is installed in the memory of a computer built into dedicated hardware and executed, or the program is loaded into a general-purpose computer capable of executing various processing. It can be installed and run.
  • the program can be pre-recorded on a recording medium.
  • the program can be received via a network such as a LAN (Local Area Network) or the Internet and installed in a recording medium such as an internal hard disk.
  • a system is a logical collective configuration of a plurality of devices, and the devices of each configuration are not limited to being in the same housing.
  • map generation unit that generates a map using sensor-obtained information from a sensor attached to a moving body, and map structure information that indicates the relationship between a plurality of maps generated by the map generation unit is generated. It has a map structure analysis part.
  • the map structure analysis unit determines the node position of the first map transition node, which is the map generation end node of the first map, and the first map transition node.
  • the node position of the second map transition node which is the map creation start node of the second map, is recorded in the map structure information storage unit as map transition node information.
  • the map generating unit is instructed to store the first map in the map information storage unit and to start generating the second map.
  • map information storage unit 40 map structure information storage unit 100 information processing device 101 atmospheric pressure sensor 102 IMU (inertial measurement unit) 103 wheel encoder 104 LiDAR 105 Elevator movement analysis unit 106 Map generation unit 107 Map structure analysis unit 108 Map information storage unit 109 Map structure information storage unit 121 Camera 131 UI unit 141 Communication unit 145 Data transmission unit 161 Inclination analysis unit 171 GPS signal reception unit 172 Magnetic sensor 173 GPS switching determination unit 200 information processing system 210 robot management server 301 CPU 302 ROMs 303 RAM 304 bus 305 input/output interface 306 input unit 307 output unit 308 storage unit 309 communication unit 310 drive 311 removable media 321 sensor 322 drive unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
PCT/JP2022/001992 2021-03-11 2022-01-20 情報処理装置、および情報処理方法、並びにプログラム Ceased WO2022190662A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2023505177A JPWO2022190662A1 (https=) 2021-03-11 2022-01-20

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021039592 2021-03-11
JP2021-039592 2021-03-11

Publications (1)

Publication Number Publication Date
WO2022190662A1 true WO2022190662A1 (ja) 2022-09-15

Family

ID=83226623

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/001992 Ceased WO2022190662A1 (ja) 2021-03-11 2022-01-20 情報処理装置、および情報処理方法、並びにプログラム

Country Status (2)

Country Link
JP (1) JPWO2022190662A1 (https=)
WO (1) WO2022190662A1 (https=)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07229728A (ja) * 1994-02-22 1995-08-29 Nikon Corp 路面検出装置
JP2000284830A (ja) * 1999-03-30 2000-10-13 Komatsu Ltd 無人走行制御装置
JP2010015194A (ja) * 2008-06-30 2010-01-21 Ihi Corp 自律移動ロボット装置及び自律移動ロボット装置の制御方法
US20100223031A1 (en) * 2009-02-27 2010-09-02 Regis Vincent Method and apparatus for mapping of multiple-floor structures
JP2013190265A (ja) * 2012-03-13 2013-09-26 Hitachi Ltd 移動量推定システム、移動量推定方法、移動端末
JP2013222446A (ja) * 2012-04-19 2013-10-28 Toyota Central R&D Labs Inc 障害物検出装置及びそれを備えた移動体
JP2017044470A (ja) * 2014-01-15 2017-03-02 旭化成株式会社 情報処理装置、マップマッチング装置、情報処理方法、およびプログラム
JP2019023848A (ja) * 2017-05-01 2019-02-14 サビオーク インコーポレイテッドＳａｖｉｏｋｅ， Ｉｎｃ． ロボットによる自動マッピング用コンピュータシステム及び方法
CN110696004A (zh) * 2019-09-18 2020-01-17 五邑大学 一种智能导航机器人及其控制方法
US20200116493A1 (en) * 2018-10-11 2020-04-16 Zillow Group, Inc. Automated Mapping Information Generation From Inter-Connected Images

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07229728A (ja) * 1994-02-22 1995-08-29 Nikon Corp 路面検出装置
JP2000284830A (ja) * 1999-03-30 2000-10-13 Komatsu Ltd 無人走行制御装置
JP2010015194A (ja) * 2008-06-30 2010-01-21 Ihi Corp 自律移動ロボット装置及び自律移動ロボット装置の制御方法
US20100223031A1 (en) * 2009-02-27 2010-09-02 Regis Vincent Method and apparatus for mapping of multiple-floor structures
JP2013190265A (ja) * 2012-03-13 2013-09-26 Hitachi Ltd 移動量推定システム、移動量推定方法、移動端末
JP2013222446A (ja) * 2012-04-19 2013-10-28 Toyota Central R&D Labs Inc 障害物検出装置及びそれを備えた移動体
JP2017044470A (ja) * 2014-01-15 2017-03-02 旭化成株式会社 情報処理装置、マップマッチング装置、情報処理方法、およびプログラム
JP2019023848A (ja) * 2017-05-01 2019-02-14 サビオーク インコーポレイテッドＳａｖｉｏｋｅ， Ｉｎｃ． ロボットによる自動マッピング用コンピュータシステム及び方法
US20200116493A1 (en) * 2018-10-11 2020-04-16 Zillow Group, Inc. Automated Mapping Information Generation From Inter-Connected Images
CN110696004A (zh) * 2019-09-18 2020-01-17 五邑大学 一种智能导航机器人及其控制方法

Also Published As

Publication number Publication date
JPWO2022190662A1 (https=) 2022-09-15

Similar Documents

Publication Publication Date Title
EP3715785B1 (en) Slam assisted ins
TWI827649B (zh) 用於ｖｓｌａｍ比例估計的設備、系統和方法
Borenstein et al. Histogramic in-motion mapping for mobile robot obstacle avoidance
WO2023000294A1 (zh) 位姿估计方法、激光雷达-惯性里程计、可移动平台及存储介质
Mulgaonkar et al. The tiercel: A novel autonomous micro aerial vehicle that can map the environment by flying into obstacles
Wang et al. A mono-camera and scanning laser range finder based UAV indoor navigation system
JP2020079997A (ja) 情報処理装置、情報処理方法、及びプログラム
RU195749U1 (ru) Интеллектуальная система технического зрения беспилотного летательного аппарата для решения задач навигации, построения трехмерной карты окружающего пространства и препятствий и автономного патрулирования
Zhang et al. An indoor navigation aid for the visually impaired
WO2023219058A1 (ja) 情報処理方法、情報処理装置及び情報処理システム
WO2022238189A1 (en) Method of acquiring sensor data on a construction site, construction robot system, computer program product, and training method
JP7722503B2 (ja) 情報処理方法、情報処理装置およびプログラム
EP3682306B1 (en) Action plan generation when the own position is unknown
EP3992747B1 (en) Mobile body, control method, and program
US20180364046A1 (en) Apparatus and method for determining a vehicle position in a fixed-traffic-node coordinate system
JP2019197241A (ja) 通行人追従型の移動ロボットの誘導
Muller et al. A probabilistic sonar sensor model for robust localization of a small-size blimp in indoor environments using a particle filter
JP2023551948A (ja) シャフトに沿ってドローンを制御するための方法
KR102252295B1 (ko) 실내 토폴로지맵 생성 방법 및 이를 위한 자율주행 로봇
JP7351609B2 (ja) 経路探索装置及びプログラム
KR20240019606A (ko) Uwb 및 관성 센서 기반의 로봇 위치 추정 장치 및 방법
WO2022190662A1 (ja) 情報処理装置、および情報処理方法、並びにプログラム
Poomarin et al. Automatic docking with obstacle avoidance of a differential wheel mobile robot
CN113632029B (zh) 信息处理装置、程序及信息处理方法
CN114694018B (zh) 一种对象信息检测方法、装置和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22766629

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023505177

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22766629

Country of ref document: EP

Kind code of ref document: A1