JP2020149186A - Position attitude estimation device, learning device, mobile robot, position attitude estimation method, and learning method - Google Patents

Abstract

To provide a position attitude estimation device that can accurately acquire a position attitude at a construction site while suppressing the increase of weight.SOLUTION: A position attitude estimation device includes: an image acquisition unit that acquires image data photographed in any position attitude at a construction site; and a position attitude information calculation unit that calculates position attitude information corresponding to the image data, using a learned model which has learned the relation between either of the position attitudes at the construction site and an image viewed from a first-person eye point from the position attitude among in three-dimensional model data of a construction object at the construction site.SELECTED DRAWING: Figure 1

Description

The present invention relates to a position / posture estimation device, a learning device, a mobile robot, a position / posture estimation method, and a learning method.

Self-position posture estimation such as SLAM (Simultaneus Localization and Mapping) technology using LIDAR (Light Detection and Ranger) or Visual SLAM technology using a camera to improve the productivity of construction sites. And site management methods are being utilized.
Since LIDAR can directly measure the distance, it has an advantage that the shape accuracy is high as compared with the photogrammetry (Structure from Motion) technique.
For example, by measuring the shape of the construction object in the construction site every day or every few days with LIDAR, to what stage the construction object is constructed at the time of measurement and how the construction status is. It is possible to grasp whether it is in such a state. As a result, the progress of construction can be managed, and the finished shape can be managed by estimating the finished shape based on the construction status.
There is a position / posture estimation device that applies the self-position / posture estimation technique to, for example, the estimation of the current position and posture of an automatic traveling robot used in a construction site (for example, Patent Document 1).

JP-A-2018-164966

However, LIDAR cannot be mounted on a lightweight robot (for example, a small robot or a drone) because the equipment is generally expensive and heavy.
In addition, LIDAR has a drawback that it does not include RGB information of the acquired point cloud.

On the other hand, Visual SLAM contains RGB information, but has a drawback that the measurement accuracy is generally lower than that of LIDAR because it is a survey by parallax.

When a mobile robot is to measure the shape of a construction object in order to manage the construction status and the finished shape, it is necessary to accurately estimate its own position and orientation. That is, when the estimation accuracy of the self-position posture is low, the accuracy of the finished product management is also lowered. Further, in order for the mobile robot to move, it is desirable that the weight of the machine body does not increase.

The present invention has been made in view of such a problem, and is a position / posture estimation device, a learning device, a mobile robot, and a position capable of accurately obtaining a position / posture at a construction site while suppressing an increase in weight. The purpose is to provide a posture estimation method and a learning method.

In order to solve the above-mentioned problems, one aspect of the present invention is a three-dimensional image acquisition unit that acquires image data taken at any position or posture in a construction site, and a construction object at the construction site. Positions for obtaining position / orientation information corresponding to the image data using a learned model that has learned the relationship between the position / orientation of any of the construction sites and the image when viewed from the first-person viewpoint from the position / orientation in the model data. It is a position / posture estimation device having a posture information calculation unit.

Further, one aspect of the present invention includes a position / orientation information acquisition unit that acquires position / orientation information representing the position / orientation of any of the construction sites in the three-dimensional model data of the construction object at the construction site, and the tertiary from the first-person viewpoint. It is a learning device having an image acquisition unit that acquires an image when viewed from the position / orientation in the original model data, and a learning unit that learns a condition indicating a relationship between the position / orientation information and the image.

Further, one aspect of the present invention uses the above-mentioned position / orientation estimation device and a trained model in which the relationship between the avoidance target image data representing the avoidance target and the identification information for identifying whether or not the avoidance target is learned is used. A determination unit for determining whether or not the image acquired by the captured image acquisition unit includes an object to be avoided, and the acquired image includes an object corresponding to the avoidance target image. If so, a route generation unit that generates a detour route that bypasses the relevant route and a movement control that moves the position / attitude information obtained by the position / orientation information calculation unit along the generated detour route. It is a mobile robot having a unit and.

Further, in one aspect of the present invention, the captured image acquisition unit acquires image data taken at any position and orientation in the construction site, and the position and orientation information calculation unit is a tertiary of the construction object at the construction site. Obtain the position / orientation information corresponding to the image data by using the trained model in which the relationship between the position / orientation of any of the construction sites and the image when viewed from the first-person viewpoint is learned in the original model data. This is a position / orientation estimation method.

Further, in one aspect of the present invention, the position / orientation information acquisition unit acquires the position / orientation information indicating the position / orientation of any of the construction sites in the three-dimensional model data of the construction object at the construction site, and the image acquisition unit This is a learning method in which an image viewed from the position / orientation in the three-dimensional model data is acquired from the first-person viewpoint, and the learning unit learns a condition representing the relationship between the position / orientation information and the image.

As described above, according to the present invention, it is possible to accurately obtain the position / orientation at the construction site while suppressing the increase in the weight of the device required for estimating the position / attitude information.

It is a schematic block diagram which shows the structure of the situation management system 1 by one Embodiment of this invention. It is a figure explaining the process flow of the situation management system 1. It is a flowchart explaining the process of the autonomous mobile robot 40.

Hereinafter, the situation management system 1 using the position / orientation estimation device according to the embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic block diagram showing a configuration of a situation management system 1 according to an embodiment of the present invention.
The situation management system 1 includes a three-dimensional model image generation device 10, a first-person viewpoint image database 20, a learning device 30, an autonomous mobile robot 40, and a situation management device 50.

The three-dimensional model image generation device 10 stores three-dimensional model data of a construction object at a construction site. The three-dimensional model image generation device 10 uses the three-dimensional model data of the construction object at the construction site to generate an image when viewed from a first-person viewpoint from any position or orientation of the construction site. Images from a first-person perspective are generated at a number of different points within the construction site. For example, the position / orientation for generating the image from the first-person viewpoint may be any position / orientation as long as it can be a path for the autonomous mobile robot 40 to move in the construction object. The construction object may be, for example, a structure or a building. When the object to be constructed is a building or the like, images from a first-person viewpoint are generated for various places on a certain floor on each floor.

When the three-dimensional model image generation device 10 generates an image from the first-person viewpoint, the three-dimensional model image generation device 10 generates an image from the first-person viewpoint in each of the three-dimensional model data at each stage during construction. As a result, it is possible to generate an image showing what the construction object looks like during the construction.

In addition, the three-dimensional model image generation device 10 generates position / orientation information indicating the position / orientation in the first-person viewpoint when the image is generated. This position / attitude information includes latitude, longitude, altitude, and attitude angle, and can specify the position / attitude at the construction site when viewed from the first-person viewpoint.
Further, the three-dimensional model image generation device 10 has, for example, a function of superimposing a coordinate system representing latitude, longitude, altitude, and attitude angle on the coordinate system of the three-dimensional model, and this superposition is performed. Therefore, it is possible to grasp the latitude, longitude, altitude, and attitude angle for any position and orientation of the construction object in the three-dimensional model.

The image generated here is preferably a spherical image, but may be a simulation of a camera having a wide angle of view. By using a spherical camera or a camera having a wide angle of view, it is possible to obtain an image capable of estimating the position and orientation regardless of the direction in which the autonomous mobile robot 40 has taken a picture.

The three-dimensional model image generation device 10 includes image data when viewed from any position / orientation of the construction site from a first-person viewpoint, and position / orientation information indicating a position / orientation at the first-person viewpoint when the image is generated. Output image information.
The three-dimensional model image generation device 10 may use the three-dimensional model data represented by BIM (Building Information Modeling) or the three-dimensional model data represented by CAD (computer-aided design) as the three-dimensional model data. ..

The first-person viewpoint image database 20 stores image information (combination of image data and position / orientation information) output from the three-dimensional model image generation device 10.

The learning device 30 uses the three-dimensional model image generation device 10 to generate a sufficient amount of images at all points at the construction site, and then inputs the image and the position / orientation information of the image by the deep learning technique to obtain the image. Learn the conditions that express the relationship with the position and orientation (latitude, longitude, altitude, attitude angle).
The learning device 30 may be configured by combining a workstation for image analysis, a small computer, and the like.

More specifically, the learning device 30 includes an image information acquisition unit 31, a position / posture information learning unit 32, an avoidance target learning unit 33, and an output unit 34.
The image information acquisition unit 31 acquires image information from the first-person viewpoint image database. This image information includes the position / orientation information indicating the position / orientation of any of the construction sites in the 3D model data of the construction object at the construction site, and the position / orientation when viewed from the position / orientation in the 3D model data from the first-person viewpoint. Includes image data. This image information is image data for each position and posture at a construction site.
The position / orientation information learning unit 32 uses the image information acquired by the image information acquisition unit 31 to learn the conditions representing the relationship between the position / orientation information and the image based on the learning algorithm, and generates a trained model. To do.
For example, the position / orientation information learning unit 32 performs image processing on the image data included in the image information, and performs image processing on the object (pillar, beam, wall, floor, ceiling, window, equipment, fixture, etc.) included in the image data. By analyzing the shape (contour, etc.), color, size, degree of light reflection, etc., these are acquired as feature quantities (feature data). The position / attitude information learning unit 32 learns the relationship between the feature amount and the position / attitude information, and generates a learned model. Here, in the learning in the position / posture information learning unit 32, for example, the condition indicating the relationship between the feature amount and the position / posture information may be learned. This condition in the position / orientation information learning unit 32 is a model structure representing the correlation between the image data and the position / attitude information, or various parameters for determining the model structure.

The learning of the position / orientation information learning unit 32 may be any learning method, for example, machine learning using AI (Artificial Intelligence) technology, enhanced learning, and deep learning by a neural network including a plurality of intermediate layers. Any of the techniques (deep learning) can be used. Further, as the learning method, either supervised learning or unsupervised learning may be used.
When performing unsupervised learning, the position / orientation information learning unit 32 is based on, for example, the shape, color, size, degree of light reflection, and the like of an object included in the image data based on the feature amount of the image data. By performing cluster analysis, the image data may be classified according to the position / orientation information.

For example, the position / orientation information learning unit 32 does not know the correlation between the feature amount of the image data and the position / orientation information at the start of learning, but gradually identifies the features and correlates as the learning progresses. To interpret. As the learning progresses, the position / orientation information learning unit 32 receives the feature amount of the image data (shape (contour, etc.), color, size, light reflection degree, etc. of the object included in the image data) and position / orientation information (latitude). , Longitude, altitude, attitude angle) can be brought closer to the optimal solution. The position / orientation information learning unit 32 learns the relationship between the feature amount of the image data and the position / attitude information, but the relationship between the image data and the position / attitude information is not used without using the feature amount of the image data. You may try to learn the condition that represents. In this case, the position / orientation information learning unit 32 may learn, for example, the conditions representing the relationship between each pixel included in the image data and the position / attitude information.

The trained model obtained in this way can calculate the position and orientation from the image taken at the actual site. The image captured at the actual site may be, for example, an image autonomously photographed by an autonomous mobile robot or a flying robot, or an image photographed by a worker patrolling the site. Since the image captured in this way and the captured position / posture can be saved, the situation can be easily managed.

The avoidance target learning unit 33 generates a learned model by learning the relationship between the avoidance target image data representing the avoidance target and the identification information that identifies whether or not the avoidance target is. The objects to be avoided are workers, heavy machinery, building materials, etc. The avoidance target learning unit 33 performs image processing on the image data included in the image information, and shapes (contours) of the objects (workers, heavy machinery, temporarily stored building materials, etc.) to be avoided included in the image data. Etc.), color, size, degree of light reflection, etc., and these are acquired as feature quantities (feature data). The avoidance target learning unit 33 learns the relationship between the obtained feature amount and the identification information that identifies whether or not it is an avoidance target. For example, learn the relationship between workers, heavy machinery, and building materials with identification information indicating that they are to be avoided. As the image data to be avoided, images taken from various types and various angles are used for each of the workers, heavy machinery, and temporarily stored building materials. This makes it possible to obtain a trained model that can accurately identify workers, heavy machinery, and temporarily stored building materials.
Further, the avoidance target learning unit 33 includes identification information indicating that the avoidance target is not an avoidance target, and an image of an object that does not need to be avoided (on the floor surface so that the worker can walk without soiling the floor surface). You may learn the relationship with the laid curing sheet, etc.).
Since the learning method of the avoidance target learning unit 33 is common to the position / posture information learning unit 32, the description thereof will be omitted.

The output unit 34 outputs the learned model generated by the position / attitude information learning unit 32 and the learned model generated by the avoidance target learning unit 33.

The autonomous mobile robot 40 includes a position / orientation estimation unit 41, an imaging unit 42, a determination unit 43, a route generation unit 44, a drive unit 45, a movement control unit 46, and a communication unit 47.
The position / attitude estimation unit 41 includes an image capture image acquisition unit 411, a position / attitude information calculation unit 412, a storage unit 413, and a position / attitude information output unit 414.
The captured image acquisition unit 411 acquires image data taken at any position and orientation in the construction site. For example, the captured image acquisition unit 411 acquires the image data captured by the image capturing unit 42. This image data is an image when viewed from a first-person viewpoint from any position or posture inside the construction object. In addition, this image data need only include an image of the construction site at least in part, and an image taken from the inside of the construction object or an image taken from the outside of the construction object should be used. Can be done.

The position / posture information calculation unit 412 has learned the relationship between the position / posture of any of the construction sites and the image when viewed from the first person's viewpoint in the three-dimensional model data of the construction object at the construction site. The model is used to obtain the position / orientation information corresponding to the image data. The trained model used at this time is a trained model generated by the learning device 30.

The storage unit 413 stores the learned model output from the position / posture information learning unit 32 and the learned model output from the avoidance target learning unit 33.

The position / attitude information output unit 414 outputs the position / attitude information obtained by the position / attitude information calculation unit 412.

The imaging unit 42 photographs the surroundings and outputs imaging data (image data). The imaging unit 42 is, for example, a camera.
The determination unit 43 avoids the image acquired by the captured image acquisition unit by using a learned model that has learned the relationship between the avoidance target image data representing the avoidance target and the identification information that identifies whether or not the avoidance target is present. Determine if the target object is included. The trained model used by the determination unit 43 is a trained model generated by the avoidance target learning unit 33.
The route generation unit 44 generates route information indicating a route for the autonomous mobile robot 40 to move in the construction site. This route information is, for example, a route that goes through the position and orientation of the target for managing the situation at the construction site.
When the acquired image includes an object corresponding to the avoidance target image, the route generation unit 44 generates a detour route that is a route that bypasses the object. This detour route is a route that returns to the route indicated by the route information after avoiding the avoidance object.

The drive unit 45 drives the movement mechanism of the autonomous mobile robot 40. As the moving mechanism, the wheels attached to the autonomous moving robot 40 are rotationally driven to move them in the front-rear direction, and the steering wheels are steered to change the traveling direction in the left-right direction.
When the autonomous mobile robot 40 is a drone, the drive unit 45 rotationally drives a rotor provided in the drone or changes the flight direction. The autonomous mobile robot 40 may be an unmanned aerial vehicle, and may be a multicopter in addition to a drone.

The movement control unit 46 moves along the detour route in which the position / attitude information obtained by the position / attitude information calculation unit 412 is generated.

The communication unit 47 communicates with the learning device 30 and the situation management device 50. The image data captured by the situation management device 50 and the position / orientation information at which the image data was captured are transmitted. Further, when transmitting such information, the communication unit 47 can also transmit information indicating the date and time when the image data was captured.

The situation management device 50 stores various information transmitted from the autonomous mobile robot 40. The information stored in the situation management device 50 can be viewed by using the terminal device used by the manager of the construction site.

Regarding the storage unit provided in the first-person viewpoint image database 20, the storage unit 413, and the situation management device 50, for example, a storage medium, for example, an HDD (Hard Disk Drive), a flash memory, or an EEPROM (Electrically Erasable Program Read Only Memory). , RAM (Random Access read / write memory), ROM (Read Only Memory), or any combination of these storage media, and for example, a non-volatile memory can be used.

FIG. 2 is a diagram illustrating a processing flow of the status management system 1.
The three-dimensional model image generation device 10 uses the three-dimensional model data to generate an image when viewed from a first-person viewpoint from any position or orientation on the inner side of the construction object. Here, a large number of images are generated by changing the position and orientation. The three-dimensional model image generation device 10 writes the generated image and the position / orientation information indicating the position / orientation of the first-person viewpoint when the image is generated in the first-person viewpoint image database 20 as image information (step S10).
Further, here, the image data to be avoided is stored in a predetermined storage area (step S11). The predetermined storage area may be a storage device external to the learning device 30, or the storage area of the situation management device 50 may be used.
The learning device 30 acquires image information when viewed from the first-person viewpoint and image data to be avoided (step S12). Then, the learning device 30 generates a learned model using the image information by the position / orientation information learning unit 32, and generates a learned model using the avoidance target image data by the avoidance target learning unit 33 (step S13). These generated trained models are stored in the storage unit 413 of the autonomous mobile robot 40 (step S14).

After the autonomous mobile robot 40 is placed in the construction site, the image pickup unit 42 images the surroundings (step S15). This shooting is performed every time the shooting timing comes. The shooting timing may be every fixed time (every few seconds, every few minutes, etc.), may be the timing when it is detected that the person has moved a certain distance, or the timing when the shooting target point is reached. It may be.

The autonomous mobile robot 40 uses the learned model generated by the position / orientation information learning unit 32 (step S16) to obtain the position / attitude information corresponding to the image captured by the imaging unit 42 (step S17). The autonomous mobile robot 40 drives the drive unit 45 so that the obtained position / attitude information follows a predetermined path information. Further, the autonomous mobile robot 40 uses the learned model generated by the avoidance target learning unit 33 to determine whether or not the image captured by the imaging unit 42 has an avoidance target, and if there is an avoidance target. Generates an avoidance route (step S18).

FIG. 3 is a flowchart illustrating the processing of the autonomous mobile robot 40.
The autonomous mobile robot 40 uses the route generation unit 44 to generate route information indicating a route that passes through the position and orientation of the target that manages the status of the construction site, and starts moving along the route information. Then, the autonomous mobile robot 40 captures the surroundings by the imaging unit 42 while moving in the construction site, and obtains image data (step S50). The autonomous mobile robot 40 analyzes the obtained image data by performing image processing or the like (step S51), and obtains position / orientation information using the learned model. Here, the position / posture information calculation unit 412 determines whether or not the obtained position / posture information can be correctly recognized (step S52). For example, the speed obtained from the time from the time when the previous position / attitude information was calculated to the time when the current position / attitude information was calculated, and the distance between the position / attitude information obtained last time and the position / attitude information obtained this time. However, if the moving speed of the autonomous moving robot 40 exceeds a certain level, it is determined that the autonomous moving robot 40 is not correctly recognized. In this case, it is possible to prevent the obtained position / posture information from being adopted as being incorrect, and it is possible to prevent the accuracy of the position / posture information calculation from being lowered. In this case, the judgment is made based on the moving speed, but if the distance between the position / posture information obtained last time and the position / posture information obtained this time is a certain distance or more, it is correctly recognized. You may decide that it will not be done.
On the other hand, when the position / orientation information can be correctly recognized, whether or not there is an object to be avoided in the image data used for calculating the position / orientation information has been learned by the avoidance target learning unit 33. Recognize using the model (step S54).

The route generation unit 44 generates a movement route (route information) based on the recognition result of whether or not there is an avoidance target and the route information. Here, when there is no avoidance target, route information for moving from the current position / posture recognized in step S52 to the position / posture of the target for managing the status of the construction site is generated (step S55). When generating route information, for example, three-dimensional model data such as BIM or CAD is used, and a movable route is generated based on the surrounding environment (passage, wall, pillar, etc.) of the autonomous mobile robot 40. On the other hand, when there is an avoidance target, the route generation unit 44 generates route information for moving to the position / posture of the target for managing the status of the construction site while avoiding the avoidance target. As a result, even if images of workers, materials, heavy machinery, etc. that do not exist in the 3D model data are included in the image data, they are recognized as avoidance targets, and a detour route that avoids them is generated and detoured. be able to.
When the route information is generated, the autonomous mobile robot 40 controls the position and attitude by driving the robot so as to move along the generated route information (step S56).

In this way, the autonomous mobile robot 40 returns to the start position / posture after passing through a plurality of positions / postures of the target for managing the situation at the construction site, and the photographed image data and the position / orientation in which the image data is photographed. Information and information are transmitted to the status management device 50. Here, date and time data representing the date and time when the image data was captured is also transmitted. As a result, the situation management device 50 can store the image data obtained from the autonomous mobile robot 40, the position / orientation information, and the date and time when the image data was taken in association with each other.

According to the embodiment described above, by effectively using BIM, which has been introduced in recent years, to learn the position / orientation information, no special device is required, and only an inexpensive camera and computer are used. By utilizing deep learning technology, it is possible to acquire position and attitude information and manage on-site progress at the same time. This facilitates the introduction of autonomous mobile robots and the like, and can contribute to improving the productivity of the construction industry.

Further, according to the present embodiment, it is possible to acquire the position / attitude information necessary for the movement of the robot and the photograph associated with the position / attitude information necessary for the progress management by the lightweight system. it can. Further, by returning the result to BIM, it is possible to manage the finished product on a daily basis, which can contribute to better on-site management and improvement of learning accuracy of the learning device 30.

Further, conventionally, LIDAR and Visual SLAM are methods for sequentially estimating the position and orientation, and do not always calculate the absolute position and orientation. In addition, although all of them are used for progress management (finished form management) of construction sites, they are not lightweight systems that have both absolute self-position posture estimation and progress management.

On the other hand, in the present embodiment, a first-person viewpoint image is generated from BIM (Billing Information Modeling) and CAD data, which have been introduced in recent years, and position / orientation information is learned by deep learning technology, and a lightweight camera image is used in the field. Progress management can be facilitated by estimating the self-position / orientation from the above and saving the image associated with the position / orientation information. In addition, since it is possible to suppress an increase in the weight of the equipment required for estimating the self-position and posture, it is possible to mount the equipment necessary for estimating the self-position and posture on a mobile robot or the like.

Further, since the situation at the construction site changes every day, it is desirable that the learning with the learning device 30 is carried out every few days. Further, by accumulating the image data in the situation management device 50, the finished shape is managed based on the image data, and by reflecting it in the three-dimensional model data, the position / orientation information is estimated using the learning model. It can be expected that the accuracy will also improve.

Although the position / posture estimation unit 41 described above has been described as one function provided inside the autonomous mobile robot 40, it may be configured as an independent device separate from the autonomous mobile robot 40. In this case, by mounting the device having the function of the position / orientation estimation unit 41 on the existing autonomous mobile robot, even the existing autonomous mobile robot can function as the above-mentioned autonomous mobile robot 40.

The learning device 30 and the position / posture estimation unit 41 in the above-described embodiment may be realized by a computer. In that case, the program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. It may also include a program that holds a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or a client in that case. Further, the above program may be for realizing a part of the above-mentioned functions, and may be further realized for realizing the above-mentioned functions in combination with a program already recorded in the computer system. It may be realized by using a programmable logic device such as FPGA (Field Programmable Gate Array).

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and the design and the like within a range not deviating from the gist of the present invention are also included.

1 ... Situation management system, 10 ... Three-dimensional model image generator, 20 ... First-person viewpoint image database, 30 ... Learning device, 31 ... Image information acquisition unit, 32 ... Position / orientation information learning unit, 33 ... Avoidance target learning unit, 34 ... Output unit, 40 ... Autonomous mobile robot, 41 ... Position and orientation estimation unit, 42 ... Imaging unit, 43 ... Judgment unit, 44 ... Path generation unit, 45 ... Drive unit, 46 ... Movement control unit, 47 ... Communication unit, 50 ... Situation management device, 411 ... Captured image acquisition unit, 412 ... Position / orientation information calculation unit, 413 ... Storage unit, 414 ... Position / orientation information output unit

Claims (6)

An image acquisition unit that acquires image data taken at any position and orientation within the construction site,
Using a trained model that learned the relationship between the position and orientation of any of the construction sites and the image when viewed from the first-person perspective from the position and orientation in the three-dimensional model data of the construction object at the construction site, the image The position / attitude information calculation unit that obtains the position / attitude information corresponding to the data,
Position and orientation estimation device. The position / orientation estimation device according to claim 1, wherein the image is an image when viewed from a first-person viewpoint from any position / orientation inside the construction object. A position / attitude information acquisition unit that acquires position / attitude information indicating the position / orientation of any of the construction sites in the three-dimensional model data of the construction object at the construction site.
An image acquisition unit that acquires an image when viewed from the position and orientation in the three-dimensional model data from the first-person viewpoint,
A learning unit that learns conditions that represent the relationship between the position / posture information and the image,
Learning device with. The position / orientation estimation device according to claim 1 or 2.
Using a trained model that has learned the relationship between the avoidance target image data representing the avoidance target and the identification information that identifies whether or not it is the avoidance target, the object that is the avoidance target in the image acquired by the captured image acquisition unit. A judgment unit that determines whether or not is included, and
When the acquired image includes the object to be avoided, a route generation unit that generates a detour route that is a route that bypasses the object, and a route generation unit.
A movement control unit that moves the position / attitude information obtained by the position / attitude information calculation unit along the generated detour route, and a movement control unit.
Mobile robot with. The captured image acquisition unit acquires image data taken at any position and orientation in the construction site, and
The position / posture information calculation unit has learned the relationship between the position / posture of any of the construction sites and the image when viewed from the first person's viewpoint in the three-dimensional model data of the construction object at the construction site. A position / posture estimation method for obtaining position / posture information corresponding to the image data using a model. The position / posture information acquisition unit acquires the position / posture information representing the position / posture of any of the construction sites in the three-dimensional model data of the construction object at the construction site.
The image acquisition unit acquires an image when viewed from the position and orientation in the three-dimensional model data from the first-person viewpoint.
A learning method in which a learning unit learns a condition representing a relationship between the position / posture information and the image.

Images