JP2019212203A - Three-dimensional (3d) model generation system - Google Patents

Abstract

To provide a 3D model generation system that, when generating a 3D model using images from a single lens camera, is freely movable in a photographing object, and can generate the 3D model of high accuracy.SOLUTION: A configuration of a 3D model generation system (a generation system 100) according to the present invention includes: a first moving body 110; a second moving body 120; two single lens cameras (a first single lens camera 130a and second single lens camera 130b) that are mounted to each of the first moving body 110 and the second moving body 120; and a computation device 140 that generates the 3D model. The computation device 140 is configured to: generate the 3D model from images photographed by the single lens camera of the first moving body 110; collate images photographed by the single lens camera of the second moving body 120 with the 3D model, and modify the 3D model.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a 3D model creation system that creates a 3D model from an image photographed by a monocular camera mounted on a moving body.

  Methods for estimating the surrounding space and self-position based on images from a monocular camera mounted on a moving body are called Monochromatic SFM (Structure from Motion) and Monocular VO (Visual Odometry), and research is ongoing in various places. . The technique using a monocular camera has the advantage that the apparatus can be miniaturized because the number of cameras is half that of the technique using a stereo camera. On the other hand, however, the method using a monocular camera has a disadvantage in that the accuracy and efficiency of the 3D model to be created is reduced because the amount of information from the camera is smaller than the method using a stereo camera.

  As a method for compensating for the disadvantages of the above-described method using a monocular camera, for example, Patent Document 1 discloses an external environment recognition apparatus including a rotating camera. In the external environment recognition apparatus of Patent Document 1, for example, a monocular camera is installed in a heavy machine, and the external environment is recognized from an image captured by the monocular camera by rotating the monocular camera at a certain rotation center and rotation radius. .

JP 2006-156629 A

  According to Patent Document 1, it is possible to accurately estimate the surrounding three-dimensional information and the 3D surface using only a monocular camera. However, with the configuration of Patent Document 1, the monocular camera can be moved only on the path of a predetermined heavy machine, and the movement of the monocular camera is restricted. Then, the technique of Patent Document 1 cannot be applied when attempting to search by moving freely in a vast area. Therefore, there is room for further improvement in the technique of Patent Document 1.

  In view of such a problem, the present invention is capable of freely moving in a region to be imaged when creating a 3D model using an image from a monocular camera, and creating a highly accurate 3D model. It is an object of the present invention to provide a 3D model creation system capable of performing

  In order to solve the above problems, a typical configuration of a 3D model creation system according to the present invention includes a first moving body, a second moving body, a first moving body, and a second moving body, respectively. The computing device including two mounted monocular cameras and a computing device that creates a 3D model creates a 3D model from an image captured by the monocular camera of the first moving body, and The 3D model is corrected by collating the image photographed by the monocular camera with the 3D model.

  According to the above configuration, by operating the first moving body and the second moving body, it is possible to freely move the moving bodies, and thus the monocular cameras mounted on them, in the region within the imaging target. . Therefore, it is possible to photograph the area within the photographing target in detail. Then, a 3D model is created from an image photographed by a monocular camera mounted on the first moving body, and the 3D model is corrected by an image photographed by the monocular camera of the second moving body. It is possible to improve the accuracy of the.

  The first moving body may be a traveling body, and the second moving body may be a flying body. According to such a configuration, it becomes possible to shoot a region to be imaged more diversified.

  The 3D model creation system further includes a light source provided on the upper surface of the traveling body and a light receiving unit provided on the lower surface of the flying object, and the arithmetic unit is configured to receive the light from the light source at the light receiving unit. It is good to control the position of. Thereby, the horizontal position of the flying object which is the second moving object can be suitably controlled.

  The first moving body and the second moving body are both traveling bodies, and the first moving body and the second moving body may travel in a state of being shifted back and forth. Even when the first moving body and the second moving body are both traveling bodies as in this configuration, the same effect as described above can be obtained. Further, by using both of the two moving bodies as traveling bodies, it becomes possible to reduce the cost compared to the case where one of them is a flying body.

  Of the first moving body and the second moving body, a traveling body that travels forward is provided with a marker, and the traveling body that travels rearward among the first moving body and the second moving body. May travel while photographing the marker of the traveling body traveling forward. According to this configuration, the relative position and the relative distance between the first moving body and the second moving body can be calculated by referring to the marker on the image captured by the traveling body traveling behind.

  In order to solve the above-described problem, another configuration of the 3D model creation system according to the present invention includes a first moving body that moves in a facility, a monocular camera mounted on the first moving body, and a 3D model. A computing device that creates a reference 3D model of equipment arrangement in the facility, creates a 3D model from an image captured by a monocular camera of the first moving body, and creates a reference 3D The 3D model created by collating the model with the created 3D model is modified.

  Also with the above configuration, by operating the first moving body, the first moving body, and thus the monocular camera mounted on the first moving body, can be freely moved in the region within the imaging target. Then, a 3D model is created from an image photographed by a monocular camera mounted on the first moving body, and the 3D model created from the image is separately created based on equipment information such as a device layout drawing. It is possible to improve the accuracy of the 3D model created from the image by correcting with the 3D model for use. In particular, when an unknown object that does not exist in equipment information such as a device layout diagram exists by comparing a 3D model created from an image captured by a monocular camera with a reference 3D model, the unknown object Refinement of the 3D model is also possible.

  The arithmetic device calculates a position of the first moving body from an image photographed by the monocular camera of the first moving body, and corrects the position by comparing the photographed image with the corrected 3D model. It is characterized by. According to such a configuration, the position of the first moving body can be accurately grasped.

  According to the present invention, when a 3D model is created using an image from a monocular camera, the 3D model can be freely moved in the region to be imaged and can create a highly accurate 3D model. A creation system can be provided.

It is a figure explaining the 3D model creation system concerning a 1st embodiment. It is a figure explaining the state of the 1st moving body at the time of image photography, and a 2nd moving body. It is a figure explaining the process of the arithmetic unit in 1st Embodiment. It is a figure explaining the 3D model creation system concerning 2nd Embodiment. It is a figure explaining the process of the arithmetic unit in 3rd Embodiment.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiments are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

(First embodiment)
FIG. 1 is a diagram illustrating a 3D model creation system (hereinafter simply referred to as a creation system 100) according to the first embodiment. As shown in FIG. 1, a creation system 100 according to the present embodiment includes a first moving body 110, a second moving body 120, two monocular cameras, a first monocular camera 130a and a second monocular camera 130b, and an arithmetic operation. The apparatus 140 is comprised. The first monocular camera 130a and the second monocular camera 130b are mounted on the first moving body 110 and the second moving body 120, respectively. The arithmetic device 140 creates a 3D model.

  FIG. 2 is a diagram illustrating the state of the first moving body 110 and the second moving body 120 during image shooting. In the first embodiment, the first moving body 110 is a traveling body, and the second moving body 120 is a flying body. The second moving body 120 is supplied with power from the first moving body 110 through the power line 122. Thus, by using the traveling body and the flying body as the two moving bodies (the first moving body 110 and the second moving body 120), as shown in FIG. Can be photographed from a low position, and the second moving body 120 can photograph the region to be measured from a high position. Therefore, it becomes possible to shoot the area to be imaged more diversified.

  As shown in FIG. 2, in the creation system 100 of the first embodiment, the light source 152 is provided on the upper surface of the traveling body that is the first moving body 110, and the flying body that is the second moving body 120. A light receiving portion 154 is provided on the lower surface. The arithmetic device 140 (see FIG. 1) controls the position of the flying object (second moving body 120) so that the light from the light source 152 is received by the light receiving unit 154.

  According to the above configuration, in a state where the light from the light source 152 is received by the light receiving unit 154, the second moving body 120 is accurately positioned above the first moving body 110. Therefore, the horizontal position of the flying body which is the second moving body 120 can be controlled favorably, and the positional relationship between the first moving body 110 and the second moving body 120 can be properly grasped. It becomes possible. Note that the positional relationship (altitude difference) between the two in the vertical direction can be grasped by the length of the power line connecting them or a sensor such as a barometer mounted on the second moving body 120.

  FIG. 3 is a diagram for explaining processing of the arithmetic device 140 according to the first embodiment. In the present embodiment, the arithmetic device 140 uses the image (2D image shown in FIG. 3A) captured by the first monocular camera 130a of the first moving body 110 to perform 3D shown in FIG. Create a model. When the 3D model is created as shown in FIG. 3B, the arithmetic device 140 corrects the angle of the 3D model so as to approximate the viewpoint of the second monocular camera 130b as shown in FIG. 3C.

  FIG. 3D is an image (2D image) captured by the second monocular camera 130 b of the second moving body 120. When the angle of the 3D model is corrected as described above, the arithmetic device 140 causes the image captured by the second monocular camera 130b illustrated in FIG. 3D and the 3D model in which the angle illustrated in FIG. And match. An image of collation performed inside the arithmetic device 140 is as shown in FIG.

  Then, the arithmetic unit 140 determines that the feature point P1 (illustrated by a black circle) of the image in FIG. 3C is close to the feature point P2 (illustrated by a white circle) of the image in FIG. Modify the 3D model. As a result, a corrected 3D model is created as shown in FIG.

  As described above, according to the creation system 100 of the first embodiment, by operating the first moving body 110 and the second moving body 120, those moving bodies and thus the monocular cameras mounted on them ( The first monocular camera 130a and the second monocular camera 130b) can be freely moved in the area within the subject to be imaged. Therefore, it is possible to photograph the area within the photographing object in detail, and to obtain a sufficient image necessary for creating the 3D model.

  Then, the arithmetic device 140 creates a 3D model based on the image photographed by the first monocular camera 130a, and corrects the 3D model with the image photographed by the second monocular camera 130b. As a result, it is possible to obtain a highly accurate 3D model as compared to a case where a 3D model is created only by an image captured by the first monocular camera 130a.

  Also, usually, by mounting the second moving body 120 on the first moving body 110, the region within the object to be imaged is moved, and only a 3D model is created from the image captured by the first monocular camera 130a. The 3D model can be corrected with an image taken by the second monocular camera 130b from the position where the second moving body 120 is mounted on the first moving body 110 without correction. Thereby, it can move freely even in a narrow place where there is a height restriction. In order to improve the accuracy of the 3D model, the second moving body 120 may be raised as appropriate, and the 3D model may be corrected with an image captured by the second monocular camera 130b. Therefore, since the area within the object to be photographed can be freely moved without limitation, various portions can be modeled without omission, and the 3D model can be precisely modified as necessary.

  Furthermore, if necessary, the state of obstacles around the first moving body 110 can be confirmed by raising the second moving body 120 and photographing the first moving body 110. As a result, it becomes possible to check the situation of obstacles on the side and rear where the first moving body 110 becomes a blind spot in the investigation of the place where there are many obstacles, and the operational safety is improved. improves.

(Second Embodiment)
FIG. 4 is a diagram illustrating a 3D model creation system (hereinafter referred to as creation system 200) according to the second embodiment. In the following embodiments, components that are the same as those in the above-described embodiments are denoted by the same reference numerals, and description thereof is omitted.

  As shown in FIG. 4A, the creation system 200 of the second embodiment differs from the creation system 100 of the first embodiment in that the second moving body 120 is not a flying body but a traveling body. That is, in the creation system 200 of the second embodiment, both the first moving body 110 and the second moving body 120 are traveling bodies.

  Even when both the first moving body 110 and the second moving body 120 are traveling bodies as in the above-described configuration, the first moving body 110 and the second moving body 110 are the same as in the creation system 100 of the first embodiment. The moving bodies 120 and the monocular cameras (the first monocular camera 130a and the second monocular camera 130b) mounted thereon can be freely moved in the area within the photographing target. Further, by using both of the two moving bodies as traveling bodies, it becomes possible to reduce the cost compared to the case where one of them is a flying body.

  In the creation system 200 of the second embodiment, as shown in FIG. 4A, the first moving body 110 and the second moving body 120 travel in a state where they are shifted back and forth. As a result, it is possible to efficiently shoot the region to be imaged from two different angles. Hereinafter, of the first moving body 110 and the second moving body 120, the first moving body 110 is a traveling body that travels forward, and the second moving body 120 is a traveling body that travels backward. This will be explained assuming that.

  In particular, in the creation system 200 of the second embodiment, as shown in FIG. 4B, the first moving body that is a traveling body that travels ahead of the first moving body 110 and the second moving body 120. A marker 202 is attached to 110. And among the 1st moving body 110 and the 2nd moving body 120, the 2nd moving body 120 which is a traveling body which travels back is photographing the marker 202 of the 1st moving body which travels ahead. Run.

  Specifically, as shown in FIG. 4B, the first moving body 110 that is a traveling body that moves forward is provided with a square marker 202 having a side length of x. . FIG. 4C is a diagram illustrating an image captured by a traveling body traveling behind. In the image illustrated in FIG. 4C, the lengths of the two sides of the marker 202 attached to the first moving body 110 are a and b, and the length of the upper side is c.

  Therefore, according to the above configuration, the arithmetic device 140 calculates the ratio between the actual length of the marker 202 and the length of each side of the marker 202 on the image taken by the traveling body (second moving body 120) traveling behind. Referring to (see FIG. 1), it is possible to calculate the relative position and the relative distance of the first moving body 110 and the second moving body 120. Note that the creation of the 3D model in the arithmetic device 140 is the same as that in the first embodiment, and a description thereof will be omitted.

  In the second embodiment, the operation method in which shooting is performed using both the first monocular camera 130a of the first moving body 110 and the second monocular camera 130b of the second moving body 120 has been described. It is not limited to. For example, as long as sufficient shooting can be performed with a single-lens camera of either one of the moving bodies, the one moving body may be shot from a position separated by the single-lens camera of the other moving body. Thereby, the condition of the obstacle around one moving body can be confirmed. Therefore, when investigating a place with many obstacles, it is possible to check the situation of obstacles on the side or behind where a single-lens camera of one moving body becomes a blind spot, improving operational safety. .

(Third embodiment)
The 3D model creation system (hereinafter simply referred to as a creation system) of the third embodiment includes only the first moving body 110, not two moving bodies. The first moving body 110 may be either the traveling body or the flying body exemplified in the creation system 100 of the first embodiment. In the present embodiment, a facility in which facility information related to device arrangement is known in advance, such as a device arrangement drawing being available, is measured.

  FIG. 5 is a diagram for explaining processing of the arithmetic device 140 according to the third embodiment. In the creation system of the third embodiment, the arithmetic device 140 first refers to equipment information such as equipment layout in the measurement target facility shown in FIG. 5A, and performs 3D of equipment placement shown in FIG. 5B. A model (reference 3D model) is acquired. In the example shown in FIG. 5A, two control panels including control panels 1 and 2 are installed in the facility. The widths of the control panels 1 and 2 are both d, and the distance between the control panels 1 and 2 is e.

  Next, the arithmetic device 140 uses the image (2D image) photographed by the first monocular camera 130a of the first moving body 110 shown in FIG. 3A to generate the 3D model shown in FIG. create. When the 3D model is created based on the image of the first monocular camera 130a, the arithmetic device 140 includes the reference 3D model of the device arrangement shown in FIG. 5B and the first monocular camera 130a shown in FIG. The 3D model created from the image is collated. An image of collation performed inside the arithmetic device 140 is as shown in FIG.

Then, the arithmetic device 140 determines that the feature point P4 (illustrated by a black circle) in the image of FIG. 5C is close to the feature point P3 (illustrated by a white circle) of the image in FIG. 5B. Modify the 3D model. As a result, a corrected 3D model is created as shown in FIG.
As described above, according to the creation system of the third embodiment, the image taken by the first monocular camera 130a of the first moving body 110 by using the reference 3D for the device arrangement that has been known in advance. It is possible to dramatically improve the accuracy of the 3D model created using the.

  In addition, according to the method using the reference 3D model of the device arrangement as in the third embodiment, when an unknown object that does not exist in the device arrangement drawing or the like exists in the field, the 3D model of the unknown object is also refined. It becomes possible. For example, in FIGS. 5C to 5E, it can be seen that an unknown object X that does not exist in the device layout diagram exists between the two control panels 1 and 2. Here, as shown in FIG. 5C, a 3D model is also created for the unknown object X from the image of the first monocular camera 130a. Then, as shown in FIG. 5E, the 3D model of the unknown object X is also corrected in accordance with the correction of the 3D model of the control panels 1 and 2 existing in the equipment layout diagram. In addition, since the distance e between the two control panels 1 and 2 is clear from the equipment layout drawing, etc., the width of the unknown object x can be grasped by calculating the ratio between the distance e and the width x of the unknown object. Is possible.

(Fourth embodiment)
In the 3D model creation system of the fourth embodiment, in addition to the above-described embodiment, the arithmetic device 140 calculates the position of the first moving body 110 from the image captured by the monocular camera 130a of the first moving body 110. Further, the position of the first moving body 110 is corrected by comparing the photographed image with the 3D model modified as described above.

  Specifically, an image that should be visible in the corrected 3D model is generated from the calculated position (generated image). Then, when the captured image and the generated image are collated, there is a possibility that a positional shift has occurred at each feature point. By correcting the position of the first moving body 110 so as to reduce this positional deviation, the accurate position of the first moving body 110 can be grasped.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  The present invention can be used as a 3D model creation system that creates a 3D model from an image captured by a monocular camera mounted on a moving body.

P1 ... feature point, P2 ... feature point, 100 ... creation system, 110 ... first moving body, 120 ... second moving body, 122 ... power line, 130a ... first monocular camera, 130b ... second monocular camera, DESCRIPTION OF SYMBOLS 140 ... Arithmetic unit, 152 ... Light source, 154 ... Light receiving part, 200 ... Creation system, 202 ... Marker

Claims (7)

A first moving body;
A second moving body;
Two monocular cameras mounted on each of the first moving body and the second moving body;
A computing device for creating a 3D model;
The arithmetic device includes:
Creating a 3D model from an image taken by the monocular camera of the first moving body;
3. A 3D model creation system, wherein the 3D model is corrected by collating an image photographed by a monocular camera of the second moving body with the 3D model.   The 3D model creation system according to claim 1, wherein the first moving body is a traveling body, and the second moving body is a flying body. The 3D model creation system further includes a light source provided on an upper surface of the traveling body, and a light receiving unit provided on a lower surface of the flying object,
The 3D model creation system according to claim 2, wherein the arithmetic device controls the position of the flying object so that light from the light source is received by the light receiving unit. The first moving body and the second moving body are both traveling bodies,
2. The 3D model creation system according to claim 1, wherein the first moving body and the second moving body travel in a state of being displaced forward and backward. Of the first moving body and the second moving body, a marker is attached to the traveling body that travels forward,
The traveling body that travels rearward among the first moving body and the second moving body travels while photographing a marker of the traveling body that travels forward. 3D model creation system. A first moving body that moves within the facility;
A monocular camera mounted on the first moving body;
A computing device for creating a 3D model;
Including
The arithmetic unit is:
Obtain a reference 3D model of the equipment layout in the facility,
Creating a 3D model from an image taken by the monocular camera of the first moving body;
3. A 3D model creation system, wherein the created 3D model is corrected by comparing the reference 3D model with the created 3D model. The arithmetic unit is:
Calculating a position of the first moving body from an image taken by a monocular camera of the first moving body;
The 3D model creation system according to any one of claims 1 to 6, wherein the position is corrected by comparing the photographed image with the modified 3D model.