JP2023011454A - Three-dimensional design data creation method, construction work method, three-dimensional design data creation system, and three-dimensional design data creation program - Google Patents

Abstract

To provide a 3D design data creation method, a construction work method, a 3D design data creation system, and a 3D design data creation program, by which three-dimensional design data is no longer required to be created in advance, data with a target height is automatically created based on positional information of a construction site, and executing construction based on the data can enhance construction accuracy and shorten construction steps.SOLUTION: A method includes: a first step of, while causing a construction machine 10 to travel, photographing markers M placed at locations where height information within a construction range or outside the construction range is required, by a single or multiple cameras installed in the construction machine 10; a second step of obtaining, from images of the markers photographed by the cameras, positional coordinates of the markers in the images and positional coordinates of the cameras; and a third step of creating 3D design data by identifying a planer plane from the positional coordinates of at least one marker and the positional coordinates of the cameras or the positional coordinates of the multiple markers.SELECTED DRAWING: Figure 4

Description

The present invention relates to a technique for creating 3D design data as a construction target based on images captured by a camera mounted on a construction machine.

In recent years, the shortage of construction engineers has become more serious, and as a countermeasure, a measure called "i-Construction" that utilizes ICT (Information and Communication Technology) is being promoted. In construction work based on "i-Construction", surveying is carried out to acquire initial shape data, which is the shape data of the construction area before construction, create target shape data as construction targets, and operate construction machinery to create target shapes. Construction is performed so that it becomes a shape. In addition, the shape of the construction object after construction is measured, and compatibility with the target shape data is inspected. In each process, by utilizing 3D design data and 3D survey data, labor can be saved by automation (for example, Non-Patent Document 1).

Specific examples of such "i-Construction" include GNSS (Global Navigation Satellite Systems) mounted on construction machinery (e.g., hydraulic excavators, bulldozers, and trucks) and IMU (Inertial Measurement Unit). A construction work method is known in which the current shape data (for example, terrain data) of a construction target is grasped by synthesizing a plurality of data obtained by a measuring device, etc., and the work is performed using the shape data. (For example, Non-Patent Document 1).

16A and 16B are diagrams showing an example of a conventional construction method for constructing roadbeds without using ICT. FIG. 16A shows measurement points for performing water string inspection within the construction area, and FIG. 16B shows a cross-sectional view of the construction area having a roadbed 160b on the upper layer of the roadbed 160a. Conventionally, a water line is stretched on a stake set at a survey point, and the amount of drop from the line to the roadbed is measured. While watching, the blade (working tool) of the construction machine was manually adjusted up and down until the design height as shown in FIG. 16B was reached.

FIG. 17A is a diagram showing an example of a construction method for constructing a roadbed using conventional ICT, and FIG. 17B is a cross-sectional view of a construction area having a roadbed 170b above a roadbed 170a. As shown in FIG. 17A, the coordinates of four points ABCD are surveyed in advance by a surveying instrument such as TS (Total Station), and the design data (usually the final finished surface, the height of the paved surface) TIN ( Create a Triangulated Irregular Network. The created TIN is read by the PC built in the construction machine, and the height from the design surface is calculated by communicating the construction machine and the TS. For example, in local coordinates, the pavement design height at a certain point is 0.7 (m), the distance between the lower end of the blade (working tool) of the construction machine and the prism installed on the construction machine is h, and TS Assuming that the height of the prism installed on the construction machine being observed is z, the operator is displayed or automatically controlled so that the following equation holds.
0.7-0.05-z+h=0

In Patent Document 1, for the purpose of improving the accuracy of work and shortening the process, surveying is performed, and the first step of acquiring initial shape data, which is the shape data of the object to be constructed before construction, and the initial shape data and a second step of formulating a construction plan based on target shape data as a construction target, and a third step of performing construction using a construction work machine according to the construction plan. Techniques are disclosed for working with shape data and current shape data that is partially updated based on point cloud data acquired by a 3D laser scanner.

Further, Patent Document 2 discloses a construction machine capable of moving in a construction site area for the purpose of performing the monitoring simply and with high precision by autonomously monitoring the construction site by the construction machine as seen from the construction machine. Using the detection sensor unit mounted in the construction site area, the detection result of the external index installed at the position that becomes the reference point of the construction site area, and the detection result of the detection sensor unit attached to the movable work tool of the construction machine Based on the result of detection of the movable index by the detection sensor unit, a survey calculation is performed using the positional relationship between each position of the external index and the movable index and the position of the detection sensor unit, and the external index is detected. and the detection sensor unit, recognizes the positional relationship between the movable index and the detection sensor unit, further recognizes the positional relationship between the movable index and a location to be worked by the movable work tool, A technique for recognizing position information including three-dimensional coordinate values of a construction site by the movable work tool in the construction site area when the reference point is used as a reference by combining the recognition results of the above.

JP 2019-218766 A Patent No. 6684004

Ministry of Land, Infrastructure, Transport and Tourism i-Construction - Productivity revolution at construction sites - Reference material March 2016 (URL: www.mlit.go.jp/common/001127740.pdf)

However, in the conventional technology, it is necessary to create three-dimensional design data (sometimes referred to as 3D design data) using CAD (Computer Aided Design) or the like before construction at the construction site. Creating data requires considerable knowledge and skill, as well as a great deal of labor and time.

The present invention has been made in view of such circumstances, and does not need to create 3D design data in advance, and automatically creates design data based on the position information of markers installed at the construction site. , to provide a 3D design data creation method, a construction method, a 3D design data creation system, and a 3D design data creation program that can improve construction accuracy and shorten the process by carrying out construction based on the data. aim.

(1) In order to achieve the above objects, the present invention takes the following means. That is, the 3D design data creation method of the present invention is a 3D design data creation method for creating 3D design data as a construction target based on an image captured by a camera mounted on a construction machine. a first step of photographing a marker installed at a location where height information is required inside or outside the construction area with a single or a plurality of cameras mounted on the construction machine, and photographing with the camera. a second step of obtaining the position coordinates of the markers in the image and the position coordinates of the camera from the obtained image of the markers; and a third step of creating 3D design data by specifying a plane from the position coordinates.

With this configuration, there is no need to create 3D design data in advance, and the result of construction can be immediately reflected in the next construction, making it possible to perform highly accurate construction. Moreover, even if the operator of the construction machine does not have sufficient knowledge and experience, it is possible to easily perform the desired construction. In TS (Total Station), the TS and prism may be shielded during construction, and measurement may be interrupted. and errors due to multipath, the present invention is less subject to these restrictions. ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to implement|achieve information-aided construction with high precision and high efficiency.

(2) Further, in the 3D design data creation method of the present invention, in the second step, using VisualSLAM technology, the position coordinates of the camera and three-dimensional information of an object existing in the shooting range are acquired, and the camera The positional coordinates of the marker in the image are obtained by image recognition or recognition of the three-dimensional point cloud shape based on the image taken in the image.

With this configuration, even if the image is blocked, VisualSLAM technology can match the 3D information of the surroundings acquired in advance with the current camera image to estimate the current position. can be continued.

(3) In addition, in the 3D design data creation method of the present invention, in the third step, the relative positional relationship between a plurality of markers is obtained using VisualSLAM technology, and based on the relative positional relationship, It is characterized by creating TIN (Triangulated Irregular Network) data and specifying a plane.

With this configuration, it is possible to create 3D design data based on relative positional relationships between multiple markers obtained using VisualSLAM technology.

(4) In addition, in the 3D design data creation method of the present invention, in the first step, a marker pillar having three or more markers is installed at a location where height information is required within or outside the construction range, The second step is characterized in that the marker column is photographed by the camera.

With this configuration, it is possible to specify a plane from the position coordinates of each marker pillar and the position coordinates of the camera, or from the position coordinates of each marker pillar and create 3D design data.

(5) Further, in the 3D design data creation method of the present invention, in the third step, if there is one marker pillar photographed by the camera, the lower end of the marker pillar and the position coordinates of the camera are converted into The plane is identified by a straight line connecting the position of the work tool of the construction machine and a horizontal line perpendicular to this straight line and passing through the lower ends of the marker pillars, and when there are two marker pillars photographed by the camera. , where the plane is specified by three points consisting of the lower end of each marker pillar and the position of the working tool of the construction machine converted from the position coordinates of the camera, and the number of marker pillars photographed by the camera is three. is characterized in that the plane is specified by three points consisting of the lower end of each marker column.

With this configuration, it is possible to specify a plane according to the number of marker columns and create 3D design data.

(6) Further, in the 3D design data creation method of the present invention, a fourth step of acquiring posture information of the camera in a three-dimensional space based on the image of the marker column photographed by the camera; The method further includes a fifth step of converting the posture information of the camera in the above into the posture information of the construction machine to specify the construction target.

With this configuration, it is possible to obtain the posture information of the camera in a three-dimensional space such as roll, pitch, and yaw based on the image of the marker post captured by the camera. By converting the posture information of the camera into the posture information of the construction machine, it becomes possible to specify the work target of the work tool provided on the construction machine. This makes it possible to realize effective information-aided construction.

(7) Further, the construction work method of the present invention transmits 3D design data created by the 3D design data creation method according to any one of (1) to (6) above and the position coordinates of the camera to the construction machine. and the work tool of the construction machine operates based on the 3D design data.

With this configuration, it is possible to realize machine guidance simply and inexpensively using only a camera and a computer for calculation. According to the present invention, there is no need to create 3D design data in advance, and construction results can be immediately reflected in the next construction, making it possible to perform highly accurate construction.

(8) Further, the 3D design data creation system of the present invention is a 3D design data creation system for creating 3D design data as a construction target, which is mounted on a construction machine, and during running of the construction machine, a construction range A single or a plurality of cameras for photographing markers installed inside or outside the construction range where height information is required, and from the image of the marker photographed by the camera, the position coordinates of the marker in the image and the 3D for creating 3D design data by specifying a plane from the position coordinates of at least one of the markers and the position coordinates of the camera, or from the position coordinates of the plurality of markers; and a design data creation unit.

With this configuration, there is no need to create 3D design data in advance, and the result of construction can be immediately reflected in the next construction, making it possible to perform highly accurate construction. Moreover, even if the operator of the construction machine does not have sufficient knowledge and experience, it is possible to easily perform the desired construction. In TS (Total Station), the TS and prism may be shielded during construction, and measurement may be interrupted. and errors due to multipath, the present invention is less subject to these restrictions. ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to implement|achieve information-aided construction with high precision and high efficiency.

(9) Further, a 3D design data creation program of the present invention is a 3D design data creation program for creating 3D design data as a construction target based on images captured by a camera mounted on a construction machine. While the machine is running, a process of photographing markers installed at locations where height information is required inside or outside the construction area with a single or a plurality of cameras mounted on the construction machine; and A process of acquiring the position coordinates of the marker and the position coordinates of the camera in the captured image of the marker, and the position coordinates of at least one of the markers and the position coordinates of the camera, or the positions of a plurality of the markers. and a process of creating 3D design data by specifying a plane from the coordinates.

With this configuration, there is no need to create 3D design data in advance, and the result of construction can be immediately reflected in the next construction, making it possible to perform highly accurate construction. Moreover, even if the operator of the construction machine does not have sufficient knowledge and experience, it is possible to easily perform the desired construction. In TS (Total Station), the TS and prism may be shielded during construction, and measurement may be interrupted. and errors due to multipath, the present invention is less subject to these restrictions. ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to implement|achieve information-aided construction with high precision and high efficiency.

According to the present invention, there is no need to create 3D design data in advance, and construction results can be immediately reflected in the next construction, making it possible to perform highly accurate construction.

1 is a diagram showing a schematic configuration of a 3D design data creation system according to this embodiment; FIG. 1 is a block diagram of a 3D design data creation system according to this embodiment; FIG. It is a figure which shows one form of the marker used in the 3D design data creation system which concerns on this embodiment. FIG. 5 is a diagram showing another form of markers used in the 3D design data creation system according to this embodiment; FIG. 2 is a diagram showing an example of a construction range when constructing a parking lot of a convenience store; FIG. 3 is a cross-sectional view of a newly paved range in FIG. 2 ; FIG. 3 is a diagram showing a state in which markers are placed at points ABCD in FIG. 2; FIG. 5 is a diagram showing how a TIN (Triangulated Irregular Network) is created based on the position coordinates of the markers in FIG. 4; 6 is a cross-sectional view showing the relationship between the design height Z by TIN in the local point group coordinates obtained by VisualSLAM in FIG. 5, the camera coordinates z, and the height difference h between the camera and the lower end of the work tool. FIG. 4 is a flowchart of operation example 1 of the 3D design data creation system according to embodiment 1; 9 is a flow chart of operation example 2 of the 3D design data creation system according to embodiment 1; 9 is a flowchart of an operation example 3 of the 3D design data creation system according to the embodiment 1; FIG. 4 is a diagram showing an image display example used for detecting the relative position between a camera and an object (marker) from an image captured by the camera; FIG. 11 shows the diameter of the marker; FIG. 10 is a diagram showing the appearance of a marker post M14 having five markers M3; FIG. 10 is a diagram showing a state in which marker pillars M14 are installed at a predetermined height at characteristic points inside and outside the construction site; It is a part of plan view of the construction range. It is a front view of a part of construction range. It is a side view of a part of construction range. It is a part of plan view of the construction range. It is a front view of a part of construction range. It is a side view of a part of construction range. It is a part of plan view of the construction range. It is a front view of a part of construction range. It is a side view of a part of construction range. FIG. 10 is a diagram showing water thread knot points within a construction range in an example of a conventional construction method for constructing a roadbed without using ICT. FIG. 3 is a diagram showing a cross-sectional view of a construction range having a roadbed 160b on the upper layer of a roadbed 160a; It is a figure which shows an example of the construction construction method at the time of constructing a roadbed using the conventional ICT. FIG. 3 is a diagram showing a cross-sectional view of a construction range having a roadbed 170b on the upper layer of a roadbed 170a;

The inventors of the present invention have focused on the fact that it has traditionally required a great deal of time and effort to create 3D design data before construction. The inventors have found that 3D design data can be created in real time based on the positional information of the construction machine and the marker by photographing the marker, leading to the present invention.

That is, the 3D design data creation method of the present invention is a 3D design data creation method for creating 3D design data as a construction target based on an image captured by a camera mounted on a construction machine. a first step of photographing a marker installed at a location where height information is required inside or outside the construction area with a single or a plurality of cameras mounted on the construction machine, and photographing with the camera. a second step of obtaining the position coordinates of the markers in the image and the position coordinates of the camera from the obtained image of the markers; and a third step of creating 3D design data by specifying a plane from the position coordinates.

As a result, the inventors of the present invention have eliminated the need to create 3D design data in advance, and have made it possible to immediately reflect the results of construction in the next construction, thereby performing highly accurate construction. Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. In this specification, "3D design data" means not only data created in advance in the design stage, but also data generated in real time on site in the construction stage and used as construction targets.

FIG. 1A is a diagram showing a schematic configuration of a 3D design data creation system according to this embodiment. In the 3D design data creation system 1, the construction machine 10 is one of the general machines used for construction, which directly changes the shape of a construction object such as land or a building. In this embodiment, the construction machine 10 is assumed to be a bulldozer or a hydraulic excavator. A construction machine 10 is provided with one or more cameras 12 and a computer 14 . A monocular camera, a stereo camera, a depth camera, or the like can be applied to the camera 12 . It is assumed that the computer 14 is installed with an application for processing data, which will be described later. A tilt sensor, a gyro sensor, or the like can be used to monitor the movement of the work implement (blade) of the construction machine 10 .

The 3D design data creation system according to this embodiment is a so-called "standalone type", and can complete data processing by itself. On the other hand, as a "client-server type", communication is performed with a server 18 via an access point 16 and a network 17, an image captured by the camera 12 is transmitted to the server 18 via the network 17, and data processing is performed in the server 18. It is also possible to employ a configuration in which the processed data is returned to the computer 14. FIG. By adopting the “client-server type”, even if the amount of data to be processed becomes large, the data can be processed by the server 18 and only the processing result can be transmitted to the computer 14 . According to this embodiment, machine guidance can be performed simply and inexpensively because the only necessary equipment is a camera and a computer.

FIG. 1B is a block diagram of the 3D design data creation system according to this embodiment. As shown in FIG. 1B, the 3D design data creation system according to the present embodiment uses a single or a plurality of cameras 12 mounted on the construction machine 10 to capture images within the construction area or within the construction area while the construction machine 10 is running. Take a picture of a marker (described later) installed at a place where outside height information is required. Further, the position coordinates of the marker in the image and the position coordinates of the camera 12 are obtained from the image of the marker captured by the camera 12 by the position coordinate obtaining unit 14a. The 3D design data creating unit 14b creates 3D design data by specifying a plane from the position coordinates of at least one marker and the position coordinates of the camera 12, or from the position coordinates of a plurality of markers. The output unit 14c is an output interface such as a display and a speaker, and can be configured to display an image after data processing on the display, or to guide the location to be constructed by voice. In addition, each component includes various devices for exhibiting functions, such as processor units (CPU, etc.), storage devices (RAM, ROM, flash memory, magnetic disks, etc.), input/output devices (transmitting and receiving data, signals, etc.). interface, etc.) as appropriate. A program for realizing the function of each component may be stored in the storage device. Hardware (such as an electronic device) that constitutes each component is configured to be able to exchange data and the like with other hardware by wire or wirelessly. This allows machine guidance to be implemented.

FIG. 1C is a diagram showing one form of markers used in the 3D design data creation system according to this embodiment. Here, a configuration is adopted in which a single spherical marker M1 is supported by a tripod M10. FIG. 1D is a diagram showing another form of markers used in the 3D design data creation system according to this embodiment. Here, a configuration is adopted in which five markers M2 are provided on the board M12. Like the marker M1 and the board M12 having five markers M2, the markers to be photographed by the camera do not exist in the surrounding environment and must have a distinctive configuration that allows them to be clearly distinguished. Next, an example of the 3D design data creation system according to this embodiment will be described.

FIG. 2 is a diagram showing an example of a construction range when constructing a small-scale parking lot. As shown in FIG. 2, the area surrounded by four points ABCD is constructed. Let the coordinates of point A be (x, y, z) = (0, 0, 0), the coordinates of point B be (x, y, z) = (0, 10, 0), and the coordinates of point C be ( Let x, y, z)=(10, 10, 1) and the coordinates of point D be (x, y, z)=(10, 0, 1). Assume that the xz plane has a +10 percent slope in the direction from point A to point D, as shown in FIG.

FIG. 3 is a cross-sectional view of the newly paved range in FIG. In the pavement range 20 to be newly constructed, for example, the roadbed 20b is formed on the upper layer of the roadbed 20a. The roadbed 20b is configured to have a thickness of, for example, 100 mm. A pavement 20c made of asphalt mixture is provided on the roadbed 20b. The pavement 20c is configured to have a thickness of, for example, 50mm. In a small parking lot or the like, as shown in FIG. 3, in the vicinity of a new pavement area 20 there are existing pavements 21 such as store buildings and sidewalks that serve as entry points, and existing structures such as curbs. For this reason, the parking lot pavement to be constructed must be constructed so as not to create a level difference with the existing pavement or existing structure that has been completed in advance. In addition, the existing pavement or existing structures may not match the drawings due to deformation due to aging, so it may not be possible to construct according to the drawings. According to the 3D design data creation system according to the present embodiment, since 3D design data can be created in real time, there is no need to prepare 3D design data in advance, and such construction needs can be met. It is possible.

FIG. 4 is a diagram showing a state in which markers are placed at the positions of points ABCD in FIG. That is, a marker M is placed at a location where height information is required. No design data is required here, nor is there a need for pre-survey. The height of each marker M is "offset" by an arbitrary length from the finished height (height of the paved surface in the final finish). Here, with an offset of 1 m, the coordinates of each marker M are (x, y, z)=(0, 0, 1) at point A' and (x, y, z) at point B'. = (0, 10, 1), at point C' (x, y, z) = (10, 10, 2), and at point D' (x, y, z) = (10, 0, 1) becomes.

Next, as shown in FIG. 4, the construction machine 10 on which the camera is installed is run. Then, the VisualSLAM technology is used to acquire the positional coordinates of the camera and the three-dimensional information of the surroundings. These coordinates are "local coordinates", and the initial position and direction of the camera differ depending on the analysis conditions. As for the VisualSLAM technology, it is possible to use the technology disclosed in JP-A-2009-237848, JP-A-2020-160594, and International Publication WO2012/106068. Also, the installed marker M is recognized and extracted from the image or the shape of the three-dimensional point group, and the position coordinates of the marker are separately stored. Furthermore, by observing the entire surroundings, for example, three-dimensional information inside and outside the construction area such as buildings J and trees and position information of the marker M are acquired.

FIG. 5 is a diagram showing how a TIN (Triangulated Irregular Network) is created based on the positional coordinates of the markers in FIG. That is, a TIN consisting of a marker M and two markers M adjacent to the marker M is created. This TIN is higher than the finished pavement height by the offset (1 m in this case) when the marker M is installed. After creating the TIN, using the TIN and the position information of the camera, the difference in height between the TIN and the camera is calculated, and displayed to the operator so that the lower end of the work tool of the construction machine 10 is at the design height, or automatically control. For example, let "Z" be the design height by TIN in the local point cloud coordinates obtained by VisualSLAM, "z" be the camera coordinates, and "h" be the height difference between the camera and the lower end of the work implement. Here, "h" varies depending on the vertical movement of the work tool, but by installing a tilt sensor or a gyrometer on the work tool, it is possible to grasp the amount of change from the initial state "h0". It is possible.

FIG. 6 is a sectional view showing the relationship between the design height Z by TIN in the local point group coordinates obtained by VisualSLAM in FIG. 5, the camera coordinates z, and the height difference h between the camera and the lower end of the work tool. For example, when constructing a roadbed, the construction machine 10 is controlled so as to satisfy the following equation.
Z−(1+0.05)=zh

Next, the operation of the 3D design data creation system according to the first embodiment configured as described above will be described.
[Operation example 1]
FIG. 7 is a flowchart of operation example 1 of the 3D design data creation system according to the first embodiment. First, the camera 12 is attached to the construction machine 10 (step S01), and a sensor for detecting vertical movement of the work tool of the construction machine 10 is installed (step S02). Next, the initial distance "h0" between the camera 12 and the cutting edge of the work tool is measured (step S03), and a marker M is placed at a location within or outside the work area where height information is required (step S04). . Here, the height of the marker M is offset by an arbitrary height from the design surface (construction surface). In Example 1, the offset is 1 m.

Next, while the construction machine 10 is running, the inside and outside of the construction area are photographed by the camera 12 (step S05). Next, using VisualSLAM, the self position coordinates (the position coordinates of the camera 12) are acquired and stored (step S06), and the surrounding three-dimensional coordinates are acquired and stored (step S07). Next, the marker M in the image is extracted from the image or the three-dimensional point cloud data acquired by the camera 12 (step S08), and the three-dimensional coordinate information of the marker M is acquired and stored (step S09). After that, a TIN consisting of each marker M is created (step S10).

Next, based on the data obtained from steps S06 and S10, based on the current self-position coordinates, the TIN consisting of each marker M, the offset amount of the marker M, the initial distance h0, and the vertical movement amount of the work tool, A height difference between the design surface of the construction layer and the cutting edge of the work tool is calculated (step S11). Finally, the height difference calculated in step S11 is transmitted to the operator of the construction machine 10 (step S12). Here, it is possible to display on the display of the computer 14 or to give voice guidance from the speaker. Further, in step S12, the working implement may be automatically controlled so that the height difference is 0 (step S12).

[Operation example 2]
FIG. 8 is a flowchart of an operation example 2 of the 3D design data creation system according to the first embodiment. First, the camera 12 is attached to the work tool of the construction machine 10 (step S13), and the initial distance h0 between the camera 12 and the cutting edge of the work tool is measured (step S14). In addition, a marker M is installed at a location within the construction area or outside the construction area where height information is required (step S15). Here, the height of the marker M is offset by an arbitrary height from the design surface (construction surface). In Example 1, the offset is 1 m.

Next, while the construction machine 10 is running, the inside and outside of the construction area are photographed by the camera 12 (step S16). Next, using VisualSLAM, the self position coordinates (the position coordinates of the camera 12) are acquired and stored (step S17), and the surrounding three-dimensional coordinates are acquired and stored (step S18). Next, the marker M in the image is extracted from the image or the three-dimensional point cloud data acquired by the camera 12 (step S19), and the three-dimensional coordinate information of the marker M is acquired and stored (step S20). After that, a TIN consisting of each marker M is created (step S21).

Next, based on the data obtained in steps S17 and S21, the design plane of the construction layer and the work The height difference of the cutting edge of the tool is calculated (step S22). Finally, the height difference calculated in step S22 is transmitted to the operator of the construction machine 10 (step S23). Here, it is possible to display on the display of the computer 14 or to give voice guidance from the speaker. Further, in step S23, the working implement may be automatically controlled so that the height difference is 0 (step S23).

[Operation Example 3]
FIG. 9 is a flowchart of operation example 3 of the 3D design data creation system according to the first embodiment. First, the camera 12 is attached to the construction machine 10 (step S24). Next, the initial distance "h0" between the camera 12 and the cutting edge of the work tool is measured (step S25). Also, a marker M is installed at a location within or outside the construction area where height information is required (step S26). Here, the height of the marker M is offset by an arbitrary height from the design surface (construction surface). In Example 1, the offset is 1 m. Furthermore, the marker B is installed on the work tool (step S27).

Next, while the construction machine 10 is running, the inside and outside of the construction area are photographed by the camera 12 (step S28). Next, using VisualSLAM, the self position coordinates (the position coordinates of the camera 12) are acquired and stored (step S29), and the surrounding three-dimensional coordinates are acquired and stored (step S30). Next, the marker M and the marker B in the image are extracted from the image or the three-dimensional point cloud data acquired by the camera 12 (step S31), and the three-dimensional coordinate information of the marker M is acquired and stored (step S32). . After that, a TIN consisting of the marker M is created (step S33). Also, in step S31, based on the position coordinates of the marker B, the relative position between the camera 12 and the work implement is calculated and stored (step S34).

Next, based on the data obtained in steps S29, S33, and S34, the current self-position coordinates, the TIN consisting of the marker M, the offset amount of the marker M, the initial distance h0, the relative distance between the camera 12 and the work implement Based on the target position, the height difference between the design surface of the construction layer and the cutting edge of the work tool is calculated (step S35). Finally, the height difference calculated in step S35 is transmitted to the operator of the construction machine 10 (step S36). Here, it is possible to display on the display of the computer 14 or to give voice guidance from the speaker. Further, in step S12, the work implement may be automatically controlled so that the height difference is 0 (step S36).

As described above, according to the first embodiment, unlike TS and GNSS, there are fewer restrictions such as the environment and conditions in the construction range. In addition, with TS, the space between the TS and the prism may be blocked during operation, and the measurement may be interrupted. Path error may occur. In contrast, according to Example 1, even after the image is blocked, the SLAM can match the 3D information of the surroundings acquired in advance with the current camera image, and can estimate the current position, so construction can be continued. is.

In the second embodiment, the camera 12 mounted on the construction machine 10 photographs the marker pillars arranged inside and outside the construction area. Three or more markers are placed on the marker pillar, and the positions and sizes of the markers are obtained from the images taken by the camera. Calculate the relationships (x, y, z, roll, pitch, yaw). Marker pillars to be installed inside and outside the construction area are set at positions higher than the target height of construction by a predetermined height. The color of the marker may be changed according to the installation height so that it can be identified by the camera.

Thereby, the difference between the shooting position of the camera and the target height can be calculated, and the construction machine 10 can be appropriately steered or controlled based on the calculation result.

[Principle of calculating the relative position of the camera and the marker column]
FIG. 10A shows an image display example used for detecting the relative position between the camera and the object (marker) from the image captured by the camera. FIG. 10B is a diagram showing the diameter of the marker. As shown in FIG. 10A, an object is detected from the image (a sphere is detected in the case of FIG. 10A), and from the size information on the image, the relative distance between the camera and the object and the xy coordinates of the object on the image are obtained. It is possible.

FIG. 11 is a diagram showing the appearance of a marker post M14 having five markers M3. As shown in FIG. 11, a marker pillar M14 is prepared on which three or more circular markers M3 are arranged (in this case, five markers are arranged). The relative positional relationship of each marker M3 within the marker column M14 is measured in advance. When the camera 12 photographs the marker pillar M14, the relative position of the camera 12 from the marker pillar M14 is obtained from the relative distance between the camera 12 and each marker M3 arranged on the marker pillar 14. FIG. When the camera 12 is arranged in a three-dimensional space, there are six-dimensional unknowns of the (x, y, z) position and camera direction (yaw, roll, pitch). Marker M3 should be used.

FIG. 12 is a diagram showing how the marker pillars M14 are installed to have a predetermined height at characteristic points inside and outside the construction site. This "characteristic point" means the periphery of the site and the change point of the gradient. For feature points, the xy coordinates in the field coordinate system may be unknown. As for the z-coordinate, it is arranged at a predetermined height from the finished surface of construction. For example, the lower end of the marker column M14 can be installed so that it is on the finished surface, or it can be installed at a position 10 cm higher than the finished surface. If the direction of the marker pillar M14 is determined in the field coordinate system, for example, by setting the marker pillar M14 vertically, the relative direction (yaw, roll, pitch) between the marker pillar M14 and the camera 12 is determined in the field coordinate system. The orientation of camera 12 is determined. Knowing the direction of the camera 12 in the site coordinate system means knowing the posture of the construction machine 10 to which the camera is attached. That is, it becomes clear whether the construction machine 10 is oriented upward or downward with respect to the horizontal plane of the site.

[How to create 3D design data in real time]
The method of creating 3D design data differs depending on the number of marker pillars captured by the camera.

[When the number of marker pillars captured by the camera is 1]
13A is a plan view of part of the construction range, FIG. 13B is a front view of part of the construction range, and FIG. 13C is a side view of part of the construction range. If there is only one marker pillar M14 captured by the camera, consider a coordinate system in which the lower end of the marker pillar M14 is the point P1 (0, 0, 0). Let the position of the working tool of the construction machine converted from the camera C10 be a point C (Xc, Yc, Zc). The design surface of the construction layer is assumed to be a plane passing through the points P1 and C, but since two points cannot be determined as one plane, a straight line L passing through the point P1, orthogonal to the vector (P1C), and z = 0 is assumed. A plane including this straight line is used as 3D design data.

[When the number of marker pillars captured by the camera is 2]
14A is a plan view of part of the construction range, FIG. 14B is a front view of part of the construction range, and FIG. 14C is a side view of part of the construction range. When there are two marker pillars M14 captured by the camera, point P1 (0, 0, 0), point P2 (X2, Y2, Z2) of marker pillar M14, point C (Xc, Yc, Zc ) is used as 3D design data.

[When the number of marker pillars captured by the camera is 3]
15A is a plan view of part of the construction range, FIG. 15B is a front view of part of the construction range, and FIG. 15C is a side view of part of the construction range. When there are three marker pillars M14 captured by the camera, points P1 (0, 0, 0), points P2 (X2, Y2, Z2), and points P3 (X3, Y3, Z3) of the marker pillars M14 are Let the included plane be the 3D design data.

If the number of marker pillars M14 is four or more, it can be handled in the same way as the case where the number of marker pillars M14 is three, but there is an advantage in that the error can be distributed.

As described above, according to the second embodiment, construction can be performed with high accuracy without preparing 3D design data in advance. In addition, since the result of construction is immediately reflected in the next construction (because of real-time measurement), even an operator with little knowledge or experience can easily perform construction with high accuracy. Furthermore, unlike TS and GNSS, there are fewer restrictions such as the environment and conditions in the construction range.

1 3D design data creation system 10 construction machine 12 camera 14 computer 14a position coordinate acquisition unit 14b 3D design data creation unit 14c output unit 16 access point 17 network 18 server 20 pavement range 20a roadbed 20b roadbed 20c pavement 21 existing pavement B marker C10 Camera M Marker M1 Marker M10 Tripod M12 Board M14 Marker post M2 Marker M3 Marker

Claims (9)

A 3D design data creation method for creating 3D design data as a construction target based on an image captured by a camera mounted on a construction machine, comprising:
A first step of photographing a marker installed at a location where height information is required within or outside a construction area with a single or a plurality of cameras mounted on the construction machine while the construction machine is running;
a second step of acquiring the position coordinates of the marker in the image and the position coordinates of the camera from the image of the marker captured by the camera;
and a third step of creating 3D design data by specifying a plane from the position coordinates of at least one of the markers and the position coordinates of the camera, or the position coordinates of a plurality of the markers. How to create design data. In the second step, VisualSLAM technology is used to acquire three-dimensional information of an object existing in the position coordinates and shooting range of the camera, and image recognition or three-dimensional point recognition is performed based on the image taken by the camera. 2. The method of creating 3D design data according to claim 1, wherein the position coordinates of the markers in the image are obtained by recognizing the group shape. In the third step, a relative positional relationship between a plurality of markers is obtained using VisualSLAM technology, TIN (Triangulated Irregular Network) data is created based on the relative positional relationship, and a plane is specified. 3. The method of creating 3D design data according to claim 1, characterized by: In the first step, a marker pillar having three or more markers is installed at a location where height information is required within or outside the construction range,
2. The method of creating 3D design data according to claim 1, wherein in said second step, said marker post is photographed by said camera. In the third step, if there is one marker pillar photographed by the camera, a straight line connecting the lower end of the marker pillar and the position of the work tool of the construction machine converted from the position coordinates of the camera The plane is specified by this straight line and a horizontal line that passes through the lower end of the marker pillar. When there are two marker pillars photographed by the camera, the plane is converted from the lower end of each marker pillar and the position coordinates of the camera. The plane is specified by three points consisting of the position of the work tool of the construction machine, and when there are three marker pillars photographed by the camera, the plane is specified by three points consisting of the lower end of each marker pillar. 5. The method of creating 3D design data according to claim 4, wherein the 3D design data is specified. a fourth step of acquiring posture information of the camera in a three-dimensional space based on the image of the marker column captured by the camera;
6. The method according to any one of claims 1 to 5, further comprising a fifth step of converting the posture information of the camera in the three-dimensional space into the posture information of the construction machine to identify the construction target. 3D design data creation method. transmitting the 3D design data created by the 3D design data creation method according to any one of claims 1 to 6 and the position coordinates of the camera to the construction machine;
A construction method, wherein the work tool of the construction machine operates based on the 3D design data. A 3D design data creation system for creating 3D design data as a construction target,
a single camera or a plurality of cameras mounted on a construction machine for photographing markers installed at locations where height information is required inside or outside the construction area while the construction machine is running;
a position coordinate acquisition unit that acquires the position coordinates of the marker in the image and the position coordinates of the camera from the image of the marker captured by the camera;
a 3D design data creation unit that creates 3D design data by specifying a plane from the position coordinates of at least one of the markers and the position coordinates of the camera, or the position coordinates of a plurality of the markers. 3D design data creation system. A 3D design data creation program for creating 3D design data as a construction target based on images captured by a camera mounted on a construction machine,
While the construction machine is running, a process of photographing markers installed at locations where height information is required within or outside the construction range with a single or multiple cameras mounted on the construction machine;
a process of acquiring the position coordinates of the marker in the image and the position coordinates of the camera from the image of the marker captured by the camera;
a process of creating 3D design data by specifying a plane from the position coordinates of at least one of the markers and the position coordinates of the camera, or the position coordinates of a plurality of the markers, and causing a computer to execute 3D Design data creation program.

Images