JP6996012B1 - 3D shape estimation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional shape estimation system capable of estimating a three-dimensional shape of a soil pile formed in a work room.
SOLUTION: In a three-dimensional shape estimation system for estimating a three-dimensional shape of an earthen mountain M existing in a work chamber used in a pneumatic cason method, a relative between an excavator 100 and an object located around the excavator 100. Tsuchiyama recognizes the apex position, angle of repose, and boundary point position of the earthen mountain M from the position information based on the relative position between the excavator and the object measured by the measuring means 217 for measuring the position and the excavator and the object. It is characterized by comprising a recognition means 218.
[Selection diagram] FIG. 9

Description

The present invention relates to a three-dimensional shape estimation system.

Conventionally, the shape of the earth and sand that is the object to be loaded is measured at the excavation site and the civil engineering site in the natural environment, the work pattern of the power shovel etc. is decided based on the measurement result, and the earth and sand is loaded according to the decided work pattern. The process is unattended. For example, Patent Document 1 discloses an automatic excavator that measures a distance to an excavated object by a stereo method using parallax between two cameras.

Further, in recent years, in order to realize automation of work by a work machine, parameters related to an excavated object have also been acquired. For example, Patent Document 2 discloses a technique of calculating a distance from a wheel loader to a ground as a parameter relating to the ground and controlling a working machine so that the ground is excavated by a bucket. Parameters related to the ground include the distance from the wheel loader to the ground, the grain size of the rock soil constituting the ground, the height of the ground, the shape of the ground, and the like.

Japanese Unexamined Patent Publication No. 10-08625 JP-A-2019-178598

By the way, in the pneumatic caisson method in which the ground of the floor of the work room is excavated and the excavated earth and sand are carried out to the ground and the caisson is submerged in the ground, a remote worker remotely excavates the excavator in the work room. The excavation of the ground is advanced by operating it. However, since it is difficult to secure enough cameras in the work room to fully grasp the site environment, it is difficult to measure the shape of the earth and sand to be carried out.

Therefore, the present invention has been devised in view of the above-mentioned problems, and the purpose of the present invention is to provide a three-dimensional shape estimation system capable of estimating the three-dimensional shape of the earthen hill formed in the work room. To provide.

The three-dimensional shape estimation system to which the present invention is applied is a three-dimensional shape estimation system that estimates the three-dimensional shape of a soil pile existing in a work room used in the pneumatic cason method, and is an excavator and an object located around the excavator. The earthen hill recognition means for recognizing the apex position and the rest angle of the earthen hill from the measuring means for measuring the relative position between the two and the position information based on the relative position between the excavator and the object measured by the measuring means. The measuring means measures the relative position between the shovel and the object based on the depth image data acquired by the acquiring means. Further provided with a conversion means for converting the depth image data into three-dimensional polygon data, the measuring means is based on the angle formed by the adjacent surface of the polygon included in the three-dimensional polygon data converted by the conversion means. It is characterized by measuring the top position and rest angle of the above-mentioned earthen mountain.

According to the present invention having the above-mentioned configuration, the three-dimensional shape of the earthen hill formed in the working room can be estimated based on the apex position and the angle of repose of the earthen hill in the working room.

FIG. 1 is a vertical sectional view showing the main equipment of the pneumatic caisson method according to the embodiment of the present invention. FIG. 2 is a side view of an excavator which is an example of a working machine according to an embodiment of the present invention. FIG. 3 is a block diagram showing a control system in the excavator according to the embodiment of the present invention. FIG. 4 is a flowchart showing an example of the operation of the three-dimensional shape estimation system according to the embodiment of the present invention. 5 (a) to 5 (d) are schematic perspective views showing the shape of the earthen hill that changes with the passage of time. FIG. 6 is a diagram showing an image region divided into a plurality of regions. FIG. 7 is a diagram showing a divided image area including height information. FIG. 8 is a diagram showing vertices and boundaries in the image area. 9 (a) to 9 (d) are views showing the angle of repose of Tsuchiyama.

Three-dimensional shape estimation system Hereinafter, the three-dimensional shape estimation system to which the present invention is applied will be described in detail with reference to the drawings.

FIG. 1 is a diagram showing an example of a main facility of a pneumatic caisson method in which an excavator, which is an example of a working machine according to the present invention, is used. In the pneumatic caisson method, the reinforced concrete caisson 1 is submerged underground using the excavation equipment E1, the equipment equipment E2, the soil removal equipment E3, the air supply equipment E4, and the reserve / safety equipment E5. Build a structure.

The excavation equipment E1 includes, for example, an excavator 100 (hereinafter referred to as a caisson excavator 100), an automatic earth and sand loading device 11, and a ground remote control room 13. The caisson excavator 100 is installed in the work room 2 provided at the bottom of the caisson 1. The earth and sand automatic loading device 11 loads the earth and sand excavated by the caisson excavator 100 into the cylindrical earth bucket 31. The ground remote control room 13 includes a remote control device 12 that remotely controls the operation of the caisson excavator 100 from the ground.

The fitting equipment E2 includes, for example, a man shaft 21, a man lock 22 (airlock), a material shaft 23, and a material lock 24 (airlock). The man shaft 21 is a cylindrical passage connecting the ground and the work room 2 for an operator to enter and exit the work room 2, and is provided with, for example, a spiral staircase 25. The man lock 22 is an airtight door having a double door structure provided on the man shaft 21 to adjust the atmospheric pressure on the ground and the pressure difference in the work room 2. The material shaft 23 is a cylindrical passage connecting the ground and the work room 2 in order to carry the earth bucket 31 on which the earth and sand are loaded by the earth and sand automatic loading device 11 to the ground. The material lock 24 is an airtight door having a double door structure that adjusts the atmospheric pressure on the ground and the pressure difference in the work room 2 provided on the material shaft 23 for loading and unloading materials and the like. The man lock 22 and the material lock 24 are configured so that the worker and the earth bucket 31 can move in and out of the work room 2 by suppressing the change in the air pressure in the work room 2. On the excavated ground G near the automatic earth and sand loading device 11 in the work room 2, the earth and sand excavated by the caisson excavator 100 are sequentially piled up to form a soil mountain M.

The soil removal equipment E3 includes, for example, an earth bucket 31, a carrier device 32, and a soil hopper 33. The earth bucket 31 is a bottomed cylindrical container in which earth and sand excavated by the caisson excavator 100 are loaded. The carrier device 32 is a device that pulls the earth bucket 31 up to the ground via the material shaft 23 and carries it out. The earth and sand hopper 33 is a facility for temporarily storing the earth and sand carried to the ground by the earth bucket 31 and the carrier device 32.

The air supply equipment E4 includes, for example, an air compressor 42, an air purifier 43, an air supply pressure adjusting device 44, and an automatic depressurizing device 45. The air compressor 42 is a device that sends compressed air into the working room 2 through the air supply passage 3 formed in the air supply pipe 41 and the caisson 1. The air purifying device 43 is a device that purifies the compressed air sent by the air compressor 42. The air supply pressure adjusting device 44 is a device that adjusts the amount (pressure) of the compressed air sent from the air compressor 42 into the working room 2 so that the air pressure in the working room 2 becomes substantially equal to the ground water pressure. The automatic decompression device 45 is a device that depressurizes the air pressure in the man lock 22.

The spare / safety equipment E5 includes, for example, an emergency air compressor 51 and a hospital lock 53. The emergency air compressor 51 is a device capable of sending compressed air into the work room 2 in place of the air compressor 42 when the air compressor 42 fails or is inspected. The hospital lock 53 is a decompression chamber for a worker who has worked in the work room 2 to enter and gradually acclimatize the worker's body to atmospheric pressure.

Next, the caisson excavator 100 according to the present invention will be described with reference to FIGS. 2 to 3. As shown in FIG. 2, the caisson excavator 100 includes, for example, a traveling body 110, a boom 130, and a bucket attachment 150. The traveling body 110 is attached to a pair of left and right traveling rails 4 provided on the ceiling of the work room 2, and travels along the traveling rails 4 while being suspended from the left and right traveling rails 4. The boom 130 is pivotally connected to the turning frame 121 of the traveling body 110 so as to be swingable in the vertical direction. The bucket attachment 150 is attached to the tip of the boom 130.

The traveling body 110 includes a traveling frame 111, a turning frame 121, and a traveling roller 113. The swivel frame 121 is provided on the lower surface side of the traveling frame 111 so as to be swivelable. The traveling roller 113 is four rollers on the front, rear, left, and right provided on the front and rear on the upper surface side of the traveling frame 111. The traveling body 110 is configured to rotate and drive the traveling rollers 113 in the front-rear and left-right directions to travel along the left and right traveling rails 4.

The boom 130 includes, for example, a base end boom 131, a tip end boom 132, a telescopic cylinder 133, and an undulating cylinder 134. The base end boom 131 is attached to the swivel frame 121 so as to be undulating (swingable in the vertical direction). The tip boom 132 is nested and configured with the proximal boom 131. The telescopic cylinder 133 is provided in the base end boom 131. Two undulating cylinders 134 are provided on the left and right sides of the base end boom 131. The boom 130 is configured such that when the telescopic cylinder 133 is expanded and contracted, the tip boom 132 moves in the longitudinal direction with respect to the proximal boom 131, whereby the boom 130 expands and contracts. The base ends of the two undulating cylinders 134 are rotatably attached to the left and right sides of the base end boom 131, respectively.

The bucket attachment 150 includes a base member 151, a bucket 152, and a bucket cylinder 153. The base member 151 is attached to the tip boom 132. The bucket 152 is attached to the tip of the base member 151 so as to be vertically swingable. The bucket cylinder 153 is configured to swing the bucket 152 up and down with respect to the base member 151.

As shown in FIG. 3, the control unit 165 includes a main controller 165a, a traveling body controller 165b, and a boom / bucket controller 165c. The main controller 165a receives an operation signal from the remote control device 12 and outputs a drive control signal corresponding to the operation signal. The vehicle controller 165b is configured to drive the vehicle 110 in response to a drive control signal output from the main controller 165a. The main controller 165a and the traveling body controller 165b are arranged on the turning frame 121 of the traveling body 110. The boom / bucket controller 165c is configured to drive the boom 130 and the bucket attachment 150 in response to a drive control signal output from the main controller 165a. The boom bucket controller 165c is arranged on the side of the base end boom 131 of the boom 130.

As shown in FIG. 3, the cason excavator 100 includes, for example, a traveling body position sensor 201, a turning angle sensor 202, a boom undulation angle sensor 203, a boom extension amount sensor 204, a bucket swing angle sensor 205, and the outside world. It is equipped with a sensor 206. The traveling body position sensor 201 detects the position of the traveling body 110 on the traveling rail 4. The turning angle of the turning frame 121 with respect to the turning angle sensor 202 and the traveling frame 111 is detected. The boom undulation angle sensor 203 detects the undulation angle of the boom 130 with respect to the swivel frame 121. The boom extension amount sensor 204 detects the extension amount of the boom 130. The bucket swing angle sensor 205 detects the swing angle of the bucket 152 with respect to the boom 130 (base member 151 of the bucket attachment 150). The outside world sensor 206 is provided on the traveling body 110 and acquires information such as the distance to the excavated ground G in the work room 2 and the shape of the excavated ground G.

The traveling body position sensor 201 is composed of, for example, a laser sensor arranged on the traveling frame 111 of the traveling body 110. The traveling body position sensor 201 irradiates the laser beam toward the end of the traveling rail 4 (or the wall of the working room 2) and reflects the laser light at the end of the traveling rail 4 (or the wall of the working room 2). Measure the time it takes to return. The traveling body position sensor 201 detects the distance from the end portion of the traveling rail 4 (or the wall portion of the work room 2) to the traveling body 110 based on this time. The turning angle sensor 202 is composed of, for example, an optical rotary encoder arranged on a turning frame 121 of the traveling body 110. The turning angle sensor 202 converts the turning amount of the turning frame 121 with respect to the traveling frame 111 into an electric signal. The turning angle sensor 202 calculates and processes the signal to detect the turning angle (turning direction and position) of the turning frame 121. The traveling body position sensor 201 and the turning angle sensor 202 have been described as an example, and other sensors for detecting the two-dimensional position of the traveling body and other sensors for detecting the turning angle of the turning frame 121 are used, respectively. You may.

The boom undulation angle sensor 203 is composed of, for example, a laser sensor arranged on the side of the cylinder bottom of the undulation cylinder 134. The boom undulation angle sensor 203 irradiates the laser beam toward the swivel frame 121 and measures the time until the laser beam is reflected by the swivel frame 121 and returned. The boom undulation angle sensor 203 detects the extension amount of the undulation cylinder 134 based on this time, and detects the undulation angle (undulation position) of the boom 130 with respect to the swivel frame 121 based on the extension amount of the undulation cylinder 134. The boom undulation angle sensor 203 also describes an example, and another sensor that directly detects the undulation angle of the boom 130 by an optical rotary encoder, a potentiometer, or the like may be used.

The boom extension amount sensor 204 is composed of, for example, a laser sensor disposed on the base end boom 131 of the boom 130. The boom extension amount sensor 204 irradiates a laser beam toward the base member 151 of the bucket attachment 150 attached to the tip of the tip boom 132, and measures the time until the laser beam is reflected by the base member 151 and returned. The boom extension amount sensor 204 detects the extension amount of the boom 130 (the extension amount of the tip boom 132 with respect to the proximal boom 131) based on this time. The boom extension amount sensor 204 also describes an example, and another sensor that directly measures the extension amount of the cable that expands and contracts with the expansion and contraction of the boom may be used.

The bucket swing angle sensor 205 is composed of, for example, a flow rate sensor arranged in an oil passage of the bucket cylinder 153. The bucket swing angle sensor 205 detects the flow rate of the hydraulic oil supplied to the bucket cylinder 153 and calculates the integrated value of the flow rate. The bucket swing angle sensor 205 obtains the extension amount of the piston rod of the bucket cylinder 153 based on this flow rate integrated value, and based on the extension amount of the bucket cylinder 153, the extension amount with respect to the base member 151 (boom 130) of the bucket attachment 150. The swing angle (swing position) of the bucket 152 is detected. The bucket swing angle sensor 205 is also an example, and the swing angle of the bucket 152 is directly detected by an optical rotary encoder, a potentiometer, or the like, or another sensor that obtains the extension amount of the bucket cylinder 153 by a laser sensor. A sensor may be used.

The outside world sensor 206 is composed of, for example, an RGB-D sensor arranged on the turning frame 121 of the traveling body 110. The outside world sensor 206 acquires RGB images (color images) and distance images of the excavated ground G, and acquires distance information to the excavated ground G and shape information of the excavated ground G based on these images. As another example of the RGB-D sensor, the external world sensor 206 may use a stereo camera, an ultrasonic rangefinder, a laser sensor, or the like.

Information detected by the traveling body position sensor 201, the turning angle sensor 202, the boom undulation angle sensor 203, the boom extension amount sensor 204, the bucket swing angle sensor 205, and the outside world sensor 206 is transmitted to the main controller 165a of the control unit 165. Will be sent. The main controller 165a includes a traveling body position measuring unit 211, a bucket position measuring unit 212, a ground shape measuring unit 213, an image segmenting unit 214, an acquisition unit 215, a conversion unit 216, a measuring unit 217, and Tsuchiyama recognition. A unit 218 and a difference detection unit 219 are provided. Further, the three-dimensional shape estimation system 200 according to the present embodiment includes an external world sensor 206, an image segmentation unit 214, an acquisition unit 215, a conversion unit 216, a measurement unit 217, a Tsuchiyama recognition unit 218, and a difference detection unit. It is configured to include 219.

The traveling body position measuring unit 211 includes distance information from the end portion of the traveling body 4 (or the wall portion of the working room 2) detected by the traveling body position sensor 201 to the traveling body 110, and the traveling body position 4 is the working room 2. Information on where the traveling rail is provided (this information is information on the traveling rail 4 to which the traveling body 110 is attached, and when the traveling body 110 is attached, the traveling body position measuring unit is used. (Set to 211) is used to calculate where the traveling body 110 is located in the work room 2. Further, even if the two-dimensional position (position including the direction of the traveling body 110) in the ceiling of the traveling body 110 is detected by detecting the distance information detected by the traveling body position sensor 201 for a plurality of surrounding locations. good.

The bucket position measuring unit 212 determines the turning angle (turning direction and position) of the turning frame 121 with respect to the traveling frame 111 detected by the turning angle sensor 202, and the undulation of the boom 130 with respect to the turning frame 121 detected by the boom undulating angle sensor 203. Using the angle (undulation position), the extension amount of the boom 130 detected by the boom extension amount sensor 204, and the swing angle (swing position) of the bucket 152 with respect to the boom 130 detected by the bucket swing angle sensor 205. The position of the bucket 152 with respect to the traveling frame 111 of the traveling body 110 is calculated.

The ground shape measuring unit 213 determines the position of the traveling body 110 in the work room 2 determined by the traveling body position measuring unit 211, and the turning angle (turning) of the turning frame 121 with respect to the traveling frame 111 detected by the turning angle sensor 202. Direction and position), the position of the outside world sensor 206 provided on the swivel frame 121, the direction in which the distance information is acquired by the outside world sensor 206, and the position of the excavated ground G in which the distance information is acquired by the outside world sensor 206. Is calculated.

The image segmentation unit 214 divides an RGB image or the like of the excavated ground G acquired by the outside world sensor 206. For example, the images of the excavated ground G and the earthen mountain M in the work room 2 are divided into a plurality of grid-like image regions on the xy plane.

The acquisition unit 215 acquires the depth image data of the earthen mountain acquired by the outside world sensor 206. Depth image data is data in which multiple three-dimensional relationships such as distance information, angle information, and coordinate information from a measurement point to an object are regularly recorded. For example, it is composed of a large number of pixels arranged in the vertical direction and the horizontal direction, and in detail, it is a set of data corresponding to a large number of pixels. The depth image data may include depth information for each pixel.

The conversion unit 216 converts the depth image data into any of DEM (Digital Elevation Model) data, three-dimensional polygon data, and point cloud data. Specifically, the conversion unit 216 converts the depth image data acquired by the acquisition unit 215 into three-dimensional data such as DEM data, three-dimensional polygon data, and point cloud data. The DEM data contains height information for each compartment defined by a mesh at regular intervals. The three-dimensional polygon data is data having, for example, position information of a region represented by a plurality of points shown on three-dimensional coordinates and information associated with the region. The point cloud data is, for example, three-dimensional data obtained by scanning an earthen mountain with a laser.

The measuring unit 217 measures the relative position between the caisson excavator 100 and the object based on the depth image data acquired by the acquisition unit 215. Specifically, the measurement unit 217 performs a comparison process of height information based on the three-dimensional coordinates included in the point cloud data among the DEM data, the three-dimensional polygon data, and the point cloud data converted by the conversion unit 216, and the third order. The three-dimensional shape of Tsuchiyama M is measured in real time based on either the detection process of the convex data in the adjacent mesh included in the original polygon data or the detection process of the convex portion included in the DEM data.

The Tsuchiyama recognition unit 218 recognizes the apex position, the angle of repose, and the boundary point position of the Tsuchiyama M from the position information based on the relative position between the caisson excavator 100 measured by the measurement unit 217 and the Tsuchiyama M which is an example of the object. ..

The difference detection unit 219 is the second image captured after the first image data and the first image data are captured among the plurality of image data of Tsuchiyama M continuously imaged by the external sensor 206 in time. Difference from data Detects data. The shape of the earthen mountain M changes due to the accumulation of earth and sand excavated by the caisson excavator 100. The difference detection unit 219 compares the image data of the Tsuchiyama M before the shape change with the image data of the Tsuchiyama M after the shape change, and detects the difference data of the image data.

Next, an example of the operation of the three-dimensional shape estimation system 200 in this embodiment will be described. The three-dimensional shape of the earthen mountain M is obtained by calculating the position and angle of the earthen mountain from the information of the distance image obtained by the RGB-D sensor. For example, for the three-dimensional shape of Tsuchiyama M, the position and angle of Tsuchiyama are calculated by searching the vertices by comparing the height coordinates with the adjacent points directly from the point cloud which is a set of three-dimensional points corresponding to one pixel of the distance image. It can also be obtained by doing. In the following, an example of estimating the three-dimensional shape of Tsuchiyama M by dividing the image data of Tsuchiyama M into a plurality of image regions on a horizontal plane and analyzing each divided image region will be described.

FIG. 4 is a flowchart showing an example of the operation of the three-dimensional shape estimation system 200 in the present embodiment. In the following description, as an example, a case where DEM data is created based on the depth image data and the DEM data is analyzed to estimate the shape of the Tsuchiyama M will be described.

In step S110, the acquisition unit 215 acquires depth image data. That is, the acquisition unit 215 acquires the depth image data obtained by capturing the image of Tsuchiyama M. The timing of depth image data acquisition by the acquisition unit 215 is arbitrary.

5 (a) to 5 (d) show perspective views of Tsuchiyama M imaged by the external sensor 206, which is the basis of the depth image data. 5 (a) to 5 (d) show the process of increasing the volume of the earthen hill M on the excavated ground G in the work room 2. The volume of the earthen hill M increases with the passage of time in the same place in the work room 2 in the order of FIGS. 5 (a) to 5 (d) due to the accumulation of earth and sand excavated by the caisson excavator 100. As the volume of Tsuchiyama M increases, it expands in the xy direction and the height in the z direction increases.

For example, as shown in FIG. 5A, when the caisson excavator 100 excavates a part of the excavated ground G, the earthen hill M (1) is formed on the excavated ground G. After that, it grows in the order of Tsuchiyama M (2) shown in FIG. 5 (b), Tsuchiyama M (3) shown in FIG. 5 (c), and Tsuchiyama M (4) shown in FIG. 5 (d), and has a mountain-shaped shape. Volume increases as shown.

In step S120, the conversion unit 216 converts the image data of Tsuchiyama M from the depth image data to the three-dimensional data (DEM data). The conversion unit 216 converts the depth image data into DEM data divided into a plurality of image regions in the horizontal plane H (xy plane) based on the image data of Tsuchiyama M captured by the outside world sensor 206.

FIG. 6 shows an example of an image region divided into a plurality of parts. The image data of the earthen hill M and the excavated ground G included in the range of imaging by the outside world sensor 206 is divided into a plurality of regions. In FIG. 6, for convenience of explanation, the illustration of the earthen mountain M is omitted, and only the area corresponding to the excavated ground G is shown. The image data is divided into, for example, m along the x direction and n along the y direction, and is composed of m rows and n columns of grid-like regions in the horizontal plane H. The number of divisions along the x-direction and the y-direction is arbitrary.

In step S130, the measuring unit 217 measures the relative position between the caisson excavator 100 and the target Tsuchiyama M. That is, in step S130, the measurement unit 217 describes each of the plurality of image regions R (1, 1), R (2, 1), ... R (m, n) divided into m rows and n columns. The relative position between the caisson excavator 100 and the object is measured based on the height information corresponding to the acquired values in each of the x, y, and z directions.

FIG. 7 shows DEM data including height information (height information) of each image area R (1, 1), R (2, 1), ... R (m, n). Here, as an example, an example in which an image region of 13 sections is included in each of the x-direction and the y-direction will be described. The DEM data is formed from the point cloud data of the target Tsuchiyama M. The point cloud data is a set of point data having three-dimensional spatial information consisting of plane coordinates and height information (values in each of x, y, and z directions). The DEM data contains height information for each compartment defined by a mesh at regular intervals. Further, the value of the z coordinate of the center point of each mesh is taken as the height of the mesh. The height of each mesh may be the place where the z-coordinate value is the largest.

FIG. 7 shows an example of height information of each section partitioned by a mesh. The DEM data is composed of x, y, and z coordinate values, but here, only the z coordinate value indicating the height is shown. In this embodiment, the height of the excavated ground G, that is, the value of the z coordinate is 0. The area indicated by hatching in FIG. 7 corresponds to Tsuchiyama M, and in particular, the image area R (8, 7) indicated by dots is an area in which a vertex position exists.

In step S140, the Tsuchiyama recognition unit 218 estimates the three-dimensional shape of the Tsuchiyama M. That is, the Tsuchiyama recognition unit 218 is based on the position of the apex P of the Tsuchiyama M, the four angles of repose θ1 to θ4, and the position of the boundary L from the position information based on the relative position between the caisson shovel 100 and the Tsuchiyama M. Estimate the angle of repose, spread, etc.

The position of the apex P of the earthen mountain M is the position where the height (value of the z coordinate) of the earthen mountain M shows the largest value. Specifically, among the grid-like regions in the DEM data of FIG. 7, the center position of the region showing the largest value in the z coordinate is defined as the position of the apex P of the earthen mountain M. In the DEM data of FIG. 7, 10 is the maximum value as the value of the z coordinate, and the grid-like region showing this 10 corresponds to the image region R (8, 7). Therefore, the central position of the image region R (8, 7) showing the z-coordinate value of 10 is defined as the position of the apex P of the earthen mountain M. Tsuchiyama M has a shape that spreads radially around the apex P.

The position of the apex P may be a position other than the center position of the region divided in a grid pattern. Further, when there are a plurality of regions showing the maximum value of the z coordinate in the grid-like region in the DEM data, if there are two regions showing the maximum value, the center positions of the two regions are connected to each other. The midpoint of the line segment may be defined as the position of the apex P of the earthen mountain M, and if there are three or more areas showing the maximum value, the center of gravity of the polygon connecting the center positions of the three or more areas is the earthen mountain. It may be defined as the position of the vertex P of M, but is not limited to this.

The boundary L is a virtual line defined for convenience to indicate the boundary between the earthen mountain M and the excavated ground G. Specifically, when the mesh-like region having the smallest z-coordinate value is continuous in the process of separating from the vertex P, the boundary L is the region on the side of the continuous region that is more separated from the vertex P (hereinafter, "reference"). It is a figure showing the approximate shape of the earthen hill M in the xy plane obtained by connecting the center positions of the regions).

Of the DEM data in FIG. 7, 0 at the location corresponding to the excavated ground G is the minimum as the value of the z coordinate. For example, as shown in FIG. 8, when the value of the z coordinate is separated from the position of the apex P toward the reference region indicating 0 along the x direction, the value of the z coordinate is in the image region R (8, 7). From height 10, image area R (12, 7) and the height of the image area R (13, 7) are reduced to 0. Since the reference region is the image region R (13, 7), the center position of the image region R (13, 7) is adopted as the reference of the boundary L. Similarly, the center position of the reference image region R is adopted as the reference of the boundary L in each direction extending radially from the apex P, and the boundary L is obtained by connecting these center positions.

The region where the value of the z coordinate first becomes 0 in the process of separating from the vertex P may be used as the reference region. Further, when three or more regions where the z-coordinate value is 0 are continuous, the region farthest from the vertex P may be used as the reference region, but the region is not limited thereto.

Further, when calculating the position and angle of the earthen hill, the position and angle of the earthen hill may be calculated by edge detection from the information of the distance image obtained by the RGB-D sensor (LiDAR: Light Detection and Ranging). .. It is also possible to calculate the position and angle of the earthen hill by searching the vertices by comparing the height coordinates with the adjacent points directly from the point cloud which is a set of three-dimensional points corresponding to one pixel of the distance image. In addition, the position and angle of the earthen hill can be calculated by approximating the point cloud of the distance image to a polygon, comparing the angles formed with the adjacent surfaces, and searching for vertices. In short, any method may be used as long as the position and angle of the earthen hill can be calculated from the information of the distance image obtained by the RGB-D sensor.

FIG. 8 shows the boundary L indicated by the vertex P and the alternate long and short dash line, and four line segments S connecting the vertex P and the boundary L. The four line segments S are provided so as to have substantially uniform angles radially around the position of the apex P. Specifically, the four line segments S are arranged clockwise at intervals of approximately 90 degrees. The line segment S1 is a line segment connecting the apex P and the boundary L in the image region R (8, 13) which is a reference region. The line segment S2 is a line segment connecting the vertex P and the boundary L in the image region R (13, 7) which is the reference region. The line segment S3 is a line segment connecting the vertex P and the boundary L in the image region R (8, 1) which is the reference region. The line segment S4 is a line segment connecting the vertex P and the boundary L in the image region R (2, 7) which is the reference region. In this way, since the four line segments S are provided so as to have substantially uniform angles radially around the apex P, the shape of the earthen hill M can be estimated accurately. In addition, each line segment S is an arbitrary point in the mesh-like region where the vertex P is located, and each mesh-like image region R (8, 13), image region R (13, 7), image region R (8, 1), any point in the image area R (2, 7) may be connected.

The number of line segments S connecting the apex P and the boundary L may be a number other than 4 (2, 3, 5, 6 ... n). As the number of line segments S increases, more data on the angle of repose can be obtained, so that the accuracy of estimating the shape of the earthen mountain M can be improved.

FIG. 9 shows the four angles of repose θ of Tsuchiyama M. The angle of repose θ can be calculated from the position of the apex P and the position of the boundary L. The angle of repose θ1 is an angle formed by the line segment S1 connecting the apex P and the boundary L in the image region R (8, 13) which is the reference region and the horizontal plane H. The angle of repose θ2 is an angle formed by the line segment S2 connecting the apex P and the boundary L in the image region R (13, 7) which is the reference region and the horizontal plane H. The angle of repose θ3 is an angle formed by the line segment S3 connecting the apex P and the boundary L in the image region R (8, 1) which is the reference region and the horizontal plane H. The angle of repose θ4 is an angle formed by the line segment S4 connecting the apex P and the boundary L in the image region R (2, 7) which is the reference region and the horizontal plane H.

The Tsuchiyama recognition unit 218 is based on the difference data detected by the difference detection unit 219 that detects the difference data of the continuous image data among the plurality of image data of the Tsuchiyama M imaged continuously in time. The three-dimensional shape of Tsuchiyama M may be estimated. For example, when the image data of Tsuchiyama M shown in FIGS. 5 (c) and 5 (d) are the first image data and the second image data captured continuously in time, the difference detection unit 219 The difference between the first image data and the second image data is detected. Using this difference, it is possible to estimate the three-dimensional shape of Tsuchiyama M, which has not been formed at this time.

According to the present invention having the above-described configuration, the three-dimensional shape of the earthen mountain M formed in the working room 2 is estimated based on the position of the apex P of the earthen mountain M in the working room 2 and the angle of repose θ. Can be done. Further, the plurality of line segments S are provided on the horizontal plane H so as to have substantially uniform angles radially around the apex P of the earthen mountain M. Therefore, the three-dimensional shape of Tsuchiyama M can be estimated with high accuracy. Further, the three-dimensional shape of Tsuchiyama M is estimated based on the image data captured in real time by the external world sensor 206. Therefore, the latest three-dimensional shape of Tsuchiyama M can be estimated. The three-dimensional shape of Tsuchiyama M is estimated based on the difference data detected by the difference detection unit 219 that detects the difference data of the continuous image data among the plurality of image data of Tsuchiyama M imaged continuously in time. Will be done. This makes it possible to estimate the three-dimensional shape of the earthen mountain M that has not been formed at this time.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and variations thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

1 Caisson 2 Work room 3 Air supply path 4 Travel rail 11 Automatic sediment loading device 12 Remote control device 13 Ground remote control room 100 Excavator (caisson excavator)
110 Vehicle 130 Boom 150 Bucket attachment 200 Three-dimensional shape estimation system 206 External world sensor 214 Image segmentation unit 215 Acquisition unit 216 Conversion unit 217 Measurement unit 218 Tsuchiyama recognition unit 219 Difference detection unit G Excavation ground H Horizontal plane L Boundary M Tsuchiyama P Vertex R Image area S line segment θ angle of repose

Claims (4)

In the 3D shape estimation system that estimates the 3D shape of the earthen hill existing in the work room used in the pneumatic caisson method.
A measuring means for measuring the relative position between the excavator and an object located around the excavator, and
Acquisition of the earthen mountain recognition means for recognizing the apex position and the angle of repose of the earthen mountain and the depth image data of the earthen mountain from the position information based on the relative position between the excavator and the object measured by the measuring means. Equipped with means,
The measuring means measures the relative position between the shovel and the object based on the depth image data acquired by the acquiring means, and measures the relative position between the shovel and the object.
Further equipped with a conversion means for converting the above depth image data into three-dimensional polygon data,
The three-dimensional shape is characterized in that the measuring means measures the apex position and the angle of repose of the earthen hill based on the angle formed by the adjacent surface of the polygon included in the three-dimensional polygon data converted by the conversion means. Estimating system. The tertiary according to claim 1, wherein the earthen hill recognition means recognizes the angle of repose of the earthen hill by scanning peripheral position information so as to be radially uniform angles around the apex position of the earthen hill. Original shape estimation system. The measuring means measures the object whose shape changes in the work room in real time.
The three-dimensional shape estimation system according to claim 1 or 2, wherein the earthen hill recognition means estimates the three-dimensional shape of the earthen hill based on the position information measured in real time by the measuring means. Of the above position information measured continuously in time by the above measurement means, the difference between the first measurement data and the second measurement data measured after the measurement of the first measurement data is detected. Further equipped with detection means,
The three-dimensional shape according to any one of claims 1 to 3, wherein the earthen hill recognition means estimates the three-dimensional shape of the earthen hill based on the difference data detected by the difference detecting means. Estimating system.

Images